Sr. Data Engineer (Databricks/Spark)
Role summary
A Sr. Data Engineer with expertise in Databricks and Apache Spark is sought for a financial client to design, develop, and optimize large-scale data pipelines. This role involves managing distributed computing environments, ensuring data quality, performance, and scalability. The engineer will collaborate with technical and business stakeholders, contribute to solution design, and troubleshoot data pipeline issues. Familiarity with enterprise banking or financial services environments is preferred, along with strong communication skills for a fast-paced setting.
Are you experienced in building scalable data solutions with Databricks? Apply Now!
Working with one of our top financial clients, this role calls for a Sr. Data Engineer (Databricks/Spark). This position involves designing, developing, and optimizing large-scale data pipelines using Databricks and Apache Spark, with a focus on delivering robust solutions within the banking or financial services domain. The successful candidate will communicate effectively with both technical teams and business stakeholders, contributing to high-performance data environments in a fast-paced setting.
Responsibilities
- Design, develop, and optimize data pipelines utilizing Databricks and Apache Spark
- Manage and process large-scale datasets in a distributed computing environment
- Collaborate with cross-functional teams including business and technical stakeholders
- Participate in solution design discussions and provide insightful technical recommendations
- Ensure data quality, performance optimization, and solution scalability
- Troubleshoot and resolve data pipeline issues efficiently
- Contribute to documentation and best practices for data engineering processes
Desired Skill-Set
- Proven hands-on experience with Databricks and Apache Spark development
- Familiarity with enterprise environments, preferably in banking or financial services
- Strong understanding of distributed computing and data architecture
- Excellent verbal and written communication skills
- Ability to work independently in a dynamic, fast-paced environment
- Experience working on large-scale data projects with multiple stakeholders
Nice to Have
- Experience with Kafka or other messaging systems
- Knowledge of Python programming
- Previous experience in data solution architecture or leadership roles
- Familiarity with cloud platforms and related data tools
*BeachHead is an equal opportunity agency and employer. We advocate for our candidates and welcome applicants regardless of race, color, religion, national origin, sex, age, or physical or mental disability. BeachHead or our clients may use technology-enabled tools, including automation and artificial intelligence (AI), to support parts of the recruitment process such as resume screening, application management, and candidate matching. These tools assist our recruiters and our clients, and do not replace human decision-making. This job posting represents a current or anticipated vacancy. The position may be filled at any time, and the posting may be removed without notice once the role has been filled.*
Privacy Policy