Data Engineer (Banking/Capital Markets)
Role summary
We are seeking a skilled Data Engineer for a Banking/Capital Markets role. This position involves designing, building, and maintaining scalable data pipelines and warehousing solutions using Python, PySpark, SQL, and Snowflake. The engineer will deploy and monitor applications on OpenShift, ensuring data quality and governance. Collaboration with data scientists, analysts, and business stakeholders is key. Responsibilities include automating workflows, troubleshooting performance, and documenting architecture. A Bachelor's or Master's degree in a related field and proven experience are required.
We are seeking a highly skilled Data Engineer to join our dynamic team. The ideal candidate will have a strong background in data processing, cloud platforms, and modern data warehousing solutions. This role involves designing, building, and maintaining scalable data pipelines and warehousing solutions to support analytics and business intelligence initiatives.
Key Responsibilities:
- Design, develop, and maintain robust data pipelines using Python and PySpark to process large datasets efficiently.
- Develop and optimize SQL queries for data extraction, transformation, and loading (ETL) processes.
- Implement and manage data warehousing solutions using Snowflake.
- Deploy and monitor data applications and pipelines on OpenShift container orchestration platform.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
- Ensure data quality, data security, and compliance with data governance standards.
- Automate data workflows to improve efficiency and scalability.
- Monitor and troubleshoot performance issues in data pipelines and storage systems.
- Document data architecture, pipelines, and processes clearly and comprehensively.
- Stay updated with emerging technologies and industry trends related to data engineering.
Qualifications:
- Bachelor’s or Master’s degree in Computer Science, Data Science, or a related field.
- Proven experience as a Data Engineer or in a similar role.
- Strong proficiency in Python, with hands-on experience using PySpark.
- Extensive experience working with SQL and relational databases.
- Hands-on experience with Snowflake data warehouse platform.
- Familiarity with container orchestration and cloud deployment using OpenShift.
- Knowledge of data modeling, ETL/ELT processes, and data architecture best practices.
- Experience with version control systems such as Git.
- Strong problem-solving skills and attention to detail.
- Excellent communication and teamwork skills.
Preferred Skills:
- Experience with other cloud platforms (AWS, Azure, GCP).
- Knowledge of DevOps practices and CI/CD pipelines.
- Experience with Kafka, Hadoop, or other big data technologies.
- Familiarity with Python libraries for data analysis and processing (e.g., pandas, numpy).