Senior Data Engineer - Data Platform

• Design, build, and operate end-to-end data pipelines (batch and near-real-time) that ingest, transform, and deliver data from diverse sources into the enterprise data platform.
• Develop and maintain curated data models, marts, and shared datasets in Snowflake andPostgreSQL that meet performance, quality, and access-control requirements for multiple internal customers.
• Implement data quality frameworks including automated validation, schema enforcement, reconciliation checks, duplicate detection, and exception reporting with clear audit trails.
• Partner with domain teams (e.g., Asset Management, Finance, Operations) to understand data needs, define contracts and SLAs, and deliver platform capabilities that reduce bespoke engineering and manual effort.
• Build parameterized, reusable pipeline components and templates that standardize ingestion patterns, transformations, and deployment across the platform.
• Establish and maintain data lineage, metadata, and documentation so stakeholders can trace data from source to consumption with confidence.
• Collaborate with IT and security to implement role-based access controls, data masking, encryption, and compliance requirements across platform resources.
• Own pipeline orchestration, scheduling, dependency management, and alerting using workflow tools (e.g., Airflow) to ensure reliable, recoverable execution.
• Improve platform observability through logging, metrics, SLA monitoring, and incident response practices that minimize downtime and data freshness gaps.
• Support CI/CD and infrastructure-as-code practices for data platform assets, including version control, automated testing, and safe promotion across environments.
• Evaluate and integrate new platform technologies and patterns (e.g., streaming, CDC, data mesh principles) where they improve scalability, cost efficiency, or time-to-value.
• Mentor junior engineers and contribute to platform standards, code review practices, and technical design documentation.

• Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related quantitative field.

• 5+ years of experience in data engineering or platform engineering, preferably in a financial services or regulated industry (e.g., asset management, banking, insurance, fintech).
• Strong SQL and Python skills, with a track record of building production-quality data
• pipelines, transformations, and validation frameworks.
• Proficient at using AI-assisted development tools to design, build, and iterate on data pipelines while maintaining code quality, security, and governance standards.
• Hands-on experience with Snowflake and PostgreSQL, including performance tuning, cost optimization, and secure multi-tenant data access patterns.
• Experience with pipeline orchestration and workflow management tools (e.g., Apache, Airflow, Dagster, or equivalent).
• Proficiency with Git, code review, and CI/CD practices for data platform development.
• Experience designing dimensional or domain-oriented data models and delivering curated datasets for analytics and operational use cases.
• Familiarity with data quality, lineage, and governance tooling and practices (preferred).
• Experience with cloud data services (e.g., AWS, Azure, or GCP) and infrastructure-as-code (e.g., Terraform) is strongly preferred.
• Exposure to streaming or change-data-capture (CDC) patterns and event-driven architectures is a plus.
• Understanding of financial data domains (e.g., portfolio, investor reporting, accounting) is helpful but not required; curiosity and ability to partner with domain experts is essential.
• Strong communication and collaboration skills; ability to translate ambiguous requirements into well-scoped technical designs and clear status reporting.
• Familiarity with containerization (e.g., Docker/Kubernetes) and API/integration patterns for data services is a plus.

Originally posted on Himalayas