Senior Data Engineer

### Who you are
- 5+ years of hands-on experience in data engineering, with expertise in distributed data processing and big data frameworks (e.g., Apache Spark, Apache Iceberg, Trino, Apache Airflow, dbt, Dagster)
- Advanced programming skills in Scala or Python for data transformation and automation
- Experience with real-time data streaming technologies such as Apache Flink, Spark Streaming, or Kafka
- Strong experience in performance tuning for Spark and optimizing large-scale data workflows
- Proficiency in SQL and database management, with hands-on experience in Massively Parallel Processing (MPP) databases such as Amazon Redshift, Snowflake, Teradata
- Familiarity with cloud-based data services (AWS, RDS, DynamoDB) and containerized infrastructure (EKS, Docker, Kubernetes)
- Hands-on experience integrating DevOps and CI/CD practices in data engineering using GitHub Actions, Jenkins, or Terraform
- Proven ability to build monitoring, alerting, and observability tools for data pipelines to ensure high availability and reliability
- Experience in data mapping, validation, and testing frameworks to ensure accuracy and consistency
- Exposure in machine learning/deep learning using PyTorch, Tensorflow, or Keras (PyTorch preferred)
- Exposure to machine learning workflows and familiarity with MLOps tools for model deployment and lifecycle management
- Self-starter with strong problem-solving skills who thrives in a fast-paced, agile environment
- Experience working with healthcare data standards (e.g., HL7, FHIR) and regulatory compliance
- Familiarity with reporting and visualization tools such as Superset, Grafana, Tableau
- Understanding of data security, compliance, and governance frameworks (e.g., HIPAA, GDPR, SOC 2)

### What the job involves
- We are seeking an experienced Senior Data Engineer to join our team and play a crucial role in designing, implementing, and optimizing scalable data solutions
- In this role, you will work with modern data technologies, distributed systems, and cloud platforms to build efficient, high-performance data pipelines that support analytics, AI/ML, and business intelligence initiatives
- Design, develop, and implement robust ETL/ELT data pipelines for large-scale data ingestion, transformation, and storage
- Ensure data quality, integrity, and governance by implementing validation techniques, data monitoring, and automated testing
- Collaborate with cross-functional teams, including data scientists, analysts, platform engineers, and business stakeholders, to develop scalable and reusable data solutions
- Automate deployments and testing using CI/CD pipelines with Git, Terraform, GitHub Actions, or Jenkins
- Design and build custom data tools and abstractions to support analytics, machine learning, and real-time data processing
- Work with DevOps and platform teams to establish efficient deployment and monitoring processes for internal and external data products
- Develop and implement alerting, monitoring, and observability frameworks for data pipelines to ensure reliability and proactive issue resolution
- Contribute to the data architecture and strategy, driving improvements in scalability, performance, and cost optimization
- Stay up to date with emerging technologies and industry best practices to continuously enhance data engineering capabilities

### Benefits
- Remote first culture: work from anywhere in the US, our Boston headquarters, or a combination of both
- Flexible PTO: Take what you need PTO policy with no accruals
- Medical, dental & vision: 100% of employee premiums paid by 1upHealth
- Paid leave: 16 weeks of fully paid leave plus a 4 week phased return to work schedule for birthday parents, and 6 weeks of fully paid parental leave for non-birthing parents
- Wellness benefit: Get reimbursed for up to $1,400 annually for expenses that contribute to your total wellbeing
- Anniversary bonus: At your 2 year anniversary receive $2,000, at your 5 years anniversary $5,000 towards a vacation of your choice
- Home office stipend: $400 to elevate your home office