Lead Data Engineer

Straive
is a global leader in enterprise-grade data analytics and AI solutions, committed to empowering businesses across various industries with cutting-edge technology and expert insights. Backed by EQT, a top private equity firm, we are uniquely positioned to drive innovation through significant investments and an entrepreneurial spirit.

Our core focus is on delivering advanced Data Analytics & AI Solutions. By combining sophisticated technology with subject matter expertise, we deliver material impact on our clients' topline and streamline their operations. We specialize in providing tailored solutions across financial services, CPG, legal, pharma, life sciences, retail, energy and logistics, helping them build robust data analytics and AI capabilities.

With a client base spanning 30 countries, Straive's strategically located global teams offer localized expertise with a worldwide perspective.

Join Straive to be part of a dynamic team at the forefront of data analytics and AI innovation. Here, you'll have the opportunity to contribute to transformative projects, supported by significant investments and an entrepreneurial drive fueled by our partnership with EQT.

Job Title: Lead Data Engineer

Location: New York, NY (Hybrid)

Job Type: Full-Time

Industry: Media / Entertainment (Nielsen Audience & Measurement Data) (Good to have)

About the Role

We are looking for an experienced Lead Data Engineer with deep expertise in Databricks, PySpark, and AWS. The ideal candidate will have a strong understanding of media measurement datasets (Nielsen) and a proven ability to lead multiple concurrent projects across onsite and offshore teams. This role requires a senior technical leader to define architecture, manage delivery, and mentor teams within the fast-paced New York media industry.

Key Responsibilities

• Technical Leadership: Architect and deliver enterprise-scale data platforms using Databricks, PySpark, Delta Lake, and AWS. Lead the development of media analytics pipelines, including the ingestion and modeling of Nielsen data.

• Project & Delivery Management: Manage scope, timelines, and risks for multiple simultaneous projects. Collaborate with New York-based stakeholders and drive sprint delivery.

• Onsite–Offshore Coordination: Act as the primary technical lead onsite, coordinating with offshore teams to ensure alignment, quality, and continuity.

• Team Mentorship: Mentor junior and mid-level engineers through code reviews and architectural guidance.

Required Skills

• Databricks: Delta Lake, Workflows, Unity Catalog, Notebooks, and SQL.

• Data Processing: Strong PySpark and distributed computing experience.

• AWS: S3, EMR, Glue, Lambda, Step Functions, Kinesis, and IAM.

• Domain Expertise: Proven experience handling Nielsen datasets (National/Local TV ratings, Digital Ad ratings, Panels).

• Leadership: Experience leading large data engineering programs and managing senior stakeholders.

Good to Have

• Knowledge of Snowflake, Airflow, or Dagster.

• Infrastructure as Code (Terraform / CloudFormation).

• Exposure to ad-tech ecosystems (Comscore, Kantar, IAS, Moat) or ML feature pipelines.

Soft Skills

• Strong decision-making and ownership mindset.

• Exceptional communication skills for interacting with senior business stakeholders.

• Strategic, problem-solving, and delivery-focused approach.

This job description is not intended to cover or contain a comprehensive listing of all responsibilities, duties, or activities that are required. Responsibilities, duties, and/or activities may change, or new ones may be added at any time with or without notice.

If you are a motivated professional with a passion for delivering impactful solutions, we’d love to hear from you. Apply today to be part of a dynamic and forward-thinking team at Straive.

“Straive is an Equal Opportunity Employer. Our policy is clear: there shall be no discrimination based on age, disability, sex, race, religion or belief, gender reassignment, marriage/civil partnership, pregnancy/maternity, or sexual orientation.

We are an inclusive organization and actively promote equality of opportunity for all with the right mix of talent, skills and potential. We welcome all applications from a wide range of candidates. Selection for roles will be based on individual merit alone.”