RiseMe
LoginSign Up
Haystack logo
Haystack Verified
Software, Developer Tools, Analytics

Data Engineer

Chicago, Illinois, United StatesRemoteFull Time$112,000–$140,000 /yrPosted 2 months ago

Is this role right for you?

Upload your resume and get a skill-by-skill breakdown — see exactly where you match, where you're close, and what to highlight. Not a mystery percentage.

Get a tailored resume highlighting what this role needs.

Role summary

A leading specialty insurance powerhouse seeks a Lead Data Engineer to architect and scale modern data environments using Databricks and Azure. This high-impact leadership role involves mentoring a growing team, driving technical excellence in big data pipelines, advanced analytics, and enterprise data governance. Responsibilities include designing and implementing scalable data pipelines, acting as a technical SME for big data architecture, ensuring data integrity, championing CI/CD best practices, and collaborating with stakeholders to deliver BI solutions. The role requires significant experience in Databricks, Azure Data Stack, SQL Server, and strong leadership capabilities.

Lead Data Engineer

We're working with a leading specialty insurance powerhouse on this exciting opportunity.

Step into a high-impact leadership role where you will architect and scale modern data environments using Databricks and Azure. This is a unique chance to mentor a growing team while driving technical excellence across big data pipelines, advanced analytics, and enterprise-grade data governance.

The Role

• Lead the design and implementation of scalable data pipelines using Azure Data Factory and Databricks notebooks for complex batch and real-time processing.

• Act as a technical SME for big data architecture, managing Delta Parquet files in Azure Data Lake Storage to optimize query performance and cost-efficiency.

• Drive data integrity and governance by integrating Profisee MDM and Azure Purview for seamless data cataloging and lineage tracking.

• Champion automation and CI/CD best practices using Azure DevOps to streamline deployment and optimize SQL Server database performance.

• Collaborate with high-level stakeholders to translate business objectives into robust BI solutions via Tableau and advanced SSRS reporting.

What You'll Need

• At least 6+ years of heavy-hitting experience specifically with Databricks engineering and big data analytics.

• 10+ years of professional Data Engineering experience (or 8+ with a Master’s degree) in complex enterprise environments.

• Expertise in the Azure Data Stack, including Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), and Azure Purview.

• Mastery of SQL Server Management, SSIS, and Change Data Capture (CDC) processes for real-time data synchronization.

• Strong leadership profile with a proven track record of mentoring junior engineers and managing large-scale SQL and Tableau projects.

What's On Offer

• Competitive salary range of $112,000 - $140,000 with eligibility for performance-based bonuses.

• Remote-friendly flexibility coupled with a comprehensive "Total Rewards" benefits package.

• Robust support for physical and mental well-being, including paid parental leave and mental health services.

• A dynamic, inclusive culture that prioritizes professional growth, mentorship, and cutting-edge tech adoption.

Apply via Haystack today!

Ready to apply?
You'll be redirected to Haystack's application page.

Similar roles