Senior Data Engineer - Databricks

Raleigh, NJOnsiteFull TimeSeniorPosted 1 month agoVisa sponsorship available

Is this role right for you?

Upload your resume and get a skill-by-skill breakdown — see exactly where you match, where you're close, and what to highlight. Not a mystery percentage.

Get a tailored resume highlighting what this role needs.

Required skills

Databricks
PySpark
Spark SQL
Delta Live Tables
Delta Lake
Unity Catalog
SQL
AWS
Azure
GCP

Nice to have

Lakeflow Connect
LSDP/Lakehouse
Genie
Agent Bricks/RAG
Data governance
Metadata management

Role summary

DATAECONOMY, a rapidly growing Data & Analytics company, seeks a Senior/Lead Data Engineer specializing in Databricks. The role involves building scalable ETL/ELT pipelines, ingesting various data types, developing transformations, and implementing best practices for orchestration and monitoring. Key technologies include Databricks (PySpark, Spark SQL, Delta Live Tables, Workflows, Delta Lake, Unity Catalog), strong SQL, cloud environments (AWS/Azure/GCP), and data lakehouse concepts. The position requires 7-10+ years of data engineering experience and knowledge of Medallion Architecture. Nice-to-have skills include Airflow, dbt, data governance, and cost optimization.

DATAECONOMY is one of the fastest-growing Data & Analytics company with global presence. We are well-differentiated and are known for our Thought leadership, out-of-the-box products, cutting-edge solutions, accelerators, innovative use cases, and cost-effective service offerings.

We offer products and solutions in Cloud, Data Engineering, Data Governance, AI/ML, DevOps and Blockchain to large corporates across the globe. Strategic Partners with AWS, Collibra, cloudera, neo4j, DataRobot, Global IDs, tableau, MuleSoft and Talend.
Senior/Lead Data Engineer — Databricks

Rutherford, NJ/ Jersey City, NJ

Full-time

Build scalable, production-grade ETL/ELT pipelines using Databricks (PySpark, Spark SQL, Delta Live Tables, Workflows).
Ingest structured, semi-structured, and streaming data into Bronze, Silver, and Gold layers.
Develop optimized transformations, data quality rules, and reusable framework components.
Implement best practices for job orchestration, monitoring, alerting, and automation.
Hands-on experience: Spark, Delta Lake, Workflows, Unity Catalog.
Strong SQL programming and performance tuning skills.
Experience with cloud environments (AWS/Azure/GCP).
Experience with modern data lakehouse concepts and distributed systems.
Strong understanding of Lakeflow Connect, LSDP/Lakehouse, Medallion Architecture, Data Validations, Genie, and Agent Bricks/RAG use cases.
Should be able to explain these concepts using real project examples and architecture decisions.
Knowledge of medallion architecture, DLT and unity catalog within Databricks.

Requirements

Strong Python (PySpark) and SQL programming
Databricks — Spark, Delta Lake, Workflows, Unity Catalog
ETL/ELT pipeline development — Medallion Architecture (Bronze/Silver/Gold)
Delta Live Tables, Auto-Loader, Structured Streaming
Data modeling — dimensional (star/snowflake), normalization/denormalization
CI/CD, Git, job orchestration
Cloud experience — AWS, Azure, or GCP
7–10+ years in data engineering
Knowledge of medallion architecture, DLT and unity catalog within Databricks.

Nice-to-Have Skills

Lakeflow Connect, LSDP/Lakehouse, Genie, Agent Bricks/RAG use cases
Data governance, metadata management, Unity Catalog advanced features
Airflow, dbt, or similar orchestration tools
Data security, compliance, and access models
Cost optimization and performance tuning in cloud environments
Corporate/enterprise data warehousing background

Ready to apply?

You'll be redirected to DATAECONOMY's application page.