Platform Engineer Lead
Role summary
We are seeking a highly experienced Platform Engineer Lead with strong expertise in Databricks to lead and drive Advanced Data Hub (ADH) initiatives. The ideal candidate will be responsible for designing, building, and scaling enterprise-grade data platforms, enabling data engineering, analytics, and AI workloads across the organization. This role requires deep technical expertise in Databricks, strong leadership skills, and experience in building modern data platforms on cloud environments.
Job Title: Platform Engineer Lead
Location:
Remote
Job Summary
We are seeking a highly experienced
Platform Engineer Lead
with strong expertise in
Databricks
to lead and drive
Advanced Data Hub (ADH) initiatives
. The ideal candidate will be responsible for designing, building, and scaling enterprise-grade data platforms, enabling data engineering, analytics, and AI workloads across the organization.
This role requires deep technical expertise in Databricks, strong leadership skills, and experience in building modern data platforms on cloud environments.
Key Responsibilities
- Lead the design and development of scalable
data platform architecture
using Databricks and cloud technologies
- Drive
ADH (Advanced Data Hub)
initiatives including data ingestion, transformation, governance, and consumption layers
- Architect and implement
Lakehouse architecture
using Databricks and Delta Lake
- Lead and mentor a team of data engineers and platform engineers
- Define best practices for
data engineering, performance optimization, and cost management
- Collaborate with business stakeholders, architects, and data teams to align platform capabilities with business goals
- Implement
data governance, security, and compliance frameworks
(Unity Catalog, RBAC, etc.)
- Oversee
CI/CD pipelines, automation, and infrastructure as code
for platform deployments
- Optimize
Spark jobs and Databricks workloads
for performance and scalability
- Enable
real-time and batch data processing frameworks
- Evaluate and integrate new tools and technologies to enhance platform capabilities
Required Skills
- Strong hands-on experience with
Databricks (Workspace, Jobs, Workflows, Delta Lake)
- Expertise in
Apache Spark (PySpark / Spark SQL)
- Experience in building
data platforms / data lakes / lakehouse architectures
- Strong knowledge of
cloud platforms (Azure / AWS / GCP)
- Experience with
data ingestion frameworks (ADF, Kafka, Event Hubs, etc.)
- Deep understanding of
data modeling and data warehouse concepts
- Experience with
Unity Catalog / Data Governance frameworks
- Proficiency in
Python and SQL
- Strong experience with
CI/CD tools (Azure DevOps, Jenkins, Terraform, etc.)
- Knowledge of
streaming technologies (Kafka, Structured Streaming)
