Robot Insight Technologies Verified
Robotics, Artificial Intelligence, Industrial Automation
Senior Databricks Platform Engineer- W2
United StatesRemoteContractSeniorPosted 1 month agoVisa sponsorship available
Compensation estimateAI
See base, equity, bonus, and total comp estimates for this role — free, no credit card.
Sign up to see compensation estimateSenior Databricks Platform Engineer (W2 Contract | Remote)
Only W2(No C2C/C2H/1099)
Background in Healthcare/Payments domain
We are seeking a Senior Databricks Platform Engineer with strong experience in Python/PySpark, Databricks, and AWS to support and enhance our enterprise data platform. This role involves a blend of data engineering, platform administration, and cloud optimization.
Key Responsibilities
- Design and develop scalable data pipelines using PySpark and Databricks
- Administer and optimize Databricks platform (clusters, jobs, SQL warehouses, Unity Catalog)
- Build and maintain Delta Live Tables (DLT) and streaming/batch pipelines
- Implement data ingestion using Auto Loader, CDC, and schema evolution
- Monitor and tune performance (Photon, autoscaling, query optimization)
- Manage CI/CD deployments using Databricks Asset Bundles (DABs)
- Ensure platform security, governance, and audit logging
- Work with AWS services (IAM, S3, CloudWatch, VPC, KMS)
- Collaborate with cross-functional teams and support production issues
Required Skills
- 8+ years of experience with Databricks, PySpark, and Spark
- Strong knowledge of Delta Lake, Unity Catalog, and Lakehouse architecture
- Hands-on experience with AWS (IAM, S3, networking basics)
- Experience with DLT, Auto Loader, CDC, and data pipeline design
- Solid SQL skills and experience with RDBMS (Oracle preferred)
- Familiarity with CI/CD tools and Agile methodologies
Preferred Qualifications
- Databricks Certified Data Engineer Associate
- AWS Certification (Associate level or higher)
- Experience with Scala, Spark SQL, Hive