Azure Data engineer
Role summary
Seeking an experienced Data Engineer with 3-5 years of hands-on experience in Azure data platforms. The role involves designing, building, and maintaining scalable data pipelines using Azure Databricks, PySpark, Scala, and Delta Lake. Responsibilities include orchestrating workflows with Azure Data Factory, enabling analytics with Azure Synapse Analytics, and working with Azure SQL. Experience with ADLS Gen1/Gen2, Event Hub, Service Bus, and Cosmos DB is expected. The candidate will also implement monitoring with Azure Monitor and secure data access with Azure Key Vault, focusing on performance, cost optimization, and troubleshooting in an enterprise environment.
Job Role - Data Engineer
Primary Skills:
Azure Data Platforms, Azure Databricks, Pyspark, Scala, Azure SQL
Overview:
Data Engineer with strong hands-on experience in Azure data platforms to design, build, and maintain scalable, reliable, and high-performance data pipelines in an enterprise environment.
͏Roles & Responsibilities
- Expertise in Azure Databricks, leveraging PySpark and Scala for large-scale distributed data processing, along with implementing Delta Lake to enable efficient storage, data versioning, and ACID-compliant operations.
- Experience in designing and implementing structured data pipelines using the Medallion architecture (Bronze, Silver, Gold layers) is essential.
- The role involves orchestrating end-to-end data workflows using Azure Data Factory (ADF) and enabling advanced analytics and integration using Azure Synapse Analytics.
- Strong proficiency in SQL, including working with Azure SQL Hyperscale, is required.
- The candidate should be experienced in managing and optimizing data storage in ADLS Gen1 and Gen2 and integrating real-time and event-driven data streams using Event Hub and Service Bus.
- Familiarity with NoSQL databases such as Cosmos DB is expected.
- Additionally, the role requires implementing robust monitoring and observability using Azure Monitor and Application Insights, along with ensuring secure data access and secret management using Azure Key Vault.
- The candidate should demonstrate the ability to optimize data pipelines for performance and cost, troubleshoot complex production issues, and consistently deliver high-quality, scalable data solutions in a distributed and enterprise-scale ecosystem.
Mandatory Skills: Data Engineering Full Stack .
Experience: 3-5 Years .
