Senior Azure Data Engineer

Job title:
Senior Azure Data Engineer

Location:
Texas (Onsite)

Employment Type:
Full-time (W2 only, No C2C)

Experience:
10+ Years

Job Summary:
We are seeking an experienced
Azure Data Engineer with 10+ years of experience
in designing, building, and optimizing large-scale data processing systems and data pipelines on the
Microsoft Azure cloud platform
. The ideal candidate will have strong expertise in
data engineering, ETL/ELT development, big data processing, data warehousing, and cloud-based data solutions
. The candidate will work closely with
data architects, analysts, and business stakeholders
to design and implement scalable data solutions that support enterprise analytics and reporting.

Key Responsibilities:

- Design, develop, and maintain
scalable data pipelines and ETL/ELT workflows
using
Azure Data Factory (ADF), Azure Databricks, and Azure Synapse Analytics
to process and transform large volumes of structured and unstructured data.
- Build and optimize
big data processing solutions
using
Apache Spark, PySpark, Scala, or SQL within Azure Databricks
, ensuring high performance and efficient data transformation.
- Design and implement
modern data warehouse and lakehouse architectures
using
Azure Synapse Analytics, Azure Data Lake Storage (ADLS Gen2), and Azure SQL Database
to support enterprise-level analytics and reporting.
- Develop and maintain
data ingestion frameworks
to integrate data from multiple sources such as
databases, APIs, flat files, streaming platforms, and third-party applications
into Azure-based data platforms.
- Implement
data modeling techniques
including
star schema, snowflake schema, and dimensional modeling
to support business intelligence and analytical workloads.
- Develop
SQL-based data transformation and data quality validation processes
using
T-SQL, Spark SQL, and stored procedures
to ensure accuracy, consistency, and reliability of data.
- Integrate data engineering solutions with
business intelligence and analytics tools
such as
Power BI, Tableau, or Azure Analysis Services
for reporting and data visualization.
- Implement
real-time and streaming data processing
solutions using
Azure Event Hub, Azure Stream Analytics, or Apache Kafka
to support near real-time analytics.
- Optimize
data storage, partitioning, indexing, and query performance
across Azure-based data platforms to ensure efficient data retrieval and processing.
- Implement
data governance, security, and compliance
using
Azure Active Directory (AAD), Role-Based Access Control (RBAC), data encryption, and data masking techniques
.
- Automate deployment and data pipeline orchestration using
CI/CD pipelines
with
Azure DevOps, GitHub Actions, or Jenkins
, ensuring reliable and consistent deployment processes.
- Monitor and troubleshoot
data pipelines, workflows, and cloud resources
using
Azure Monitor, Log Analytics, and Application Insights
to ensure system reliability and performance.
- Collaborate with
data architects, DevOps engineers, and cross-functional Agile teams
to design scalable data engineering solutions aligned with enterprise data strategies.
- Mentor junior data engineers and contribute to
best practices in data engineering, cloud architecture, and performance optimization
.

Technical Skills:

- Cloud Platform:
Microsoft Azure
- Azure Data Services:
Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage (ADLS Gen2), Azure SQL Database
- Big Data Technologies:
Apache Spark, PySpark, Scala, Spark SQL
- Programming Languages:
Python, SQL, Scala
- Data Warehousing:
Azure Synapse Analytics, Azure SQL Data Warehouse, Snowflake Concepts, Dimensional Modeling
- ETL/ELT Tools:
Azure Data Factory, Databricks Workflows
- Streaming Technologies:
Azure Event Hub, Azure Stream Analytics, Apache Kafka
- Databases:
SQL Server, Azure SQL Database, PostgreSQL, MySQL, NoSQL
- Data Visualization Tools:
Power BI, Tableau
- Version Control:
Git, GitHub, Bitbucket, GitLab
- CI/CD Tools:
Azure DevOps, Jenkins, GitHub Actions
- Containerization:
Docker, Kubernetes
- Operating Systems:
Windows, Linux
- Methodologies:
Agile, Scrum, DevOps, DataOps