Azure Data Engineer- HPC & ML
Role summary
Akkodis is seeking a mid-level Azure Data Engineer specializing in High-Performance Computing (HPC) and Machine Learning (ML) for a 12-month contract in Oakbrook Terrace, IL. The role involves designing, building, and maintaining hybrid data infrastructure and pipelines, bridging HPC servers and Azure cloud platforms. Responsibilities include developing scalable data pipelines, creating data architectures, implementing MLOps, and optimizing data processing for analytics and ML. The position requires strong UNIX/Linux skills, SQL proficiency, Azure data services experience, and programming in Python/shell scripting. Familiarity with containerization, data processing frameworks like Spark, and networking concepts is essential.
Akkodis is seeking an Azure Data Engineer- HPC & ML with a client in
Oakbrook Terrace, IL 60181
for 12
months contract.
Title:
Azure Data Engineer- HPC & ML
Location:
Hybrid- Oakbrook Terrace, IL 60181
Duration:
12-month Contract (possible extension)
Pay Rate
: $65-$70/hr (The rate may be negotiable based on experience, education, geographic location, and other factors)
Job Description
A mid-level Data Engineer is responsible for designing, building, and maintaining the data infrastructure and pipelines that enable efficient data processing, storage, and analysis within a hybrid computing environment. This role bridges the gap between data collection and data consumption across both
high-performance computing (HPC)
servers and Azure cloud platforms, ensuring that data is accessible, reliable, and optimized for analytics, machine learning, and business intelligence purposes.
Responsibilities
- Design, develop, and implement scalable data pipelines using modern ETL/ELT frameworks and tools in hybrid environments
- Create and maintain data architectures that span high-performance computing servers and Azure cloud services
- Build robust data integration solutions to connect various data sources across on-premises and cloud systems
- Configure and optimize databases and data stores in both UNIX/Linux environments and cloud platforms
- Implement MLOps practices to support the machine learning lifecycle from development to production
- Design and implement modern data flow architectures that support both batch and real-time processing needs
- Apply performance tuning techniques for high-volume data processing in HPC environments
- Collaborate with data scientists, ML engineers, and business stakeholders to understand data requirements
- Develop and maintain documentation for data infrastructure, pipelines, and workflow
- Implement data quality monitoring and validation across hybrid environments
- Mentor junior data engineers on best practices for hybrid cloud/on-premises architectures
Required Skills
- 3-5 years of experience in data engineering or related field
- Strong UNIX/Linux administration skills and command-line proficiency
- Experience with high-performance computing environments and workload management
- Proficiency in SQL and experience with major database systems (PostgreSQL, SQL Server, Oracle, MySQL)
- Hands-on experience with Azure cloud data services (Azure Data Factory, Synapse Analytics, Azure SQL)
- Knowledge of modern data flow architectures (Lambda, Kappa, Delta) and implementation patterns
- Experience with ML flow frameworks and ML lifecycle management tools
- Programming skills in Python and shell scripting for automation and data processing
- Familiarity with containerization (Docker) and orchestration (Kubernetes)
- Experience with data processing frameworks (Spark, Databricks, Azure Batch)
- Understanding of networking concepts for data movement between on-premises and cloud
Preferred Skills
- Experience with real-time data processing and streaming technologies (Kafka, Event Hubs)
- Knowledge of infrastructure-as-code tools (Terraform, ARM templates)
- Experience with distributed file systems (HDFS, Lustre) and object storage
- Familiarity with graph databases and specialized analytics datastores
- Understanding of data governance, security, and compliance requirements across hybrid environments
- Experience with optimization techniques for large-scale data transfers between systems
- Knowledge of GPU acceleration for data processing pipelines
- Background in implementing CI/CD for data solutions
Education
- Bachelor's degree in Computer Science, Information Systems, or related technical field
- Relevant certifications in Azure, data technologies, or HPC administration
Equal Opportunity Employer/Veterans/Disabled
Benefit offerings available for our associates include medical, dental, vision, life insurance, short-term disability, additional voluntary benefits, an EAP program, commuter benefits, and a 401K plan. Our benefit offerings provide employees the flexibility to choose the type of coverage that meets their individual needs. In addition, our associates may be eligible for paid leave including Paid Sick Leave or any other paid leave required by Federal, State, or local law, as well as Holiday pay where applicable.
*Disclaimer: These benefit offerings do not apply to client-recruited jobs and jobs that are direct hires to a client.*
*To read our Candidate Privacy Information Statement, which explains how we will use your information, please visit
https://www.akkodis.com/en/privacy-policy
.*
*The Company will consider qualified applicants with arrest and conviction records in accordance with federal, state, and local laws and/or security clearance requirements, including, as applicable:*
*· The California Fair Chance Act*
*· Los Angeles City Fair Chance Ordinance*
*· Los Angeles County Fair Chance Ordinance for Employers*
*· San Francisco Fair Chance Ordinance*