Azure Data Engineer(W2 & open for sponsorship)
Role summary
The Data Engineering team is seeking a Data Engineer to build and maintain end-to-end data pipelines for analytics solutions on the Microsoft Azure Cloud and Azure Synapse Analytics platform. Key responsibilities include designing and developing analytics solutions, creating ETL/ELT processes, ensuring data quality and pipeline performance, and driving automation. The role involves partnering with IT and business teams, documenting solutions, and participating in agile development frameworks. The position requires a Bachelor's degree in a relevant field and 10-12+ years of hands-on data engineering experience, with strong skills in SQL, data modeling, data warehousing, Azure services (Synapse, ADF, ADLS), Python, and DevOps.
Hello
Data Engineer
San Antonio, TX
Locals from Texas Only
POSITION SUMMARY:
The Data Engineering team is seeking to add a Data Engineer to our data engineering team. The data engineer will be a key contributor in building and maintaining end to end data pipelines for analytics solutions/data products within the Insights & Discovery Platform based on Microsoft Azure Cloud and Azure Synapse Analytics.
KEY RESPONSIBILITIES:
• Design, develop, implement, analytics solutions/data products on Microsoft Azure Synapse Analytics Platform
• Develop ETL/ELT processes to collect and integrate data from various sources. Ensuring data quality and data pipeline performance for large volume data ingestion
• Utilize data preparation, integration, storage & analytics tools within the Azure landscape
• Drive toward automation of repeatable data prep tasks to minimize manual data management processes
• Partner with business unit IT teams to gather and document analytics solution requirements
• Document and provide solution options and recommendations for analytics data products
• Collaborate and partner with cloud service providers, data science, data governance and other IT and business teams in the process of delivering analytics solutions
• Participate in scaled agile framework for data engineering product delivery by participating in all relevant ceremonies
• Provide trouble shooting, analysis and resolutions on existing solutions for routine production support issues as well as major/minor enhancements.
• Adhere to data product delivery and change management processes for solution deployment within the landscape
EDUCATION AND EXPERIENCE:
• Bachelor’s degree in computer science, Computer Engineering, Software Engineering, Information Systems, or related field.
• Minimum
10-12+
years of solid data engineering hands-on experience in building data pipelines, data models and maintaining/supporting enterprise data warehouse solutions
• Experience in scaled agile framework for data engineering product delivery
• Microsoft Azure certifications in various analytics technologies is a plus
• Working knowledge/familiarity with Oil & Gas industry processes is a plus
SKILLS:
• Must have strong experience with SQL, data modeling, data warehouse and medallion architecture as a data engineer
• Must have strong experience building data products using Azure Data Lake Storage,
Azure Synapse Analytics
for data warehousing,
Azure Data Factory (ADF)
, Azure Spark Pools, Databricks Notebooks, CDC tools, Python, DevOps, CI/CD
• Familiarity with data integration tools (e.g. Qlik Replicate) and with data modeling tools (e.g. Erwin or ER Studio) is a plus
• Familiarity with data lake/warehouse medallion architecture and concepts of Unified Data Models
•
Strong hands-on experience with ingesting data from a variety of structured (Databases, Flat Files, SharePoint) and unstructured, on-premises and SaaS (API based ETL) sources is a must
•
Understanding of SAP and Microsoft Dynamics 365 as a source for ingestion is a huge plus
• Strong problem-solving skills as it relates to data ingestion, modeling, validation is a must
• Strong communications skills and ability to interact with technical and non-technical business stakeholders & SME is a must
• Working experience and exposure to visualization tools such as Microsoft PowerBI, Tableau desired.
• Familiarity with Scaled Agile, DevOps, Scrum, and ITIL concepts is a plus
• Experience with ServiceNow, Azure DevOps, ADO is a plus
COMPETENCIES:
•
Strong communication and presentation skills – Ability to present and communicate with technical and non-technical stakeholders at different levels within IT & business departments during the various stages of solution design, development and implementation.
•
Collaboration & Building Partnerships - Developing and leveraging relationships within and across work groups to achieve results
• Drive for Results – Taking ownership of assigned work and prompt action to accomplish objectives; taking actions to achieve goals beyond what is required; being proactive
• Continuous Improvement – Originating actions to improve existing conditions and processes, identifying improvement opportunities, generating ideas and implementing solutions