Azure Data Engineer

Azure Data Engineer

Remote

Description:

The ideal candidate will be a seasoned Azure data stack developer, database developer and data engineer who excels at delivering value. Must have hands-on experience with Azure Data Bricks, Data Factory and Data Integration Service and data warehouse architectures. Must be familiar with full stack development using C# and PySpark including unit testing. Must have experience working within a healthcare organization with related applications and data integration.

Required

· 10+ years developing software solutions.

· 5+ years working with the Azure platform: Azure Synapse Analytics, Azure Data Factory, Function Apps

· Must have expert-level experience with Azure Databricks

· 3+ years of experience writing PySpark programs to process large datasets on a spark cluster.

· 3+ years leading the technical implementation of one or more solutions with at least 2 other developers.

· 3+ years designing and developing ETL solutions using Azure Data Factory and SSIS

· Experienced with loading data to Azure Synapse Analytics – dedicated SQL pool.

· Experience writing C# .Net programs including mocked unit tests.

· Experience writing stored procedures on Microsoft SQL Server or Azure Synapse Analytics databases.

· Experience designing dimensional data models with facts and slowly changing dimensions.

· Experience writing PowerShell scripts

· Experience using git to manage code and experienced with pull request process.

· Experience managing code reviews using git and pull requests.

· Experience creating Builds and Releases in Azure DevOps.

Preferred

· Agile and Azure DevOps experience

· Healthcare experience

· Knowledge of Medallion Architecture (Bronze, Silver, and Gold Layers)

· Experience with unit testing, test automation and tools(xUnit, NUnit), test-driven development

· Experience in developing Azure Data Factory (ADF) pipelines for ETL/ELT activities from On-premise & cloud to cloud (Azure SQL DB/DW, Azure Data Lake Store) and working on Azure SQL Databases, Azure Synapse Analytics workspaces

· Experience in deploying the Azure Data Factory pipelines using Azure DevOps release pipelines

· Experience in Microsoft BI tools SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS) and SQL Server Analysis Services (SSAS)

· Experience in creating Views, Constraints, Triggers, Joins, Cursors, Temp Table, Table Variable, Functions, Batch Scripts and Stored Procedures consumed at various stages of data migration/cleansing process and validation.

· Proficient in writing complex T-SQL using CTE, Ranking Functions, TOP(n), Pivot, XML Path

· Created DB objects (Tables, Views, Functions, Stored procs) in Azure SQL db for ETL supportive activities (data loading & cleansing) and business calculations.

· Hands on SQL expertise to write complex SQL queries to create data products.

· Design, develop, and implement SSIS packages for data extraction, transformation, and loading.

· Ensure data integrity and accuracy throughout the migration process.

· Strong experience in SQL/ETL Development (Experience with SQL, T-SQL, Azure, and other DB management systems)

· Extensive knowledge writing complex SQL (T-SQL preferred) queries in the form of stored procedures, functions, CTE’s, views, etc

Additional preferred experience:

· Ability to work with Jira, SharePoint, Confluence, Git

· QNXT claims experience is ideal

· Azure DevOps

· Ability to multitask

· ServiceNow experience a plus

· Experience working with large scale data