Sr Azure Data Engineer

Digital Intelligence Systems, LLC
Digital Intelligence Systems, LLC

Job Overview

DISYS is looking for Sr Azure Data Engineer in Columbia, MD. Job Summary The successful candidate will work to design, develop, and maintain a data analytics platform solution. Client is currently running an Azure Data Lake 2 Azure Data Factory Spark Databricks infrastructure with 10 production use cases that consolidate several streams of healthcare data. This role will primarily focus on the responsibility of production operations of the analytics infrastructure including communication, technical proficiency, collaboration and willingness to learn new things. This role will involve technical oversight of 1-2 junior data engineers, and planning and skills to implement new ingestion pipelines, troubleshoot production issues and manage production releases of new use case notebooks. ESSENTIAL DUTIES AND RESPONSIBILITIES Include the following. Other duties may be assigned. On-board new data sources into data platform Work with source system owners to understand integration needs Work with business users to understand reporting needs Implement data pipelines to ingest data to the platform, standardize the data, and transform the data into business facing datasets Perform unit and integration testing of data pipelines Document data pipelines for maintainability Maintain existing data source in data platform pipeline Monitor data pipeline job status Work with source system owners and business owners to incorporate business changes into data pipeline (i.e. updating metadata file to add new object, update curated layer query, etc.) Perform automated data quality checks on data pipelines Perform continuous improvement Required Skills Experience in a production Data Lake or Data Warehouse environment Data Integration – Intermediate Data Lake Design Patterns ndash Familiar Agile Methodologies – Familiar Production support of a complete system with Dev, Test and Prod environments and clear understanding of DevOps. Ability to manage priorities required, both individually and for a team Ability to prioritize and manage multiple job duties, including personal time management and organization skills Ability to diagram and understand technical workflows Preferred Skills SQL – Intermediate Python – Intermediate Cloud Infrastructure (Azure) – Foundational Education and Experience 4-year college degree 5+ years of working with in data integration Best, BalaMurugesh Associate Team Lead, Centralized Recruiting Digital Intelligence Systems (DISYS)Sr Azure Data Engineer 1

View More
Job Detail
Shortlist Never pay anyone for job application test or interview.