Data Engineer Lead

Job Overview

Role – Data Engineer Lead Location – Richmond, VA Term – 24+ Months Note – This is a Remote Job until COVID gets resolved. Responsibilities for Data Engineer – Need to be Very strong Technically and in Coding. Need to have Very good knowledge in Java, Python Spark Assemble large, complex data sets that meet functional non-functional business requirements. Identify, design and implement internal process improvements automating and manual process, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data resources using SQL and AWS lsquobig datarsquo technologies(EC2, EMR, S3, Lambda). Needs to have hands on experience in Spark, Scala PySpark Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics. Work with stakeholders including the Executive, product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs. Keep our data separated and secure across national boundaries through multiple data centers and AWS regions. Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader. Work with data analytics experts to strive for greater functionality in our data systems. Qualifications for Data Engineer Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases. Experience building and optimizing lsquobig datarsquo data pipelines, architecture and data sets. Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement. Strong analytical skills related to working with unstructured datasets. Build process supporting data transformation, data structures, metadata, dependency, and workload management. A successful history of manipulation, processing and extracting value from large disconnected datasets. Working knowledge of message queuing, stream processing, and highly scalable lsquobig datarsquo data stores.