Data Engineer IV


Job Overview

Senior Data Engineer 8-11 month contract VirtualPacific time zone Not open to C2C OVERVIEW Our high profile client is seeking an experienced Data Engineer with expert level complex SQL and ETL development skills and experience with technical requirements gathering for a REMOTE contract job opportunity. RESPONSIBILITIES Validating all Datanet jobs are operating in optimal fashion, follow namingcode standards, are associated with correct user accounts and checked into source control (2 weeks) Consolidation of G2GBIS code to remove duplication, split into individual jobs that operate in cascading fashion to make troubleshootingrestartingpartial runs easier (2 weeks) Setup of automation of Brady G2GBIS code that will consume the S3 CSV file (uploaded manually) instead of requiring manual ad-hoc running of the code on demand (1 week) Clean and consolidate the “holding” section of Datanet. Catalog old jobs that have been deprecated and verify they are in source control, documentation about their configuration is confirmed and descriptions of their last functionality is created (1 week) Cleanupremove unused objects in Redshfit cluster. Move backup data into backup schemas and only leave production data in main schemas (1 week) Cleanup groupsLDAPHammerstonePrivilegesetc. to automate as much as possible. Removing redundantunused items and users and document their current use (1 week) Create Tableau visualizations to enable us to more easily perform security audits. Show all users who have access to confidential and diversity data, whether they are using it, and whether they are still in the org (2 weeks) Keep up with the Phoenix V2 beta changes. Consume their beta datasets as quickly as possible to determine if their rollout will cause any issues so we can address them as quickly as possible. Integrate new V2 when the goto production (12+ weeks) Cleanup our core datasets in Tableau to minimize the duplicate columns and standardize all column namestypes (12+ weeks) Cleanup our core datasets in Redshift objects to minimize the duplicate columns and standardize all column namestypes (remove any out of date or duplicate objects or replace with a view to the new sources) (12+ weeks) Cleanup our code base to have a standardized format and make it easier to compare like code. Lots of copypasted sections of code with many formatting styles makes it difficult to see code similaritiesbugsduplications (4+ weeks) Identify duplication of code in Datanet jobs where code could be removed as it is unnecessary, already done elsewhere, or is inconsistent. Use this to document actions which should be moved to earlier pre-processing and separation of business and display logic (12+ weeks) Identify logical inconsistencies in our code base (using mcchcmap row must exist vs a direct join on key columns or using Point in Time values vs Current values) and document the issues and other areas that have the same or similar logicissues (12+ weeks) Improve G2GBIS to remove hard-coded assumptions (OA starts 1.1 or fixed values for timing of funnel progress) (12+ weeks) Improve G2GBIS to make it easier to add scenarios, especially simple changes like reduced attrition model to be done perhaps via table instead of code changes (12+ weeks) Data DictionariesData LineageData Sample Queriesusage (4+ weeks) Cleanupconsolidation of the MBR Datanet jobs to reduce code duplicationcomplexity (6 weeks) Standard Operating Procedure – Writing them, testing them out, implementing them. (4 weeks) Reports Silver Medalist (10+ weeks) nonConf version of BIS (10+ weeks) MGHDBRITOSDEFI (10+ weeks) Beeline Data (10+ weeks) Events and POD Data (beamery) (10+ weeks) Market Performance (10+ weeks) NHOC focused report (10+ weeks) REQUIRED SKILLS Bachelors Degree in Data Science or Mathematics 7+ years of Data Engineering experience Experience with complex SQL ETL Experience with (technical) requirements gathering Experience with Data transformation pipelining Experience with big data large data warehouse Experience with optimizing performance of business-critical queries Experience with joining and cleaning data PREFERRED SKILLS Masters Degree in Data Science or Mathematics a huge plus Tableau Domain working experience in HR Recruiting business andor for clients in that segments Working experience with Data scientists, BIE and Business Analysts to design and develop data infrastructure Experience with AWS technologies including Redshift, RDS, S3 or similar solutions Strong analytical skills Strong communication (written verbal) Working experience with remote co-workers Experis is an Equal Opportunity Employer (EOEAA)Data Engineer IV 1

View More
Job Detail
Shortlist Never pay anyone for job application test or interview.