Senior Data Engineer

Job Overview

Summary Perform data engineering tasks using PySpark to aggregate multiple data sources into a centralized repository on AWS Maintain and improve machine learning models and PySpark based personalizationassignmentmeasurement tools Develop new features, augment tools and create new models based on business requirements Technical Qualifications Data Engineering data flow pipelines (experience as a builder of new systems) Software engineering skills (Pull requests, unit testing, automated model tests, etc) Deep experience designing, executing, measuring experiments Experience w creating experiment results visualizationsreports applying results to drive business value Statistics knowledge (e.g. parametrized vs non-parametrized tests) Modeling experience preferred (Uplift modeling, propensity score matching, LinearLogistic Regression and other supervised and unsupervised algorithms) LanguagesTools Python (pandas, numpy) SQL (analytic functions, subqueries, etc) Git Github Spark (using pySpark) Visualization (e.g. Tableau) Hadoop AWS EMR (Debian), Hive ExcelPowerPoint