Amazon Web Services is seeking an experienced, self-directed, analytical, and strategic Data Engineer to support the analytical data needs for our fast growing Professional Services practice. This is a unique opportunity to think big, insist on the highest standards, and invent and simplify the data architecture to scale and accelerate our enterprise customers’ journey to the cloud. This is a high-visibility and high business impact role.
Amazon Web Services (AWS) provides companies of all sizes with an infrastructure web services platform in the cloud (“cloud computing”). With AWS you can requisition compute power, storage, and many other services – gaining access to a suite of elastic IT infrastructure services as your business demands them. AWS is the leading platform for designing and developing applications for the cloud and is growing rapidly with millions of customers in over 190 countries. Many of these customers seek help from AWS Professional Services in their journey to a cloud-based IT operating model.
Do you have deep expertise in the end to end development of large datasets across a variety of platforms? Are you great at designing data systems and redefining best practices with a cloud-based approach to scalability and automation? In this role, you will be responsible for scaling our existing infrastructure, incorporating new data sources, and building robust data pipelines for production level systems. In partnership with our Business Intelligence & Analytics and Product Management teams, you will work backwards from our business questions to drive scalable solutions. You will be a technical leader owning the architecture of our data platform and influence best practices across multiple teams. Above all, you should be passionate about working with data to answer business questions and drive growth.
Key responsibilities include:
· Develop and support ETL pipelines with robust monitoring and alarming
· Develop data models that are optimized for business understand-ability and usability
· Develop and optimize data tables using best practices for partitioning, compression, parallelization, etc.
· Develop and maintain metadata, data catalog, and documentation for internal business customers
· Help internal business customers troubleshoot and optimize SQL and ETL solutions to solve reporting problems
· Work with internal business customers and partner technical teams to gather and document requirements for data publishing and consumption
· Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related technical discipline
· 4+ years of industry experience in Data Engineering, BI Engineer, or related field with experience manipulating, processing, and extracting value from large datasets
· Experience with SQL
· Data modeling and ETL development experience
· Data Warehousing Experience with Oracle, Redshift, Teradata, etc.
· Experience with Big Data Technologies (Hadoop, Hive, Pig, Spark, etc.)
· Experience in functional programming languages (Scala, Python, Perl, etc.)
· Masters in computer science, mathematics, statistics, economics, or other quantitative fields
· 5+ years of experience as a Data Engineer, BI Engineer or related field in a company with large, complex data sources
· Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
· Experience working with AWS big data technologies (EMR, Redshift, S3, Glue, Athena, Kinesis and Lambda for serverless ETL)
· Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, and operations
· Strong analytical skills, 2+ years’ experience with Python and an interest in Machine Learning
· Proven success in communicating with users, other technical teams, and senior management to collect requirements, describe data modeling decisions and data engineering strategy
· Ability to adapt new solutions in a fast changing environment
Amazon is committed to a diverse and inclusive workplace.