Data Engineer

Amazon
Amazon

Job Overview

DESCRIPTION

Amazon Web Services (AWS) is the leading provider of cloud computing services in the world, offering a broad range of highly reliable, scalable, low-cost cloud services across over 190 countries. AWS infrastructure powers hundreds of thousands of enterprise, government and start-up businesses, including industry leaders such as Amazon.com, Netflix, Expedia, Airbnb, and many more, and renowned organizations such as NASA and the Centers for Disease Control and Prevention (CDC) and Coursera. Though already very successful AWS continues to be a high-growth, fast-moving division within Amazon with a start-up mentality where new and diverse challenges arise every day.

The AWS Central Economics team includes renowned experts in the area of forecasting, causal inference and machine learning and applies economic theory and econometric analysis to build systems and tools that inform critical decisions for the AWS business, such as service valuation and pricing, infrastructure and hardware investments, sales and marketing investments, etc. Informed decisions often impact the allocation of hundreds of million $.
Even the best analysts’ and scientists’ impact is dependent on having access to high quality, reliable data at scale. We are looking for a Senior Data Engineer to partner with our research team to understand data needs, establish/manage a data store, work with teams across AWS to identify normative data sources, and build data pipelines for production level systems. As a Sr. Data Engineer on this team, you will be a technical leader, owning the technical architecture of our BI and Data platforms. You will get the opportunity to work on very large data sets in one of the world’s largest and most complex data warehouse environments. You will work closely with the business and technical teams in analyzing many unique business problems and use creative problem-solving to deliver results.
The successful candidate will be a self-starter, comfortable with ambiguity, with strong attention to detail, an ability to work in a fast-paced and ever-changing environment, and an ability to work effectively in cross-functional teams. They will have a proven ability to meet tight deadlines, multi-task, and prioritize workloads, and a work ethic based on a strong desire to exceed expectations. They will be great at designing data systems and redefining best practices with a cloud-based approach to scalability and automation.
Key responsibilities include:
– Working with business and science stakeholders to understand requirements and effectively prioritizing
– Building secure, available, scalable, stable, and cost-effective data solutions using data storage technologies, distributed file system, data processing, and business intelligence best practices.
– Designing, developing, troubleshooting, evaluating, deploying, and documenting data management and business intelligence systems, enabling stakeholders to manage the business and make effective decisions.
– Designing and planning for solutions in the various engineering subject areas as it relates to data storage and movement solutions: data warehousing, enterprise system data architecture, data design (e.g., Logical and Physical Modeling), data persistence technologies, data processing, data management, and data analysis.
– Ensuring completeness and compatibility of the technical infrastructure to support system performance, availability and architecture requirements
– Reviewing and participating in testing of the data design, tool design, data extracts/transforms, networks and hardware selections

BASIC QUALIFICATIONS

· Bachelor’s Degree in Computer Science, Engineering or a related technical field
· Proficiency in SQL and at least one scripting or programming language (e.g. Python, Kornshell, Java)
· Experience with Big Data technologies (e.g. Hadoop, Hive, Presto, Hue, Spark, etc)
· 8+ years of experience with and detailed knowledge of data warehouse technical architectures, data modeling, infrastructure components, ETL/ ELT and reporting/analytic tools and environments, data structures and hands-on SQL coding
· 4+ years of large IT project delivery for BI oriented projects
· 4+ years of working with very large data warehousing environment

PREFERRED QUALIFICATIONS

· Graduate degree in Computer Science, Engineering or related technical field
· Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
· Experience building data products incrementally and integrating and managing datasets from multiple sources
· Experience providing technical leadership and mentoring other engineers for the best practices on the data engineering space
· Experience in designing and delivering cross functional custom reporting solutions
· Experience with Massively Parallel Processing (MPP) databases – Redshift, Teradata etc
· Experience leading large-scale data warehousing and analytics projects, including using AWS technologies – Redshift, S3, EC2, Data-pipeline and other big data technologies
· Experience with distributed systems and NoSQL databases
· Excellent oral and written communication skills including the ability to communicate effectively with both technical and non-technical stakeholders
· Strong analytical skills, 2+ years’ experience with Python and an interest in Machine Learning

Amazon is committed to a diverse and inclusive workplace.

View More
Job Detail
Shortlist Never pay anyone for job application test or interview.