Software Development Engineer – Big Data, AWS Elastic MapReduce (EMR)

Amazon
Amazon

Job Overview

DESCRIPTION

Want to change the world with Big Data and Analytics? Come join us on the Amazon EMR team in Amazon Web Services!

Amazon EMR is a web service which enables customers to run massive clusters with distributed big data frameworks like Apache Hadoop, Hive, Tez, Flink, Spark, Presto, HBase, Hudi and more, with the ability to effortlessly scale up and down as needed. We run large number of customer clusters, enabling processing on vast datasets.

We are developing innovative new features including our next-generation cluster management system, improvements for real-time processing of big data, ways to process more data faster, and to enable customers to more easily interact with their data. We’re looking for top engineers to build them from the ground up.

Here are sample features that we have delivered:
· Launched Hudi on EMR: https://aws.amazon.com/emr/features/hudi/
· Support improved Apache HBase performance on Amazon S3 with persistent HFile tracking, and Apache Hive ACID transactions on HDFS and Amazon S3: https://aws.amazon.com/about-aws/whats-new/2020/12/amazon-emr-release-6-2-supports-apache-hbase-on-amazon-s3-apache-hive-acid-transactions-on-hfds-and-amazon-s3/
· Support EC2 instances powered by AWS Graviton2 processors with EMR: https://aws.amazon.com/about-aws/whats-new/2020/12/amazon-emr-now-provides-up-to-30-lower-cost-and-up-to-15-improved-performance/
· Support of High Availability of Applications on EMR: https://aws.amazon.com/about-aws/whats-new/2019/04/amazon-emr-announces-support-for-multiple-master-nodes-to-enable-high-availability-for-EMR-applications/
This is a hands-on position where you will be do everything from designing and building extremely stable components and cutting-edge features for the savviest customers in the business to help them get the best results.

You will have a chance to work with the open source community and contribute significant portions its software to open source projects possibly including Apache HBase and Hudi. You need to not only be a top software developer with excellent programming skills, an understanding of big data and parallelization, and a stellar record of delivery but also excel at leadership and customer obsession and have a real passion for massive-scale computing. If you want to truly test your mettle against the hardest challenges in distributed systems to build solutions for large scale problems in a wide variety of domains, come join our group.

Your responsibilities will include:
– Translation of complex functional and technical requirements into detailed architecture and design
– Deliver systems and features with top-notch quality, on time
– Develop new technologies for monitoring production clusters
– Own the software development process end-to-end, including: working with engineers and product managers to develop requirements; designing, architecting, planning, implementing, and testing new systems and features; deploying, and operating the production EMR systems.

In joining our team, you will get to work with a minimum of technical supervision, while playing a variety of roles as needed to respond efficiently to multiple program priorities. You will get to collaborate with some of the best and brightest minds in the industry. You’ll enjoy a competitive salary, great benefits, a creative and agile work environment, and the exciting opportunity to be part of a fast-paced and growing team and one of the most innovative technology companies – but most of all, you will get the satisfaction of making products that millions use everyday to great effect!

For more information:
· AWS Big Data Blog: https://aws.amazon.com/blogs/big-data/
· AWS EMR: https://aws.amazon.com/emr/
· AWS re:Invent 2020 Keynote – Andy Jassy: https://www.youtube.com/watch?v=xZ3k7Fd6_eU

Inclusive Team Culture
Here at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust.

Work/Life Balance
Our team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.

Mentorship & Career Growth

Our team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future.

BASIC QUALIFICATIONS

· 2+ years of non-internship professional software development experience
· Programming experience with at least one modern language such as Java, C++, or C# including object-oriented design
· 1+ years of experience contributing to the architecture and design (architecture, design patterns, reliability and scaling) of new and current systems.
· 3+ years of experience with Master’s degree in Computer Science or equivalent OR 5+ years experience with Bachelor’s Degree in Computer Science or equivalent, in: software development, including design, implementation, debugging, and support
· Very strong Computer Science fundamentals in algorithm design, data structures, problem solving, and complexity analysis
· Strong proficiency in developing objected-oriented software, with deep experience in one or more relevant languages (Java, C, C++, C#)

PREFERRED QUALIFICATIONS

· Very strong Computer Science fundamentals in algorithm design, data structures, problem solving, and complexity analysis
· Security background in identifying security vulnerabilities and assessing risks
· Experience designing and building highly-scaled distributed systems and web services
· Thorough understanding of parallel algorithms, concurrency, asynchronous architectures
· Experience working in an agile software development organization
· Knowledge of and contribution to Hadoop ecosystem
· Experience with one or more of Ruby, Python, Perl
· Proven ability to effectively drive cross-team solutions that may have complex dependencies
· Masters in Computer Science with emphasis on distributed systems and Big Data architectures is a plus.

Amazon is committed to a diverse and inclusive workplace.

View More
Job Detail
Shortlist Never pay anyone for job application test or interview.