AWS Data Engineer (Big Data, Python, Apache Spark)

Softsages LLC
Softsages LLC

Job Overview

Role – AWS Data Engineer (Big Data, Python, Apache Spark) Location Philadelphia, PA We need a seasoned Data engineer with 6+ years of experience working with Data Centric Applications and Services in AWS Cloud with at least 3 years of experience working in Apache Spark What we’re looking for Expertise with Python, PySpark, Scala, PyData, DataFrames, Jupyter Notebook and understands the fundamentals of functional programming language. Excellent data modelling experience and understanding of core concepts of Big Data and has practical experience working with 100s of Terabytes to Petabytes of data Solid experience and understanding of various core AWS services such as EC2, S3, EMRSpark, Glue, SQS, KinesisFirehose, DynamoDBDAX, Step functions, ElasticSearch, Athena and Redshift. Hands on programming experience using AWS SDK (Boto3), Java SDK or CLI Deep understand AWS Infrastructure, networking best practices, multi-region latency and fail overdisaster recovery strategy Has practical experience transferring big data in and out of AWS, transforming and computing in AWS at scale Experience in developing distributed applications and dynamic workflows using serverless architecture Analytical skills and experience in solving big data problems effectively using SparkEMR Experience with public cloud data sharing practices and security Experience working with CICD pipelines The ideal candidate has Real time data streaming, batching and transformations on data in transit Nice to have experience in infrastructure automation using Infrastructure as Code methodology and well versed with CloudFormation or Terraform Non-Technical Experience Effective Collaborator and Team Player Quick Learner and Self-starter with excellent debugging skills Ability to foresee scalability, budget issues and make judgements Additional Skills Required Spark, PySpark, EMR, Big Data technologies The top skills to target Apache Spark, Python, Big Data Email Peter at softsages dot comAWS Data Engineer (Big Data, Python, Apache Spark) 1

View More
Job Detail
Shortlist Never pay anyone for job application test or interview.