Big Data Engineer


  • US
  • Post Date: 19 August 2020
  • Views 12
Job Overview

ServerLogic has partnered with a local company to identify a contract Big Data Engineer for their building solutions for Digital. You will be working on development projects related to consumer behavior, commerce, and web analytics for a 12 month project. Responsibilities Design and implement distributed data processing pipelines using Spark, Hive, Sqoop, Python, and other tools and languages prevalent in the Hadoop ecosystem. Ability to design and implement end to end solution. Build utilities, user defined functions, and frameworks to better enable data flow patterns. Research, evaluate and utilize new technologiestoolsframeworks centered around Hadoop and other elements in the Big Data space. Define and build data acquisitions and consumption strategies Build and incorporate automated unit tests, participate in integration testing efforts. Work with teams to resolving operational performance issues Work with architectureengineering leads and other teams to ensure quality solutions are implements, and engineering best practices are defined and adhered to. Qualification MSBS degree in a computer science field or related discipline 6 yearsrsquo experience in large-scale software development 1 year experience in Hadoop Strong Java programming, Python, shell scripting, and SQL Strong development skills around Hadoop, Spark, MapReduce, Hive, and Pig Strong understanding of Hadoop internals Good understanding of file formats including JSON, Parquet, Avro, and others Experience with databases like Oracle Experience with performancescalability tuning, algorithms and computational complexity Experience (at least familiarity) with data warehousing, dimensional modeling and ETL development Ability to understand ERDs and relational database schemas Proven ability to work cross functional teams to deliver appropriate resolution Nice to have Experience with AWS components and services, particularly, EMR, S3, and Lambda Experience with open source NOSQL technologies such as HBase, DynamoDB, Cassandra Experience with messaging complex event processing systems such as Kafka and Storm Automated testing, Continuous Integration Continuous Delivery Scala Machine learning frameworks Statistical analysis with Python, R or similar

View More
Job Detail
Shortlist Never pay anyone for job application test or interview.