Need for Data Engineer

Job Overview

Sr.Data Engineer Durham, NC 12 month contract Top Three Skills 1) 7+ years of software development with two recent years of Scala development. 2) 3+ years of using Spark 3) Current experience working with Kafka. Job Description bullGood knowledge and Hands on Scala Programming bullGood skills on modular program development in Scala bullData pipeline using Spark Scala bullLoad disparate data sets by leveraging Kafka consumers bullAbility to utilize Hive, Spark, Cassandra, Mesos and Kafka bullExperience with AWS components and services, particularly EMR, S3, and Lambda bullGood understanding of file formats including JSON, Parquet, Avro, and others bullExperience with open source NoSQL technologies such as Cassandra bullExperience with messaging and complex event-processing systems such as Kafka and Storm bullExtensive hands on Data frame and Dataset operations of Spark bullBasic Understanding of Bigdata Technologies and Hadoop bullGood knowledge on Hive and HiveQL bullKnowledge on Cassandra Architecture and CQL is desirable bullBuild reusable code, with the ability to scale with very large data volumes bullKnowledge on Java Programming is desirable bullData MigrationETL knowledge is desirable bullExpert knowledge of software development tools, practices, and lifecycle methodologies bullExtensive experience with legacy and modern data warehouse concepts and implementations bullExtensive experience with data integration concepts and implementations including ETL, ELT, CDC, streaming, pubsub bullAble to produce high quality design documentation and provide thorough reviews of design documentation produced by others Ability to establish and maintain effective working relationships with co-workers, managers and clients bullFlexibility, ability to simultaneously juggle multiple complex projects and adapt to rapidly changing priorities