Lead Big Data Engineer

Cynet Systems
Cynet Systems

  • US, Media
  • Post Date: 25 August 2020
  • Views 3
Job Overview

We are looking for Lead Big Data Engineer for our client in Media, PA Job Title Lead Big Data Engineer Job Location Media, PA Job Type Contract Job Description The client is looking for Data Technical Lead experienced to design, build and maintain Big Data workflowspipelines for collecting, storing, processing, and analyzing huge sets of data into and out of AWS data lake. Engage in application design and data modeling discussions. Guide team with design specifications and development. Build and test data workflowspipelines. Troubleshoot and resolve application and data processing issues. Code optimization and fine-tune application performance. Qualifications BSBA degree in Computer Science, Information Systems, or related field. At least 8 yearsrsquo experience in the Big Data Hadoop technology area, specifically in Cloudera and AWS. Must have led a team of data engineers to deliver data integration projects in these technology areas. Highly skilled in Spark and Scala. Experienced in the Databricks platform. Worked in the AWS environment in storing, processing data on S3, and transforming data with complex computing into other data models. Strong knowledge in SQL and UnixLinux scripts. Exposure to other Hadoop Ecosystem technologies like YARN, Zookeeper, HDFS, Avro, Parquet, etc. Connecting to various data sources such as RDBMS, NoSQL, FTPSFTP, etc., and troubleshooting connection issues. Experience with designing and developing pipelines in cleansing, preparing large, complex data sets for reporting and analytics. Must have used Data Integration Tools such as Talend to develop data pipelines and workflows. Knowledge of storage design concepts including partitioning. Maintain, modify, and improve a large set of structured and unstructured data. Has created or worked with frameworks that apply generic behavior for large data pipelines and implements audit, balance, and control mechanisms effectively. Monitoring and troubleshooting data integration jobs. Exposure to using APIs in retrieving and distributing data. Experience in CICD tools for complete automation of development and deployment. Added Valuable Skills Exposure to Databricks is highly preferred. NoSQL such as HBase. Distributed Messaging such as Kafka. Data architecture.Lead Big Data Engineer 1

View More
Job Detail
Shortlist Never pay anyone for job application test or interview.