Data Engineer (Big Data-Hadoop) – W2 – Remote job

Job Overview

bull Analyze and understand data sources APIs bull Design and Develop methods to connect collect data from different data sources bull Design and Develop methods to filtercleanse the data bull Design and Develop SQL , Hive queries, APIs to extract data from the store bull Work closely with data Scientists to ensure the source data is aggregated and cleansed bull Work with product managers to understand the business objectives bull Work with cloud and data architects to define robust architecture in cloud setup pipelines and work flows bull Work with DevOps to build automated data pipelines Total Experience Required bull 4 years 10 of relevant experience bull The candidate should have performed client facing roles and possess excellent communication skills Business Domain knowledge Finance banking systems, Fraud, Payments Required Technical Skills Big Data-Hadoop, NoSQL, Hive, Apache Spark Python PySpark HBase Java REST GIT and Version Control Desirable Technical Skills bull Familiarity with HTTP and invoking web-APIs bull Exposure to machine learning engineering bull Exposure to NLP and text processing bull Experience with pipelines, job scheduling and workflow management Personal Skills Experienced in managing work with distributed teams bull Experience working in SCRUM methodology bull Proven sense of high accountability and self-drive to take on and see through big challenges bull Confident, takes ownership, willingness to get the job done bull Excellent verbal communications and cross group collaboration skills