Senior Data Engineer – Wallet & Apple Pay


Job Overview

Key Qualifications

  • 10+ years experience with ETL, BI & Data Analytics
  • 5+ years of professional experience with Big Data systems, data pipelines and data processing
  • Practical hands-on experience with technologies like Apache Hadoop, Apache Pig, Apache Hive, Apache Sqoop & Apache Spark
  • Ability to understand API Specs, identify relevant API calls , extract data and implement data pipelines & SQL friendly data structures
  • Understanding on various distributed file formats such as Apache AVRO, Apache Parquet and common methods in data transformation
  • Expertise in Python, Unix Shell scripting and Dependency driven job schedulers!
  • Expertise in Core JAVA, Oracle, Teradata and ANSI SQL
  • Familiarity with Apache Oozie and PySpark
  • Knowledge on Scala and Splunk is a plus
  • Familiarity with Data visualization tools such as Tableau.
  • Familiarity with rule based tools and APIs for multi stage data correlation on large data sets
  • Excellent time management skills to manage work to tight deadlines and handle the pressure of executive requests and product launches
  • Experience with mentoring and leading data engineers
  • Description

    Translate business requirements by business team into data and engineering specificationsBuild scalable data sets based on engineering specifications from the available raw data and derive business metrics/insights Work with engineering and business stakeholders to define and implement the data engagement relationships required with partners Understand and Identify server APIs that needs to be instrumented for data reporting and align the server events for execution in already established data pipelines Analyze complex data sets, identify and formulate correlational rules between heterogeneous sources for effective analytics Process, clean and validate the integrity of data used for analysis Develop Python and Shell Scripts for data ingestion from external data sources for business insights Work hand in hand with the DevOps team and develop monitoring and alerting scripts on various data pipelines and jobs Mentor a team of talented engineers

    Education & Experience

    Minimum of bachelor’s degree, preferably in Computer Science, Information Technology or EE, or relevant industry experience is preferred

    View More
    Job Detail
    Shortlist Never pay anyone for job application test or interview.