Senior Real-Time Data Engineer

Job Overview

Responsible for maintaining scalable and reliable data pipelines that support data operations for Reporting, Analytics, Applications, and Data Science by gathering and processing raw data at scale. Delivers solutions by developing, testing, and supporting cloud-based streaming applications. Develops data set processes for data modeling, mining, and consumption.

Actively and consistently supports all efforts to simplify and enhance the customer experience.

Create and maintain scalable, reliable, consistent and repeatable pipelines that support data operations for Reporting, Analytics, Applications, and Data Science.

Gather and process real-time data at scale (including writing APIs, stream processing jobs, and aggregation jobs).

Expertly use cloud-based tools to ingest and process data.

Profile data to measure quality, integrity, accuracy, and completeness.

Manage life cycle of multiple data sources.

Increase speed to delivery by implementing workload/workflow automation solutions.

Interact with various stakeholders to understand their business needs, communicate project status and develop relationships to ensure satisfaction

Perform other duties as assigned.

Skills/Abilities and Knowledge
Ability to read, write, speak and understand English
Ability to use a wide variety of open source technologies and cloud services
Strong coding experience using Scala or Java
Strong background in Linux/CentOS installation and administration
Strong knowledge in data storage that demonstrates knowledge of when to use a file system, relational database, or NoSQL variant
Strong experience with Spark or Hadoop/Hive
Experience receiving, converting, and cleansing big data
Ability to identify and resolve end-to-end performance, network, server, and platform issues
Attention to detail with the ability to effectively prioritize and execute multiple tasks
In-depth knowledge of Agile development methodologies

Bachelor’s degree in an engineering discipline or computer science

Related Work Experience
3-5 years of designing and building Scala or Java applications
2+ years of designing and building cloud native applications
2+ years of Linux/Unix/CentOS system admin
1+ year(s) of designing and building Kafka or Kinesis applications

Office environment

Associated topics: data analytic, data architect, data center, data engineer, data manager, data quality, database administrator, erp, mongo database, sybaseSenior Real-Time Data Engineer 1

View More
Job Detail
Shortlist Never pay anyone for job application test or interview.