Big Data Engineer/Architect

Idexcel Inc.
Idexcel Inc.

Job Overview

Job Title Big Data EngineerArchitectLocation Dallas, TXDuration Long Term Contract Requirements Bachelorrsquos degree or equivalent in computer scienceengineering, computer information systems or information technology. At least 5 yearsrsquo experience in Big DataHadoop based Infrastructure. At least 6 yearsrsquo experience in Developing and Designing Software Applications. Experience with multiple SQL flavors, MapReducePigHive, Sparkscala, Python, Kafka, HBasePhoenix, Druid, Airflow. Developing petabyte-scale data-processing workflows with complicated business requirements covering various search, visualization, analytics, and reporting functions Certified Big Data Architect. Expertise and experience in software architecture, development and on-prem application design and migration. HortonWorks HDP (Big Data), Hadoop, MapReduce, Pig, Hive, Spark Core Big Data toolsproducts Kafka, HBasePhoenix, Druid, Airflow Languages like Scala, Python and Java JavaJ2EE, spring boot, Groovy, RESTful web services Web API, R. Agile Role Description Developing petabyte-scale data-processing workflows with complicated business requirements covering various search, visualization, analytics, and reporting functions Subject matter expert for the few dozen data sources that the System collects Ensuring data integrity for processed data sources All development related to data processing job orchestration Supporting machine learning experts that use the processed data New development, including bug fixes, feature enhancements, hotfixes Automation of builds and deployments Researching and prototyping new technologies and features. Key Responsibilities Subject matter expert for data processing and serving systems using legacy and state-ofthe-art frameworks, systems, and languages including but not limited to multiple SQL flavors, MapreducePigHive, Sparkscala, Python, Kafka, HBasePhoenix, Druid, Airflow Production support of two data-processing Hadoop based infrastructures that power web applications used by security groups. Developing petabyte-scale data-processing workflows with complicated business requirements covering various search, visualization, analytics, and reporting functions Subject matter expert for the few dozen data sources that the System collects Ensuring data integrity for processed data sources All development related to data processing job orchestration New development, including bug fixes, feature enhancements, hotfixes Automation of builds and deployments Researching and prototyping new technologies and features. Skills RequirementsBig Data, HortonWorks HDP (Big Data), Hadoop, MapReduce, Pig, Hive, Spark, Python, JavaJ2EE, spring boot, Groovy, RESTful web services Web API, R About IDEXCEL, INCIdexcel is an IT services organization, with a mission to bring great people and great organizations together. Our diverse client base represents a wide range of industries, including technology, telecom, insurance, healthcare, manufacturing, banking financial services, food commodities trading and federal organizations. Our teams of experienced recruiters directly work with client companies seeking exceptional people to help with their business initiatives. Idexcel, Inc. is an Equal Opportunity Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, age, disability, military status, national origin or any other characteristic protected under federal, state, or applicable local law.Big Data Engineer/Architect 1

View More
Job Detail
Shortlist Never pay anyone for job application test or interview.