Big Data Architect

IDC Technologies
IDC Technologies

Job Overview

Apache Spark Should have working experience in Spark Programming and Spark jobs Should have working experience in Apache Spark Installation Infrastructure setup Infrastructure setup for Hadoop Usage Spark jobs will be used for streaming real-time data processing from multiple data sources Note Hadoop Mapreduce was explored and did not fit the need as there was too much latency and was not deriving the real-time data. Apache Hadoop Should have working experience in Hadoop Installation (distributed clusters) Infrastructure setup Usage To Store Data side of all the real-time processing dump that the spark jobs would stream from Oracle, Couchbase, Elastic data sources. Security Should have experience in the Authentication Authorization modules like – Kerebros LDAP S3 authentication (plan is to explore one of the authentication model) AWS Amazon S3 Oracle or any database foundation 4 Must have very strong SQL knowledge. Presto Query Engine (Plug and play) for distributed SQL engine module Docker Kubernetes Monsoon VMs Nice to have Kafka basic knowledgeBig Data Architect 1

View More
Job Detail
Shortlist Never pay anyone for job application test or interview.