Data engineer with Big Data & spark

BuzzClan LLC
BuzzClan LLC

Job Overview

Title Data Engineer with Big Data Spark Location Remote for now Duartion 6 months C-H 7+ years of progressively responsible work experienceas a Data Engineer in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. Deep hands-on expertise with architecting, building, and maintaining a large-scale distributed data processing pipeline. Strong analytic skills related to working with unstructured datasets. Experience with end-to-end data pipeline automation process, including data input, data quality validation, data extraction and data reporting Build processes supporting data transformation, data structures, metadata, dependency and workload management. Working knowledge of message queuing, stream processing, and highly scalable lsquobig datarsquo data stores. Excellence in communication-, collaboration-, and organizational skills. supporting and working with a diverse cross-functional remote teams in a dynamic environment. Experience using the following softwaretools Experience with big data technologies Spark, Distributed File Systems, Hive, Parquet and column stores in general Experience with data streaming and stream-processing systems Azure Stream Analytics, Spark-Streaming, Azure Event Hub and Service Bus, Rabbit MQ, etc. Experience with relational SQL and NoSQL databases, including MSAzure SQL, CosmosDb, Azure Data Explorer and PostgreSQL. Experience with data pipeline and workflow management tools Azure Data Factory, Azure Logic Apps Experience with Azure cloud services Azure SQL, CosmosDb, Azure DWSynapse, Azure Data Lake, IoT HubEdge, Azure Databrick Experience with object-orientedobject function scripting languages Python, C, Scala, etc. Experience with data service frameworks such as GraphQL and OData Understanding of Data visualization tools such as Power BI and Grafana Contact-Ext 0022)Data engineer with Big Data & spark 1

View More
Job Detail
Shortlist Never pay anyone for job application test or interview.