Data Engineering – Big Data

Tek Energy LLC.
Tek Energy LLC.

Job Overview

Hello, Role Data Engineering – Big Data Work Location ndash Philadelphia, PA( preferred) or NJ or NY Duration 6-12 Months Major account abilities bull Responsible for Data Engineering, Foundry Data Pipeline Creation, Foundry Analysis Reporting, Slate Application development, re-usable code development management and Integrating Internal or External System with Foundry for data ingestion with high quality. bull Have good understanding on Foundry Platform landscape and itrsquos capabilities bull Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues. bull Defines company data assets (data models), Pyspark, spark SQL, jobs to populate data models. bull Designs data integrations and data quality framework. bull Design Implement integration with Internal, External Systems, F1 AWS platform using Foundry Data Connector or Magritte Agent bull Collaboration with data scientists, data analyst and technology teams to document and leverage their understanding of the Foundry integration with different data sources – Actively participate in agile work practices bull Coordinating with Quality Engineer to ensure the all quality controls, naming convention best practices have been followed Desired Candidate Profile bull Strong data engineering background bull Experience with Clinical Data Model is preferred bull Experience in o SQL Server ,Postgres, Cassandra, Hadoop, and Spark for distributed data storage and parallel computing o Java and Groovy for our back-end applications and data integration tools o Python for data processing and analysis o Cloud infrastructure based on AWS EC2 and S3 bull 7+ years IT experience, 2+ yearsrsquo experience in Palantir Foundry Platform, 4+ yearsrsquo experience in Big Data platform bull 5+ years of Python and Pyspark development experience bull Strong troubleshooting and problem solving skills bull BTech or master’s degree in computer science or a related technical field bull Experience designing, building, and maintaining big data pipelines systems bull Hands-on experience on Palantir Foundry Platform and Foundry custom Apps development bull Able to design and implement data integration between Palantir Foundry and external Apps based on Foundry data connector framework bull Hands-on in programming languages primarily Python, R, Java, Unix shell scripts bull Hand-on experience in AWS Azure cloud platform and stack bull Strong in API based architecture and concept, able to do quick PoC using API integration and development bull Knowledge of machine learning and AI bull Skill and comfort working in a rapidly changing environment with dynamic objectives and iteration with users. bull Demonstrated ability to continuously learn, work independently, and make decisions with minimal supervision.Data Engineering - Big Data 1

View More
Job Detail
Shortlist Never pay anyone for job application test or interview.