Applied Resource Group

Data Engineer

Source: Lensa

*This Position is Contract to Hire*

*Manufacturing Experience Preferred*

Applied Resource Group is looking for a Data Engineer to join our clients Global Digital Team. He/she will aid in the optimization of operations by manipulating and aggregating the disparate operational and back-office data sources into a format that is easily consumable by both data scientists and statistically adept colleagues. His/her primary responsibility will be to combine large volumes of disparate complex data, conduct quality checks on the data, manipulate the data and ensure continuous access to a clean format of the operational data for data scientists and other stakeholders. In addition, he/she will also assist in developing the data pipeline to ensure ongoing data collection, consolidation, and management

Responsibilities

Design and Develop data ingestion pipelines and processes based on requirements in Python and PySpark

Create error handing, exception management and data quality routines to expose the anomalies in the data

Profile and analyze data to identify gaps and potential data quality issues

Identifies relationships between disparate data sources

Uses Python, Databricks and Spark to code the data Engineering routines

Perform unit and integration testing

Works with the group of data scientists and business SMEs to get the requirements and present the details in data.

Designs and jointly develops the data architecture with data architect and ensures security and maintenance

Explores suitable options, designs, and creates data pipeline (data lake / data warehouses) for specific analytical solutions

Identifies gaps and implements solutions for data security, quality and automation of processes

Builds data tools and products for effort automation and easy data accessibility

Supports maintenance, bug fixing and performance analysis along data pipeline

  • Diagnoses existing architecture and data maturity and identifies gaps

by Jobble


11 minutes ago



Atlanta, Ga

Applied Resource Group

Full Time

Apply at Uzoes
By registering, you are giving consent for uzoes to inform you of jobs by email according to your search. I declare that I have read and accept the Terms of Service , Privacy Policy and Cookies Policy of beBee Affinity Social Network SL

Related Jobs

  • https://dktn8eak9qti5.cloudfront.net/img/letters/C.png

    data engineer/data scientist

    Somebody who has enough experience in spark data bricks can code in spark, build pipelines in pyspark can also take a business problem and build a data sci

    Collabera Inc. - Atlanta, Georgia - jooble

  • https://www.energyjobline.com/sites/default/files/styles/squared_logo/public/job-logo/get-logo.php__66094.png?itok=dubCpYH-

    Data Engineer (Data Engineering)

    The Government Technology Agency (GovTech) aims to transform the delivery of Government digital services by taking an "outside-in" view, putting citizens and businesses at the heart of everything w

    Government Technology Agency - Emory university, Georgia - beBee S

  • https://www.energyjobline.com/sites/default/files/styles/squared_logo/public/job-logo/get-logo.php__65589.png?itok=rFmDzxEr

    Data Engineer (Data Engineering)

    The Government Technology Agency (GovTech) aims to transform the delivery of Government digital services by taking an outside-in" view, putting citizens and businesses at the heart of everything we

    Government Technology Agency - Emory university, Georgia - beBee S

  • https://dktn8eak9qti5.cloudfront.net/img/letters/C.png

    Data Engineer – Big Data

    Job Description Overview Job Title: Data Engineer – Big Data Duration : 12+months contract -C2C Job Description Seeking an experienced Data Engineer – Big Data

    Cerebra Consulting Inc - Atlanta, Georgia - beBee S