Data Engineer

Ubique SystemsPrayagraj(Allahabad), IN

30+ days ago

Job description

Primary skills : Python, SQL, data lakes, azure

Experience : 4+ years

Immediate Joiner

Key Responsibilities

Pipeline Development & Automation

Design, build, and maintain CI / CD pipelines to automate deployment of DQ rules and data services across environments.
Optimize data pipelines and jobs for efficiency, scalability, and enterprise-grade reliability within Azure and Databricks.
Implement best practices in version control, testing, and automated deployments.

Data Quality & Rule Development

Develop, optimize, and maintain DQ rules in PySpark / Python for self-serve capabilities.

Design and implement profiling frameworks for rule generation and automated remediation.

Ensure DQ frameworks align with enterprise standards, governance, and audit requirements.

UI & Self-Serve Integration

Collaborate with front-end teams (Node.js / Angular) to enable rule configuration, validation, and monitoring via a low-code UI.

Develop APIs and services to expose DQ results and outputs for dashboards and self-service tools.

Collaboration & Governance

Partner with Data Stewards, GPOs, and business owners to translate requirements into technical solutions.

Contribute to BRDs, SRDs, and design reviews for DQ rules and pipelines.

Provide inputs on governance, compliance, and change management processes.

Support periodic reviews and continuous improvements of pipelines, rules, and dashboards.

Required Skills & Qualifications

Strong programming skills in PySpark, Python, and SQL.

Hands-on experience with Databricks (clusters, notebooks, Delta Lake).

Experience designing and implementing CI / CD pipelines (Azure DevOps / GitHub Actions / Jenkins).

Familiarity with Node.js & Angular for UI integration.

Solid understanding of data governance, DQ frameworks, and MDM concepts.

Strong analytical, problem-solving, and stakeholder engagement skills.

Cloud expertise, preferably Azure Data Lake, Data Factory, and Synapse.

Education & Experience

Bachelor’s or Master’s degree in Computer Science, Information Systems, or related field.

4–8 years of relevant experience in data engineering, CI / CD, and DQ rule development.

Proven expertise in Databricks, Azure data services, and low-code self-serve frameworks.

Create a job alert for this search

Data Engineer • Prayagraj(Allahabad), IN

Related jobs

Promoted

Data Engineer

RecroPrayagraj(Allahabad), IN

Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake. AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago

Data Engineer

BAE SystemsAU

Opportunity to work on enterprise-wide data and integration platforms.Collaboration with cross-functional teams in a modern data-driven environment. Career development and learning pathways in cloud...Show moreLast updated: 1 day ago

Promoted
New!

Data Engineer

NexionPro ServicesPrayagraj(Allahabad), IN

PySpark-based data processing workflows.Collaborate with data architects, analysts, and cross-functional teams to understand business requirements and translate them into technical solutions.Optimi...Show moreLast updated: 14 hours ago

Senior Data Engineer

Professional Search GroupAU

PSG are proud to be partnering with a large-scale mining services organisation that plays a critical role in keeping goods and services moving across Australia. With a rapidly growing customer base ...Show moreLast updated: 30+ days ago

Data Engineer

BethanieAU

Digital Enablement team to support the delivery of our data and systems integration requirements.Reporting to the Manager – Digital Enablement this is a permanent role available full-time or part-t...Show moreLast updated: 7 days ago

Promoted

Data Engineer

OWOWPrayagraj(Allahabad), IN

Data Engineer – AI-Powered Marketing Personalization Platform.AI-powered marketing personalization platform (V2.You’ll design and implement a robust multi-database infrastructure that enables real-...Show moreLast updated: 30+ days ago

•AWS Data Engineer •

AccentureAU

Combine the best of your ingenuity and Cloud technologies to build a better world and manage clients towards change.Join us and do extraordinary things. Accenture is a global professional services c...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

Randstad EnterprisePrayagraj(Allahabad), IN

Shift Timing : 2 : 00 Pm - 11 : 00 Pm.Experience : 2- 4 years relevant Experience only ( this is a Junior position with us ). GCP - 2 years minimum working Experience.Worked with global stakeholders.Ran...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

IntraEdgePrayagraj(Allahabad), IN

We are seeking a highly skilled Data Engineer with strong experience in Python, PySpark, Snowflake, and AWS Glue to join our growing data team. You will be responsible for building scalable and reli...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

Tata Consultancy ServicesPrayagraj(Allahabad), IN

TCS has been a great pioneer in feeding the fire of Techies like you.We are a global leader in the technology arena and there’s nothing that can stop us from growing together.Your role is of key im...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

Insight GlobalPrayagraj(Allahabad), IN

GCP DATA ENGINEER - Contract (Long term).Data Engineer with hands-on support for Google Looker.Strong experience in data modeling and building data marts. Proficiency in ETL / ELT pipeline development...Show moreLast updated: 30+ days ago

Promoted
New!

Data Engineer

JIT Global Infosystems Pvt. Ltd.Prayagraj(Allahabad), IN

Applications, Analytics, and Consulting services to clients worldwide.By combining technological expertise with deep domain knowledge, we ensure tangible improvements in business performance.Our ex...Show moreLast updated: 14 hours ago

Promoted

Data Engineer

BayOne SolutionsPrayagraj(Allahabad), IN

We are seeking a highly experienced Data Engineer to join our MarTech team and play a pivotal role in driving innovation within our microservices architecture, with a strong emphasis on data engine...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

DigitalzonePrayagraj(Allahabad), IN

As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics. Develop and maintain scalable ETL / ELT pipelines using Py...Show moreLast updated: 16 days ago

Promoted

Data Engineer - Palantir Foundry

NP GroupPrayagraj(Allahabad), IN

Data Engineer - Palantir Foundry, Workshop, Pyspark & Typescript.Long Term (initially 6 months) full time contract.We have an immediate requirement for an experienced Data Engineer to join the glob...Show moreLast updated: 30+ days ago

Promoted

GCP Data Engineer

AdastraPrayagraj(Allahabad), IN

We are looking for a proactive and solution-oriented GCP Data Engineer to join our team.This role requires hands-on experience in Google Cloud Platform (GCP), especially with BigQuery and Airflow, ...Show moreLast updated: 3 days ago

Promoted
New!

Data Engineer (Remote)

QAAgility TechnologiesPrayagraj(Allahabad), IN

Remote

The ideal candidate will be responsible for designing, developing, and maintaining robust data pipelines that support key business functions across the supply chain domain.This role requires strong...Show moreLast updated: 14 hours ago

Promoted
New!

Data Engineer

MyData Insights Pvt LtdPrayagraj(Allahabad), IN

The ideal candidate has a strong background in building scalable data solutions on.CI / CD pipelines, infrastructure automation using Terraform, and cloud-native data engineering practices.Design, bu...Show moreLast updated: 14 hours ago