Talent.com
This job offer is not available in your country.
Data Engineer

Data Engineer

KeasisLucknow, IN
4 days ago
Job description

We are seeking a Data Engineer with strong Apache NiFi expertise to design and implement pipelines that move and transform data from Cloudera (HDFS / Hive / Impala) into Apache Iceberg tables, with downstream integration into Snowflake and Databricks. The ideal candidate will have hands-on experience with modern data lakehouse architectures and will play a critical role in enabling scalable, governed, and high-performance data platforms.

Key Responsibilities

Data Ingestion & Pipeline Development=

Design, configure, and maintain NiFi data flows to extract, transform, and load data from Cloudera into Iceberg tables.

Implement streaming and batch ingestion pipelines with NiFi processors and custom scripting where needed.

Optimize NiFi workflows for scalability, reliability, and monitoring.

Data Lakehouse Enablement

Build and manage Apache Iceberg-based datasets for structured, semi-structured, and unstructured data.

Ensure schema evolution, partitioning, and metadata management in Iceberg.

Develop integration flows from Iceberg to Snowflake and Databricks for analytics, ML, and reporting use cases.

Integration & Orchestration

Work with Snowflake to ingest curated data from Iceberg for enterprise reporting and commercial insights.

Collaborate with Databricks teams to enable advanced analytics and machine learning use cases.

Integrate NiFi pipelines with orchestration tools (Airflow, Oozie, or AWS / Azure / GCP schedulers).

Performance, Security & Governance

Tune NiFi flows and Snowflake / Databricks ingestion for performance and cost optimization.

Implement role-based security and ensure compliance (HIPAA, GDPR, SOX if applicable).

Work with governance teams to enable lineage, metadata tracking, and auditability.

Qualifications

Bachelor’s degree in Computer Science, Information Systems, or related field.

5+ years of data engineering experience, with at least 2+ years working with Apache NiFi.

Strong experience with Cloudera ecosystem (HDFS, Hive, Impala, Spark).

Hands-on expertise with Apache Iceberg (schema evolution, time travel, partitioning, compaction).

Working knowledge of Snowflake and Databricks integration patterns.

Proficiency in SQL and one programming language (Python, Java, or Scala).

Understanding of data lakehouse architectures and ETL / ELT best practices.

Create a job alert for this search

Data Engineer • Lucknow, IN

Related jobs
  • Promoted
Data Engineer

Data Engineer

EveriseLucknow, IN
Join us on our mission to elevate customer experiences for people around the world.As a member of the Everise family, you will be part of a global experience company that believes in being people-f...Show moreLast updated: 25 days ago
  • Promoted
Data Engineer

Data Engineer

AAA GlobalLucknow, IN
Proprietary Trading / Financial Markets.We are seeking an experienced Data Engineer to strengthen our core Data Engineering team. In this key role, you will ensure the secure, scalable, and efficien...Show moreLast updated: 4 days ago
  • Promoted
Data Engineer - Quant Trading

Data Engineer - Quant Trading

Deeter Investments LLPLucknow, IN
Deeter Investments is a founder‑led proprietary trading firm built around real‑time, data‑driven decision‑making.We prize curiosity, collaboration, and a bias for action. After years of discretionar...Show moreLast updated: 4 days ago
  • Promoted
Data Engineer

Data Engineer

Magma ConsultancyLucknow, IN
Part-time (20–25 hours per week).The ideal candidate has hands-on experience in building, optimizing, and maintaining data pipelines and architectures. You’ll work closely with analysts, developers,...Show moreLast updated: 2 days ago
  • Promoted
Data Engineer

Data Engineer

TalogyLucknow, IN
This opportunity is ideal for a determined and proactive individual who has a wide range of skills in a variety of database administration, reporting and dashboarding disciplines.This role requires...Show moreLast updated: 30+ days ago
  • Promoted
Data Engineer

Data Engineer

XebiaLucknow, IN
We’re Hiring : Data Engineer | Xebia.Any Xebia location (Hybrid, 3 days office per week).Immediate to 2 weeks – only apply if you can join early. Databricks, Python, SQL, and Postgres.The ideal candi...Show moreLast updated: 24 days ago
  • Promoted
Databricks Data Engineer

Databricks Data Engineer

Insight GlobalLucknow, IN
Insight Global is seeking a Databricks Data Engineer in India with with 3–5 years of experience to support data engineering initiatives in the pharmaceutical domain. The ideal candidate will have ha...Show moreLast updated: 2 days ago
  • Promoted
Lead Data Engineer

Lead Data Engineer

Searce IncLucknow, IN
Searce means ‘a fine sieve’ & indicates ‘to refine, to analyze, to improve’.It signifies our way of working : To improve to the finest degree of excellence, ‘solving for better’ every time.Searcians...Show moreLast updated: 14 days ago
  • Promoted
Freelance Data Engineer

Freelance Data Engineer

upGradLucknow, IN
We are seeking a highly skilled and motivated.The ideal candidate will be responsible for designing, developing, and optimizing large-scale data pipelines and data warehouse solutions, utilizing a ...Show moreLast updated: 2 days ago
  • Promoted
Data Engineer

Data Engineer

KPG99 INCLucknow, IN
Duration- 12+ months with Extensions.REQUIRED SKILLS AND EXPERIENCE.Strong hands-on experience with Databricks for data processing and pipeline development. Proficiency in SQL for data querying, tra...Show moreLast updated: 4 days ago
  • Promoted
Data Engineer - Scala

Data Engineer - Scala

Idyllic ServicesLucknow, IN
Code Optimization , Shell Scripting, Data Engineering & Scala.Big Data tools like Spark, Hadoop & Hive.The Senior Data Engineer will be responsible for designing, developing, and maintaining scalab...Show moreLast updated: 2 days ago
  • Promoted
Data Platform Engineer

Data Platform Engineer

Intuitive.CloudLucknow, IN
With the reputation of being a.Digital Transformation challenges across following Intuitive Superpowers : .Application & Database Modernization. Platform Engineering (IaC / EaC, DevSecOps & SRE).Cloud N...Show moreLast updated: 24 days ago
  • Promoted
Data Engineer

Data Engineer

Manuh TechnologiesLucknow, IN
S3, Glue, Redshift, EMR, Lambda).Develop automation scripts and tools using.Collaborate with data analysts, data scientists, and business stakeholders to ensure data availability and reliability.Tr...Show moreLast updated: 30+ days ago
  • Promoted
Data Engineer

Data Engineer

Jade Business Services (JBS)Lucknow, IN
Data Engineer (AI Knowledge Base Platform).The engineer will be responsible for ensuring reliable, scalable, and secure data movement across multiple sources, powering Retrieval-Augmented Generatio...Show moreLast updated: 30+ days ago
  • Promoted
GCP Data Engineer

GCP Data Engineer

EXLLucknow, IN
Must Have- GCP Data Engineer with Banking / Finance Institutions Experience.Google Cloud Platform (GCP) Engineers.This solution will integrate data from diverse sources including.SAS systems, Excel ...Show moreLast updated: 4 days ago
  • Promoted
Data Engineer

Data Engineer

IntraEdgeLucknow, IN
The Senior Data Engineer will design, develop, monitor and maintain a robust and scalable data platform used by other data analyst and engineering teams to deliver powerful insights to both interna...Show moreLast updated: 30+ days ago
  • Promoted
Senior Data Engineer

Senior Data Engineer

InfogainLucknow, IN
Big Data Engineer (Lead) : As a Big Data Engineer (Lead), you will be responsible for leading a team of big data engineers. You will work closely with clients and team members to understand their req...Show moreLast updated: 30+ days ago
  • Promoted
SAP Data Engineer (Datasphere)

SAP Data Engineer (Datasphere)

KPG99 INCLucknow, IN
Duration : 12-month contract with extensions.Candidates' Core Skillset Must Be : .SAP ABAP, HANA / AMDP / CDS, SOAP / OData / Rest API’s etc.Show moreLast updated: 1 day ago