Talent.com
This job offer is not available in your country.
Data Engineer

Data Engineer

Keasisdindigul, tamil nadu, in
1 day ago
Job description

We are seeking a Data Engineer with strong Apache NiFi expertise to design and implement pipelines that move and transform data from Cloudera (HDFS / Hive / Impala) into Apache Iceberg tables, with downstream integration into Snowflake and Databricks. The ideal candidate will have hands-on experience with modern data lakehouse architectures and will play a critical role in enabling scalable, governed, and high-performance data platforms.

Key Responsibilities

Data Ingestion & Pipeline Development=

Design, configure, and maintain NiFi data flows to extract, transform, and load data from Cloudera into Iceberg tables.

Implement streaming and batch ingestion pipelines with NiFi processors and custom scripting where needed.

Optimize NiFi workflows for scalability, reliability, and monitoring.

Data Lakehouse Enablement

Build and manage Apache Iceberg-based datasets for structured, semi-structured, and unstructured data.

Ensure schema evolution, partitioning, and metadata management in Iceberg.

Develop integration flows from Iceberg to Snowflake and Databricks for analytics, ML, and reporting use cases.

Integration & Orchestration

Work with Snowflake to ingest curated data from Iceberg for enterprise reporting and commercial insights.

Collaborate with Databricks teams to enable advanced analytics and machine learning use cases.

Integrate NiFi pipelines with orchestration tools (Airflow, Oozie, or AWS / Azure / GCP schedulers).

Performance, Security & Governance

Tune NiFi flows and Snowflake / Databricks ingestion for performance and cost optimization.

Implement role-based security and ensure compliance (HIPAA, GDPR, SOX if applicable).

Work with governance teams to enable lineage, metadata tracking, and auditability.

Qualifications

Bachelor’s degree in Computer Science, Information Systems, or related field.

5+ years of data engineering experience, with at least 2+ years working with Apache NiFi.

Strong experience with Cloudera ecosystem (HDFS, Hive, Impala, Spark).

Hands-on expertise with Apache Iceberg (schema evolution, time travel, partitioning, compaction).

Working knowledge of Snowflake and Databricks integration patterns.

Proficiency in SQL and one programming language (Python, Java, or Scala).

Understanding of data lakehouse architectures and ETL / ELT best practices.

Create a job alert for this search

Data Engineer • dindigul, tamil nadu, in

Related jobs
  • Promoted
Senior Data Engineer

Senior Data Engineer

HealthyMe AIDindigul, IN
We are seeking a Senior Data Engineer with deep expertise in building data pipelines and integrating healthcare data.You will report to the Head of AI and Engineering and be responsible for designi...Show moreLast updated: 9 days ago
  • Promoted
Senior Data Engineer

Senior Data Engineer

InfogainMadurai, IN
Big Data Engineer (Lead) : As a Big Data Engineer (Lead), you will be responsible for leading a team of big data engineers. You will work closely with clients and team members to understand their req...Show moreLast updated: 30+ days ago
  • Promoted
Data Engineer

Data Engineer

Forage AIDindigul, IN
Experience Level : Data Engineer- 3- 7 years of relevant experience in data engineering.Forage AI is a pioneering AI-powered data extraction and automation company that transforms complex, unstructu...Show moreLast updated: 30+ days ago
  • Promoted
Data Engineer

Data Engineer

EveriseDindigul, IN
Join us on our mission to elevate customer experiences for people around the world.As a member of the Everise family, you will be part of a global experience company that believes in being people-f...Show moreLast updated: 22 days ago
  • Promoted
Data Engineer

Data Engineer

IntraEdgeDindigul, IN
The Senior Data Engineer will design, develop, monitor and maintain a robust and scalable data platform used by other data analyst and engineering teams to deliver powerful insights to both interna...Show moreLast updated: 30+ days ago
  • Promoted
Data Engineer

Data Engineer

SapaadMadurai, IN
Our flagship product, also named Sapaad, has achieved remarkable success over the past decade, empowering.F&B businesses across 40+ countries. Driven by a passionate team of developers, designers, a...Show moreLast updated: 30+ days ago
  • Promoted
Data Engineer - Web Scraping

Data Engineer - Web Scraping

Alternative PathDindigul, IN
Alternative Path is seeking skilled software developers to collaborate on client projects with an asset management firm.In this role, you will collaborate with individuals across various company de...Show moreLast updated: 30+ days ago
  • Promoted
Senior Data Engineer

Senior Data Engineer

NOQMadurai, IN
Senior Data & Reporting Developer (Remote | India | Contract Role).NOQ Group is a technology-driven organization dedicated to delivering innovative data solutions for businesses in the food and bev...Show moreLast updated: 1 day ago
  • Promoted
Data Engineer

Data Engineer

Insight GlobalDindigul, IN
Insight Global is looking for a skilled senior level Data Engineer in Hyderabad or remote India for a large AAA gaming company on a contract for 3 months with possibility of extension or conversion...Show moreLast updated: 30+ days ago
  • Promoted
Senior Data Engineer

Senior Data Engineer

The Mastermind GroupDindigul, IN
Remote (India) – Occasional travel to Riyadh, KSA.Sunday–Thursday preferred (9 AM – 5 PM IST).About The Mastermind Group (TMG). Software Asset Management (SAM).Hardware Asset Management (HAM).We're ...Show moreLast updated: 30+ days ago
  • Promoted
Data Engineer

Data Engineer

RevXDindigul, IN
RevX helps app businesses acquire and reengage users via programmatic to retain, monetize, and accelerate revenue.We're all about taking your app businesses to a new growth level.We rely on data sc...Show moreLast updated: 30+ days ago
  • Promoted
Senior Data Engineer

Senior Data Engineer

SAIVA AIDindigul, IN
We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 23 days ago
  • Promoted
Data Engineer

Data Engineer

KPG99 INCMadurai, IN
Essential Skills and requirements.Advanced proficiency in SQL including stored procedures, views and functions.Solid knowledge of data warehousing principles. Experience working with large data ware...Show moreLast updated: 1 day ago
  • Promoted
Data Engineer

Data Engineer

HCLTechMadurai, Tamil Nadu, India
Azure Data Lead with Azure Data Factory (ADF),Azure Databricks,.The Technical Lead will be responsible for overseeing and leading projects related to Azure Data Factory (ADF), Azure Databricks, SQL...Show moreLast updated: 19 days ago
  • Promoted
Data Engineer

Data Engineer

Sikich IndiaDindigul, IN
Sikich India is seeking an experienced Data Engineer to join our Data & AI practice.You will design, build, and optimize end-to-end data solutions using Microsoft’s data platforms, including Micros...Show moreLast updated: 30+ days ago
  • Promoted
Data Engineer

Data Engineer

The Transformation GroupMadurai, IN
The ideal candidate should possess experience with ETL, data and reporting; and contributing code to deliver new features and functionality. Day-to-day you will collaborate with engineers, product m...Show moreLast updated: 11 days ago
  • Promoted
Data Engineer

Data Engineer

Kumaran SystemsDindigul, IN
Strong hands-on experience in Databricks.Proven expertise in building and managing data ingestion pipelines.Exposure to Databricks data ingestion jobs along with incident management frameworks.Expe...Show moreLast updated: 23 days ago
  • Promoted
Data Engineer

Data Engineer

KeasisMadurai, IN
We are seeking a Data Engineer with strong Apache NiFi expertise to design and implement pipelines that move and transform data from Cloudera (HDFS / Hive / Impala) into Apache Iceberg tables, with dow...Show moreLast updated: 1 day ago
  • Promoted
Data Engineer

Data Engineer

Vriba SolutionsMadurai, IN
Design, develop & maintain ETL / ELT pipelines.Ingest & transform data from APIs, DBs, files, streams.Build real-time & batch processing solutions. Data validation, quality & cleansing.Translate busin...Show moreLast updated: 11 days ago
  • Promoted
Senior Data Engineer

Senior Data Engineer

USEReadyDindigul, IN
Designation : Senior Data Engineer.As a part of the Data Warehouse Team, implement technology improvements to Enterprise Data Warehouse environment. In this role, you will be a key contributor to the...Show moreLast updated: 30+ days ago