This job offer is not available in your country.

Big Data Engineer (Spark, SparkSQL)...

Anicalls (Pty) Ltdchennai, India

18 hours ago

Job description

2 to 5 years hands-on Experience on Spark Core, Spark-SQL, Scala-Programming, and Streaming datasets in Big Data platform

Should have extensive working experience in Hive and other components of the Hadoop ecosystem (HBase, Zookeeper, Kafka, and Flume)

Should be able to understand the complex transformation logic and translate them to Spark-SQL queries.

Unix Shell Scripting and setting up CRON

Familiar with Data Warehouse concepts and Change Data Capture (CDC and SCD Types)

Should have worked on Cloudera distribution framework, Oozie Workflow (or any Scheduler), Jenkins (or any version controller).

Prior Experience in Consumer Banking Domain is an advantage.

Prior Experience in the agile delivery method is an advantage.

Excellent understanding of technology life cycles and the concepts and practices required to build big data solutions

Core Java skillset is an added advantage. Airflow is an added advantage.

Good Knowledge and Experience in any database (Teradata or Data Lake or Oracle or SQL Server) is a plus.

Create a job alert for this search

Big Data Engineer • chennai, India

Related jobs

Promoted

Senior Data Engineer

Eucloid Data SolutionsChennai, Tamil Nadu, India

Eucloid is looking for a skilled Data Engineer to join our team and contribute to the design, development, and optimization of data frameworks pipelines using Python, Elasticsearch, SQL DB and Queu...Show moreLast updated: 9 days ago

Promoted

Senior Data Engineer

Otomeyt AIChennai, IN

We are seeking a highly skilled 7+.The ideal candidate will have strong technical expertise in Azure, Data Engineering tools, and advanced ETL design along with excellent communication and problem-...Show moreLast updated: 19 days ago

Promoted

Senior Data Engineer

DeltacubesChennai, IN

Build and maintain scalable ETL / ELT pipelines.Work with Snowflake and BigQuery for data storage.Implement orchestration with Airflow or Prefect. Integrate data workflows with Python.Optimize data pi...Show moreLast updated: 16 days ago

Promoted

Lead Cloud Data Engineer - PySpark / SQL

upGradChennai

About the Job : We're hiring an cloud data engineering (preferably Azure) data pipelines and Spark.Work with Databricks platform using Spark for big data processing and an...Show moreLast updated: 30+ days ago

Promoted

Big Data Engineer - Hadoop / Spark

ImpacteersChennai

Position Overview : We are looking for a skilled and detail-oriented Big Data Engineer to design, develop, and maintain scalable data pipelines and architectures.The ...Show moreLast updated: 14 days ago

Promoted

Lead Data Engineer

Persistent SystemsChennai, Tamil Nadu, India

We are conducting an in-person hiring drive for the position of Mlops Engineer in Chennai on 6th Sep 2025.Interview Location is mentioned below : . Chennai : Olympia Pinnacle, #1, S.Old Mahabalipuram R...Show moreLast updated: 27 days ago

Promoted

Data Engineer

Otomeyt AIChennai, IN

We are seeking a highly skilled 5+.The ideal candidate will have strong technical expertise in Azure, Data Engineering tools, and advanced ETL design along with excellent communication and problem-...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

Innodata Inc.Chennai, IN

CI / CD practices, Databricks (Spark), Python, Github and SQL.The ideal candidate should have hands-on expertise in building and automating data pipelines, managing multi-environment deployments, and...Show moreLast updated: 27 days ago

Promoted

Lead Data Engineer - Scala / Spark

Ixceed SolutionsChennai

About the Role : We are seeking a highly skilled and experienced Lead Data Engineer / Scala Developer with strong hands-on expertise in Scala program...Show moreLast updated: 21 days ago

Promoted

Senior Big Data Engineer

VeltrisChennai, IN

Veltris is a Digital Product Engineering Services partner committed to driving technology-enabled transformation across enterprises, businesses, and industries. We specialize in delivering next-gene...Show moreLast updated: 16 days ago

Promoted

Big Data Developer

Tata Consultancy ServicesChennai, Tamil Nadu, India

JAVA,SCALA using Maven, Apache Spark Frameworks and Unix Shell scripting.Should be comfortable with Unix File system as well as HDFS commands. Should have worked on query languages such as Oracle SQ...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

ACL Digitalchennai, tamil nadu, in

Design, develop, and optimize Spark-based data pipelines on Databricks for large-scale data processing.Design, develop, and optimize AWS pipeline as applicable. Implement and manage GitHub asset bun...Show moreLast updated: 30+ days ago

Promoted

Big Data Engineer - SQL / Python

ANB IT Solutions Pvt LtdChennai

Job Description : - Minimum of 7+ years of progressive experience in Data Engineering, with a proven track record in pi...Show moreLast updated: 12 days ago

AWS Data Engineer

Epergne SolutionsChennai, Tamil Nadu, India

Quick Apply

Pune / Hyderabad / Chennai / Mysuru / Bhubaneswar / Mangalore / Trivandrum / Chandigarh / Jaipur / Nagpur / Indore / Gurgaon. Job Roles & Responsibilities.Design, build, and optimize data pipelines and scalable data ...Show moreLast updated: 13 days ago

Promoted

Data Engineer

TechVeritochennai, tamil nadu, in

You will play a critical role in designing, building, and optimizing data workflows that enable scalable analytics and real-time insights. The ideal candidate is hands-on, detail-oriented, and passi...Show moreLast updated: 1 day ago

Promoted

Scala Big Data Lead Engineer - 7 YoE - Immediate Joiner - Any UST Location

USTchennai, tamil nadu, in

CTC, notice period, and current location details to.Apache Hadoop, Airflow, Kubernetes, and Containers.Troubleshoot Hadoop log files and work with. Required Skills & Qualifications : .Scala, Spark, Py...Show moreLast updated: 30+ days ago

Promoted

Data Engineer - ETL / PySpark

Ixceed SolutionsChennai

Role : Data Engineer Location : Chennai (Hybrid).Job Description : Key Responsibilities : Show moreLast updated: 15 days ago

Promoted

Data Engineer

ImpetusChennai, Tamil Nadu, India

Bigdata, Pyspark, Python ,Hadoop / HDFS; Spark;.Develops and maintains scalable data pipelines to support continuing increases in data volume and complexity. Collaborates with analytics and business...Show moreLast updated: 25 days ago

Promoted

Senior Data Engineer

SAIVA AIChennai, IN

We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 9 days ago

Promoted

Big Data Engineer - Scala / Spark / Python

Idyllic Services Pvt LtdChennai

Job Title : Big Data Engineer - Scala Experience : 7 to 10Years (Minimum 3+ years in Scala) Notice Period : ...Show moreLast updated: 23 days ago