This job offer is not available in your country.

Data Engineer - ETL / PySpark

Intelliserve Konsulting Pvt. Ltd.Chennai

30+ days ago

Job description

Job Description :

One should be responsible for maintaining, expanding, and optimizing our data warehouse, data pipeline, and data products. The data engineer will support multiple stakeholders, including Business users, developers, database architects, data analysts, and data scientists to ensure an optimal data delivery. The ideal candidate should possess strong technical abilities to solve complex problems on Data and Required :

Should have a minimum 4+ years in Data Engineering, Data Analytics platform.
Should have strong hands-on design and engineering background in AWS, across a wide range of AWS services with the ability to demonstrate working on large engagements.
Should be involved in Requirements Gathering and transforming them to into Functionally and technical design.
Maintain and optimize the data infrastructure required for accurate extraction, transformation, and

loading of data from a wide variety of data sources.

Design, build and maintain batch or real-time data pipelines in production.

Develop ETL / ELT Data pipeline (extract, transform, load) processes to help extract and manipulate data from multiple sources.

Automate data workflows such as data ingestion, aggregation, and ETL processing and should have good experience with different types of data ingestion techniques : File-based, API-based, streaming data sources (OLTP, OLAP, ODS etc) and heterogeneous databases.

Prepare raw data in Data Warehouses into a consumable dataset for both technical and non-technical stakeholders.

Strong experience and implementation of Data lakes, Data warehousing, Data Lake housing architectures.

Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures.

Monitor data systems performance and implement optimization strategies.

Leverage data controls to maintain data privacy, security, compliance, and quality for allocated areas of ownership.

Experience of AWS tools (AWS S3, EC2, Athena, Redshift, Glue, EMR, Lambda, RDS, Kinesis, DynamoDB, Quick sight etc.).

Strong experience with Python, SQL, pySpark, Scala, Shell Scripting etc.

Strong experience with workflow management & Orchestration tools (Airflow,

Should hold decent experience and understanding of data manipulation / wrangling techniques.

Demonstrable knowledge of applying Data Engineering best practices (coding practices to DS, unit testing, version control, code review).

Big Data Eco-Systems, Cloudera / Hortonworks, AWS EMR etc.

Snowflake Data Warehouse / Platform.

Streaming technologies and processing engines, Kinesis, Kafka, Pub / Sub and Spark Streaming.

Experience of working with CI / CD technologies, Git, Jenkins, Spinnaker, Ansible etc

Experience building and deploying solutions to AWS Cloud.

Good experience on NoSQL databases like Dynamo DB, Redis, Cassandra, MongoDB, or Neo4j etc.

Experience with working on large data sets and distributed computing (e.g., Good to have working knowledge on Data Visualization tools like Tableau, Amazon Quick sight, Power BI, QlikView etc.

Experience in Insurance domain preferred.

ref : hirist.tech)

Create a job alert for this search

Data Engineer • Chennai

Related jobs

Promoted

Data Engineer - ETL

Cynosure Corporate SolutionsChennai

Job Title : Data Engineer Location : Guindy, Chennai Experience : 3-6 years US Shift : 11 : 00am - 8 : 00PM We are lo...Show moreLast updated: 30+ days ago

Promoted

Azure Data Engineer - PySpark / Scala

Gig ConsultantsChennai

Must-have skills : - Cloud certified in one of these categories - Azure Data Engineer ...Show moreLast updated: 26 days ago

Promoted

AWS Data Engineer - ETL / PySpark / Python

ChangeleadersChennai

Position : AWS Data Engineer Experience : 5+ Yrs Location : Bangalore, Hyderabad, Chennai, Pune, Gurgaon <...Show moreLast updated: 15 days ago

Promoted

Data Engineer - Typescript / Pyspark

NP GroupChennai, IN

Data Engineer - Python, Pyspark & Typescript.Rate : $20 - $25 per hour ($3,000 - $4,000 per month).We have an immediate requirement for an experienced Data Engineer to join the global engineering te...Show moreLast updated: 11 days ago

Promoted

Azure Data Engineer

RandomTreesChennai, IN

We are seeking a skilled and experienced.The ideal candidate will have a strong background in building and optimizing data pipelines and data sets, utilizing Azure Data Factory, Databricks, PySpark...Show moreLast updated: 26 days ago

Promoted

Azure Data Engineer - Python / PySpark

AceNetChennai

About Us : AceNet Consulting is a fast-growing global business and technology consulting firm specializing in business strategy, digital transformation, technology co...Show moreLast updated: 29 days ago

Promoted

Senior Data Engineer - Python / PySpark

DexianChennai

Python Data : 6 to 12 Years Skills : Python, Pyspark, Data warehousing, Data pipeline design Show moreLast updated: 3 days ago

Promoted

Senior Data Engineer - ETL / PySpark

Renovision Automation Services Pvt.Ltd.Chennai

Essential Duties and Responsibilities : - This section contains a list of five to eight primary responsibilities of this role th...Show moreLast updated: 19 days ago

AWS Data Engineer(Chennai, Indore,Pune)

Tata Consultancy ServicesChennai, Tamil Nadu, India

JOB LOCATION : Chennai, Indore , Pune.Good knowledge and experience in Pyspark for forming complex Transformation logic.Secondary : AWS Glue , Airflow. A minimum of 3 + years' experience and underst...Show moreLast updated: 14 days ago

Promoted

Data Engineer – ETL

Demand AIChennai, IN

ETL development and data engineering.Looking for a Data Engineer specializing in ETL processes to design and implement data pipelines that support data integration and analytics initiatives.Design,...Show moreLast updated: 22 days ago

Promoted

Data Engineer - PySpark / Hive

Pan Asia HR SolutionsChennai

Job Title : Data Engineer PySpark & Hive Location : Chennai / Pune / Bengaluru / Noida / Hyderabad Work Mode : ...Show moreLast updated: 7 days ago

Promoted

Big Data Engineer - Hadoop / Spark

Evoke HR SolutionsChennai

Primary Skill : Hadoop, Spark, Hive, ETL, Pyspark, Python.Secondary Skill : Integration and continuous deployment (CI / CD) pipelines. Location : Bangalore / Chennai (Second Round Face to Face).Notice...Show moreLast updated: 13 days ago

Promoted

Senior Data Engineer - Python / PySpark

Vtricks TechnologiesChennai

Job Title : Senior Data Engineer - Azure | ADF | Databricks | PySpark | AWS Location : Bangalore, Hyderabad, Chennai (Hybrid Mode) Experience Required : 5+...Show moreLast updated: 15 days ago

Promoted

Data Engineer - ETL / PySpark

Axim DigitechChennai

Key Skill : Pyspark, Cloudera Data Platfrorm, Big data Hadoop, Hive, Kafka Responsibilities : <...Show moreLast updated: 17 days ago

Senior ETL Developer(AWS PYSPARK)

Innova ESIChennai, Tamil Nadu, India

Location : Chennai (F2F Interview 12 / July / 2025).Skills required Senior ETL Developer : .Mandatory skills (8+ Years of experience in ETL development with 4+ Years on AWS Pyspark scripting).Experience...Show moreLast updated: 2 days ago

Data Engineer

ValueLabsChennai, Tamil Nadu, India

PySpark : Advanced proficiency in PySpark, including working with RDDs, DataFrames, and optimization techniques.Cloudera Data Platform : Strong experience with Cloudera Data Platform (CDP) components...Show moreLast updated: 11 days ago

Promoted

Data Engineer - Python / PySpark

sphere ITChennai

Responsibilities : - Ability to write clean, maintainable, and robust code in Python - Understanding and expertise of software engineeri...Show moreLast updated: 22 days ago

Promoted

Azure Data Engineer - PySpark / DataLake

SSIChennai

Responsibilities : - Proficiency in Azure Data Factory, Azure Databricks (including Spark and Delta Lake), and other Azure data services.Strong pro...Show moreLast updated: 28 days ago