Talent.com
This job offer is not available in your country.
Petrofac - Data Engineer - R / Python

Petrofac - Data Engineer - R / Python

PetrofacChennai
30+ days ago
Job description

About the Role :

We are seeking an experienced Data Engineer with a strong background in building scalable data platforms, big data engineering, and end-to-end data lifecycle management. The ideal candidate will have deep expertise in data governance, modelling, and architecture, with hands-on experience in modern data platforms, pipelines, and analytics tools. This role involves designing and maintaining robust data systems that empower business stakeholders to drive data-driven decision-making at scale.

Key Responsibilities :

Data Architecture & Governance :

  • Architect and define end-to-end data flows for Big Data / Data Lake use cases.
  • Implement best practices in data governance, data quality, master data management, and data security.
  • Collaborate with enterprise / domain architects to align data solutions with enterprise roadmaps.
  • Participate in Technical Design Authority forums to influence and validate architectural decisions.

Pipeline Development & Data Engineering :

  • Design, develop, and optimize scalable ETL / ELT pipelines across diverse data sources (cloud, on-premises, SQL / NoSQL, APIs).
  • Automate data ingestion and transformation processes, ensuring performance, scalability, and reliability.
  • Implement real-time, batch, and scheduled data ingestion using tools like Apache Sqoop, Flume, Kinesis, Logstash, FluentD.
  • Work with Databricks, Spark, Hive, Hadoop, Azure Data Factory, Scala, Python, R to deliver robust data processing workflows.
  • Optimize pipeline performance by analyzing physical / logical execution plans.
  • Data Management & Analytics Enablement :

  • Collaborate with analytics teams to improve data models feeding BI tools (e.g., Power BI, Tableau).
  • Build and maintain OLAP cubes to address BI limitations and enable complex business analysis.
  • Deliver data cleansing, validation, and enrichment solutions to ensure data accuracy.
  • Lead initiatives in data mining, statistical analysis, and advanced data modelling (Star / Snowflake schemas, SCD2).
  • Operations & Performance Optimization :

  • Estimate and optimize cluster / core sizes for Databricks clusters and Analysis Services.
  • Deploy and maintain CI / CD DevOps pipelines across development, staging, and production environments.
  • Monitor, troubleshoot, and enhance system performance, ensuring optimal data ingestion and storage.
  • Conduct continuous audits of data systems to identify gaps, performance bottlenecks, or security loopholes.
  • Leadership & Collaboration :

  • Act as a coach / mentor to junior data engineers, providing technical guidance and enforcing best practices.
  • Collaborate cross-functionally with business stakeholders, analytics teams, and engineering squads to deliver business outcomes.
  • Allocate and track tasks across the team, reporting progress and deliverables to management.
  • Essential Qualifications & Skills :

    Education : Bachelors degree in Computer Science, Engineering, or related field (Masters preferred).

    Experience :

  • 10+ years in data analytics platforms, ETL / ELT transformations, SQL programming.
  • 5+ years hands-on experience in Big Data Engineering, Data Lakes, Distributed Systems.
  • Technical Expertise :

  • Strong proficiency in Hadoop ecosystem (HDFS, Hive, Sqoop, Oozie, Spark Core / Streaming).
  • Programming in Scala, Java, Python, Shell scripting.
  • Deep experience with Azure Data Platform (Azure SQL DB, Data Factory, Cosmos DB).
  • Database expertise : Oracle, MySQL, MongoDB, Presto.
  • Data ingestion / extraction using REST API, ODATA, JSON, XML, Web Services.
  • Core Skills :

  • Strong foundation in data modelling, warehousing, and architecture principles.
  • Hands-on experience with ETL tools and best practices.
  • Solid understanding of data security (encryption, tunneling, access control).
  • Proven ability in troubleshooting and performance optimization.
  • (ref : hirist.tech)

    Create a job alert for this search

    Data Engineer • Chennai

    Related jobs
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Eucloid Data SolutionsChennai, Tamil Nadu, India
    Eucloid is looking for a skilled Data Engineer to join our team and contribute to the design, development, and optimization of data frameworks pipelines using Python, Elasticsearch, SQL DB and Queu...Show moreLast updated: 8 days ago
    • Promoted
    Eucloid - Senior / Lead Data Engineer - Python

    Eucloid - Senior / Lead Data Engineer - Python

    Eucloid Data Solutions Pvt LtdChennai
    Job Description : Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will suppor...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    DeltacubesChennai, IN
    Build and maintain scalable ETL / ELT pipelines.Work with Snowflake and BigQuery for data storage.Implement orchestration with Airflow or Prefect. Integrate data workflows with Python.Optimize data pi...Show moreLast updated: 15 days ago
    • Promoted
    Azure Data Engineer - Python / PySpark

    Azure Data Engineer - Python / PySpark

    GALAXY I TECHNOLOGIES INCChennai
    Role Description : We are seeking a highly skilled Azure Data Engineer to join our team.The ideal candidate will be responsible for designing, building, and maintaini...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Innodata Inc.Chennai, IN
    CI / CD practices, Databricks (Spark), Python, Github and SQL.The ideal candidate should have hands-on expertise in building and automating data pipelines, managing multi-environment deployments, and...Show moreLast updated: 26 days ago
    • Promoted
    Data Engineer Lead

    Data Engineer Lead

    Ashley Global Capability CenterChennai, Tamil Nadu, India
    Bachelor's or master’s degree in Computer Science, Mathematics, or related field.Years of experience in developing technology solutions, integrating, or creating AI and machine learning models.Know...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    EXLChennai, India
    Collaborate with project stakeholders (client) to identify product and technical requirements.Develop, implement, and tune large-scale distributed systems and pipelines that process large volume of...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer-RTS

    Data Engineer-RTS

    ValueLabsChennai, Tamil Nadu, India
    Notice period -Max 15 days to 30 days • •.Minimum 5+ years of development and design experience in Java / Scala / Python with Flink, Beam (or Spark Streaming using Real Time data and not batch data) and ...Show moreLast updated: 7 days ago
    • Promoted
    Data Engineer

    Data Engineer

    HISH IT SERVICESChennai, IN
    Location : Remote(Banglore,Chennai,Pune).Pay : 14LPA - 18 LPA(Based on Experience).Timings : A couple of hours overlap with EST, as the client is Canada-based (till 12AM IST).Start Date : 20th Octob...Show moreLast updated: 7 days ago
    • Promoted
    Data Engineer - ETL / Python

    Data Engineer - ETL / Python

    TalentOrient Pvt. Ltd.Chennai
    JOB OBJECTIVE : We are seeking a dynamic professional to fill in our Data Engineering role who will be responsible for designing, developing, and maintaining the data architect...Show moreLast updated: 15 days ago
    • Promoted
    Data Engineer

    Data Engineer

    IntraEdgeChennai, Tamil Nadu, India
    Onsite / Hybrid (as applicable).We are looking for a highly skilled and motivated.The ideal candidate will have strong expertise in. DBT, Snowflake, CI / CD pipelines.This role requires individuals wi...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Canopus Infosystems - A CMMI Level 3 CompanyChennai, IN
    Python expertise and hands-on experience in handling large datasets, data cleaning, analysis, and visualization.The ideal candidate should be capable of building data pipelines, performing web scra...Show moreLast updated: 18 days ago
    • Promoted
    Data Engineer

    Data Engineer

    AceolutionChennai, IN
    We are looking for a freelancer to engage with us for 20-40 hours per week.Kindly find the JD below for your reference.Design, develop, and maintain scalable data pipelines and workflows.Work exten...Show moreLast updated: 8 days ago
    • Promoted
    Data Engineer

    Data Engineer

    ImpetusChennai, Tamil Nadu, India
    Bigdata, Pyspark, Python ,Hadoop / HDFS; Spark;.Develops and maintains scalable data pipelines to support continuing increases in data volume and complexity. Collaborates with analytics and business...Show moreLast updated: 24 days ago
    • Promoted
    Data Engineer

    Data Engineer

    INFEC Serviceschennai, tamil nadu, in
    Design, develop, and optimize data pipelines and ETL processes on GCP or Azure.Work with structured and unstructured data, integrating sources such as databases, APIs, and streaming platforms.Imple...Show moreLast updated: 5 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Manuh TechnologiesChennai, IN
    Strong proficiency in Python programming (Pandas, NumPy, PySpark, or similar).Hands-on experience with Dask for large-scale distributed data processing. Proven expertise as a Data Modeler (conceptua...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    VAANTECHChennai, Tamil Nadu, India
    Immediate Joiner (or within 15 days).Includes Night Shifts (US Shift).Preferred Candidates : From Chennai.Data Pipeline Optimization & Tuning. Hadoop Infra / Cloud Platforms.Lead and mentor a team of...Show moreLast updated: 29 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Bahwan CyberTekchennai, tamil nadu, in
    We are seeking a skilled and motivated Data Engineer with 5–8 years of experience in building scalable data pipelines using Python, PySpark, and AWS services. The ideal candidate will have hands-on ...Show moreLast updated: 18 days ago