Talent.com
This job offer is not available in your country.
Glencore - Senior Data Platform Engineer - CI / CD Pipeline

Glencore - Senior Data Platform Engineer - CI / CD Pipeline

Glencore India Pvt LtdChennai
30+ days ago
Job description

Senior Data Platform Engineer

Experience 10+ Years.

THE ROLE :

  • As a Senior Data Platform Engineer, you will lead the design and governance of scalable, secure, and compliant data architectures on Microsoft Azure.
  • You will work across oil trading, refining, and retail fuel operations to align cloud data strategies with business goals, regulatory requirements, and industry best practices.
  • You will play a hands-on technical role within the data engineering team, interfacing with the CloudOps team in London to design and implement cost effective solutions across our suite of Azure products adhering to best practice principles.
  • Experience in DevOps is highly desirable as you will be responsible for designing CI / CD pipelines using Github Actions and containerization technologies.
  • In addition, the role will involve exploring new technologies and navigating implementation within Glencores private Azure tenant, according to the guidelines enforced at Group level.
  • With these in mind, you will play an advisory role across the Assets, assisting with design and implementation, and on-premises to cloud migrations as required.
  • This role sits within the Oil Assets IT team, reporting to the Data Engineering Team Lead.

PROFILE :

  • Experienced IT professional with over 10 years of experience, including 24 years leading cloud-native data architecture initiatives.
  • Proven track record in designing and delivering scalable, end-to-end data platform solutions across hybrid and multi-cloud environments.
  • Adept at driving best practices in data engineering, modernizing legacy systems, and advising stakeholders on strategic data platform decisions.
  • Strong focus on performance, reliability, and governance across enterprise-grade data ecosystems.
  • Skilled in setting up foundational components, enforcing engineering standards, and enabling team-wide productivity through automation, CI / CD, and reusable frameworks.
  • Plays a key leadership role in a growing data engineering team, working alongside senior and junior data engineers.
  • RESPONSIBILITIES :

    Platform Engineering & Architecture :

  • Design and implement scalable, secure, and cost-effective data platform solutions using Azure Synapse Analytics, Azure Data Lake, and related services.
  • Translate business and regulatory requirements into cloud-native data architectures.
  • Contribute to the architectural roadmap, including the adoption and integration of Microsoft Fabric for unified data experiences.
  • Define and enforce platform standards, patterns, and reusable components to accelerate delivery across the team.
  • CI / CD, Automation & Containerization
  • Lead the implementation of CI / CD pipelines for data engineering workflows using Azure DevOps and Git.
  • Automate infrastructure provisioning and deployment using Infrastructure as Code (e.g., Bicep, Terraform).
  • Develop and maintain containerized data services using Docker, and orchestrate deployments using Kubernetes (AKS).
  • Establish automated testing, validation, and monitoring practices to ensure data quality and platform reliability.
  • Team Enablement & Collaboration :

  • Collaborate closely with the other Senior Engineer to co-lead technical direction and ensure consistency across projects.
  • Mentor and support data engineers and developers, providing guidance on best practices, code reviews, and technical upskilling.
  • Work cross-functionally with Data Architects, Analysts, and Product Owners to translate business needs into robust data solutions.
  • Governance, Security & Operations :

  • Implement data governance and lineage practices using Microsoft Purview and Fabric-native tools.
  • Ensure compliance with security and privacy standards through RBAC, encryption, and secure data handling.
  • Monitor platform performance and proactively address operational issues and bottlenecks.
  • Innovation & Continuous Improvement :

  • Stay current with Microsofts evolving data ecosystem, including Fabric, and evaluate new tools and services for team adoption.
  • Drive continuous improvement in engineering workflows, platform scalability, and developer experience.
  • Maintain understanding of the latest trends and advancements in AI and machine learning, and advise on incorporation of these innovations across the department, alongside governance.
  • COMPETENCIES :

  • Bachelors degree in Computer Science, Information Technology, or equivalent experience.
  • Excellent understanding of core data modelling concepts and techniques.
  • Ability to manage stakeholders and develop effective relationships across different cultures.
  • Cloud & Platform :

  • Azure Synapse Analytics, Microsoft Fabric (emerging), Azure Data Lake Gen2, Azure SQL, Azure DatabricksReal-Time & IoT : Azure Event Hubs, Azure Stream Analytics, Azure IoT Hub.
  • Data Engineering :

  • Azure Data Factory, Synapse Pipelines, Spark, Delta Lake, T-SQL, Python.
  • Governance and Security :

  • Microsoft Purview, Key Vault, RBAC, Data Masking, Encryption.
  • Monitoring and Observability :

  • Azure Monitor, Log Analytics, Application Insights.
  • Governance and Security :

  • Microsoft Purview, Key Vault, RBAC, Data Masking, Encryption.
  • Monitoring and Observability :

  • Azure Monitor, Log Analytics, Application Insights.
  • DevOps and Automation :

  • Azure DevOps, Git / GitHub, CI / CD, Bicep, Terraform, YAML pipelines.
  • Containerization and Orchestration :

  • Docker, Kubernetes (AKS), Helm.
  • DESIRABLE :

  • Exposure to DataOps practices and awareness of advanced Data Architecture patterns.
  • Certifications : Azure Solutions Architect, Azure Data Engineer Associate, Azure DevOps Engineer Expert.
  • Visualisation & BI : Power BI, Azure Analysis Services.
  • Machine Learning : Azure ML.
  • (ref : hirist.tech)

    Create a job alert for this search

    Senior Data Engineer • Chennai

    Related jobs
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Eucloid Data SolutionsChennai, Tamil Nadu, India
    Eucloid is looking for a skilled Data Engineer to join our team and contribute to the design, development, and optimization of data frameworks pipelines using Python, Elasticsearch, SQL DB and Queu...Show moreLast updated: 8 days ago
    • Promoted
    Senior GCP Data Engineer

    Senior GCP Data Engineer

    GetronicsChennai, Tamil Nadu, India
    We have multiple opportunities for.Data Analytics team is seeking a GCP Data Engineer to create, deliver, and support custom data products, as well as enhance / expand team capabilities.They will wor...Show moreLast updated: 30+ days ago
    • Promoted
    Azure Data Engineer

    Azure Data Engineer

    Tata Consultancy Serviceschennai, tamil nadu, in
    Greetings from Tata Consultancy Services!!.Azure Data Factory (ADF) pipelines and Polybase.MS with minimum 3 years of relevant IT-experience post Qualification. Only Full-Time courses would be consi...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    DeltacubesChennai, IN
    Build and maintain scalable ETL / ELT pipelines.Work with Snowflake and BigQuery for data storage.Implement orchestration with Airflow or Prefect. Integrate data workflows with Python.Optimize data pi...Show moreLast updated: 15 days ago
    • Promoted
    Cloud Data Engineer

    Cloud Data Engineer

    Lemongrasschennai, tamil nadu, in
    Lemongrass is a software-enabled services provider, synonymous with SAP on Cloud, focused on delivering superior, highly automated Managed Services to Enterprise customers.Our customers span multip...Show moreLast updated: 20 days ago
    • Promoted
    Lead / Senior Data Engineer - ETL / AWS

    Lead / Senior Data Engineer - ETL / AWS

    ResourcetreeChennai
    We are seeking a highly skilled and motivated Lead / Senior Data Engineer to join our Data Integration and Analytics team. The ideal candidate will bring deep expertise in AWS Redshift, with strong kn...Show moreLast updated: 4 days ago
    • Promoted
    GCP Data Engineer

    GCP Data Engineer

    MastekChennai, Tamil Nadu, India
    Work closely with our data science team to help build complex algorithms that provide unique insights into our data.Use agile software development processes to make iterative improvements to our ba...Show moreLast updated: 24 days ago
    • Promoted
    Glencore - Senior Data Engineer - Azure Cloud

    Glencore - Senior Data Engineer - Azure Cloud

    Glencore India Pvt LtdChennai
    THE ROLE : As a Senior Data Engineer, you will be part of a team who design, build and maintain scalable, secure, and high-performance data solutions on Microsoft Azure.You wi...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Innodata Inc.Chennai, IN
    CI / CD practices, Databricks (Spark), Python, Github and SQL.The ideal candidate should have hands-on expertise in building and automating data pipelines, managing multi-environment deployments, and...Show moreLast updated: 25 days ago
    • Promoted
    Data Engineer

    Data Engineer

    EXLChennai, India
    Collaborate with project stakeholders (client) to identify product and technical requirements.Develop, implement, and tune large-scale distributed systems and pipelines that process large volume of...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    HISH IT SERVICESChennai, IN
    Location : Remote(Banglore,Chennai,Pune).Pay : 14LPA - 18 LPA(Based on Experience).Timings : A couple of hours overlap with EST, as the client is Canada-based (till 12AM IST).Start Date : 20th Octob...Show moreLast updated: 7 days ago
    • Promoted
    Backend and Data Pipeline Engineer

    Backend and Data Pipeline Engineer

    JRD SystemsChennai, IN
    Job Role : Backend and Data Pipeline Engineer - Python.We’re investing in technology to develop new products that help our customers drive their growth and transformation agenda.These include new da...Show moreLast updated: 4 days ago
    • Promoted
    AWS Data Engineer - Redshift, Pyspark, Glue

    AWS Data Engineer - Redshift, Pyspark, Glue

    Tata Consultancy ServicesChennai, Tamil Nadu, India
    Greetings from TATA Consultancy Services.Pls find the Job Description below.Good hands-on experience in Python programming and Pyspark. Data Engineering experience using AWS core services (Lambda, G...Show moreLast updated: 8 days ago
    • Promoted
    Data Engineer - Google Cloud Platform

    Data Engineer - Google Cloud Platform

    SYNCARP GLOBE LLPChennai
    Senior GCP Data Engineer Overview : We are seeking an experienced Senior Data Engineer with strong hands-on expertise in Google Cloud Platform (GCP), BigQuery (BQ), a...Show moreLast updated: 26 days ago
    • Promoted
    Data Platform Engineer

    Data Platform Engineer

    Intuitive.CloudChennai, IN
    With the reputation of being a.Digital Transformation challenges across following Intuitive Superpowers : .Application & Database Modernization. Platform Engineering (IaC / EaC, DevSecOps & SRE).Cloud N...Show moreLast updated: 5 days ago
    • Promoted
    Data Engineer

    Data Engineer

    INFEC Serviceschennai, tamil nadu, in
    Design, develop, and optimize data pipelines and ETL processes on GCP or Azure.Work with structured and unstructured data, integrating sources such as databases, APIs, and streaming platforms.Imple...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Manuh TechnologiesChennai, IN
    Strong proficiency in Python programming (Pandas, NumPy, PySpark, or similar).Hands-on experience with Dask for large-scale distributed data processing. Proven expertise as a Data Modeler (conceptua...Show moreLast updated: 30+ days ago
    • Promoted
    GCP Big Data Engineer

    GCP Big Data Engineer

    Talentmaticschennai, tamil nadu, in
    The ideal candidate will bring strong leadership capabilities, technical depth, and a proven track record of delivering end-to-end big data solutions in cloud environments.Lead and mentor teams in ...Show moreLast updated: 5 days ago