Talent.com
Platform Data Engineer

Platform Data Engineer

Insight GlobalGreater Delhi Area, India
7 days ago
Job description

Key Responsibilities

  • Administer and optimize Azure Databricks, including clusters, jobs, and user access.
  • Manage Snowflake environments (warehouses, roles / permissions, performance tuning, cost optimization).
  • Design, implement, and monitor Azure Data Factory (ADF) pipelines for batch and streaming workloads.
  • Enable data movement and replication across global environments.
  • Orchestrate pipelines using ADF, Apache Airflow, and automation frameworks.
  • Automate processes and reduce manual effort with Python, PySpark, SQL, and PowerShell.
  • Work with Azure Data Lake Storage (ADLS), Function Apps, and Key Vault for secure and efficient data operations.
  • Ensure compliance with security, governance, and cost management best practices.
  • Collaborate with Data Engineering, DNA, and Data Science teams to ensure high availability and reliable data delivery.
  • Troubleshoot platform issues, implement fixes, and contribute to runbooks and documentation.
  • Partner with architects and engineers to continuously improve platform scalability, reliability, and efficiency.

Required Skills and Experience

  • 5+ years of experience in data platform engineering / administration.
  • Strong expertise in Azure Databricks, Snowflake, and Azure Data Factory.
  • Experience on Azure Synapse, Event Hubs, or other Azure data services.
  • Hands-on experience with data pipelines, orchestration, and automation.
  • Proficiency in SQL, Python, PySpark, and scripting (PowerShell / Bash).
  • Experience with Apache Airflow for scheduling and orchestration.
  • Strong understanding of Azure services including ADLS, Function Apps, Key Vault, Networking, and Monitoring.
  • Familiarity with CI / CD pipelines and Infrastructure as Code (Terraform / ARM).
  • Solid knowledge of data governance, security, and cost optimization practices

    • Soft Skills

  • Strong problem-solving and analytical skills.
  • Ability to work independently and in a global, collaborative team environment.
  • Effective communication skills for technical and non-technical stakeholders.
  • Proactive and adaptable mindset with focus on continuous improvement.

    • Nice to Have

  • Experience in DevOps practices for data engineering.
  • Knowledge of data governance and compliance frameworks (e.g., GDPR, PCI).
  • Familiarity with agile methodologies (JIRA, Confluence).
  • Microsoft Azure certifications (e.g., DP-203, AZ-204, AZ-104, AZ-305).
    • Create a job alert for this search

    Data Platform Engineer • Greater Delhi Area, India