Talent.com
No longer accepting applications
Senior Data Engineer

Senior Data Engineer

People Prime Worldwidejunagadh, gujarat, in
9 days ago
Job description

We are seeking a highly skilled Senior Data Engineer with deep expertise in

Microsoft Azure data ecosystem to design, develop, and maintain scalable data

pipelines and architectures. The ideal candidate will play a key role in building robust

data solutions that support advanced analytics, BI, and AI workloads across the

organization. This role involves working with cross-functional teams — data scientists,

analysts, and business stakeholders — to ensure high-quality, secure, and performant

data delivery.

Total experience : 8 to 10 years

Key Responsibilities :

1. Data Architecture & Pipeline Development

  • Design, develop, and maintain data ingestion, transformation, and storage

pipelines using Azure Data Factory, Azure Databricks, and Synapse Analytics.

  • Build end-to-end ETL / ELT workflows from diverse sources such as APIs, on-prem

    • databases, SaaS applications, and data lakes.

  • Implement data modeling (star schema, snowflake, data vault, etc.) for analytical

    • and operational data stores.

  • Manage data ingestion frameworks for both batch and streaming (real-time) use

    • cases using Azure Event Hubs / Azure Stream Analytics / Kafka.

    2. Data Management & Governance

  • Ensure data quality, consistency, and integrity across all environments.
  • Implement and enforce data governance standards, including metadata

    • management, lineage tracking, and data cataloging (e.g., Azure Purview).

  • Manage security, compliance, and access controls using Azure Active Directory

    • (AAD) and RBAC principles.

  • Automate data validation, auditing, and monitoring workflows using modern

    • DevOps practices.

    3. Cloud & DevOps Integration

  • Manage and optimize Azure Data Lake Storage (ADLS Gen2) for cost and

    • performance.

  • Leverage Infrastructure as Code (IaC) tools such as Terraform, Bicep, or ARM

    • templates for environment provisioning.

  • Implement CI / CD pipelines for data workflows using Azure DevOps / GitHub

    • Actions.

  • Optimize compute and storage costs while maintaining scalability and resilience.

    • 4. Collaboration & Leadership

  • Partner with data scientists, BI developers, and product teams to deliver reliable

    • data solutions.

  • Mentor junior data engineers on best practices, coding standards, and Azure

    • services.

  • Participate in architectural reviews and contribute to data platform design

    • decisions.

  • Communicate technical insights and trade-offs to non-technical stakeholders

    • effectively.

    Required Skills :

    Core Azure Services :

  • Azure Data Factory (ADF) – pipeline orchestration, triggers, linked services,

    • integration runtime.

  • Azure Databricks – PySpark, Delta Lake, notebooks, job orchestration.
  • Azure Synapse Analytics – dedicated and serverless SQL pools, data modeling,

    • query optimization.

  • Azure Data Lake Storage (ADLS Gen2) – hierarchical namespace, lifecycle

    • management.

  • Azure Functions / Logic Apps – event-driven data workflows.
  • Azure Event Hubs / Kafka / Stream Analytics – real-time data ingestion.

    • Programming & Tools :

    Strong in Python or Scala for data transformations.

    Proficient in SQL (T-SQL, Spark SQL, or Synapse SQL).

    Experience with PySpark for distributed data processing.

    Knowledge of Git, CI / CD, and DevOps best practices.

    Familiarity with Power BI integration or other reporting tools is a plus.Data Architecture & Modeling :

  • Expertise in data warehouse design, data lakehouse architecture, and data

    • pipelines.

  • Understanding of data partitioning, indexing, and query optimization

    • techniques.

  • Experience with dimensional modeling, slowly changing dimensions, and fact

    • tables.

    Other :

  • Experience with API-based data ingestion and RESTful integrations.
  • Exposure to machine learning data pipelines or feature stores is a plus.
  • Familiarity with cost optimization and performance tuning in Azure.

    • Soft Skills

    Excellent analytical and problem-solving abilities.

    Strong communication and documentation skills.

    Ability to work in an agile, fast-paced environment.

    Proven experience collaborating in cross-functional teams.

    Self-driven with a passion for data engineering and cloud technologies.

    Create a job alert for this search

    Senior Data Engineer • junagadh, gujarat, in