We are seeking a highly skilled Senior Data Engineer with deep expertise in
Microsoft Azure data ecosystem to design, develop, and maintain scalable data
pipelines and architectures. The ideal candidate will play a key role in building robust
data solutions that support advanced analytics, BI, and AI workloads across the
organization. This role involves working with cross-functional teams — data scientists,
analysts, and business stakeholders — to ensure high-quality, secure, and performant
data delivery.
Total experience : 8 to 10 years
Key Responsibilities :
1. Data Architecture & Pipeline Development
- Design, develop, and maintain data ingestion, transformation, and storage
pipelines using Azure Data Factory, Azure Databricks, and Synapse Analytics.
Build end-to-end ETL / ELT workflows from diverse sources such as APIs, on-premdatabases, SaaS applications, and data lakes.
Implement data modeling (star schema, snowflake, data vault, etc.) for analyticaland operational data stores.
Manage data ingestion frameworks for both batch and streaming (real-time) usecases using Azure Event Hubs / Azure Stream Analytics / Kafka.
2. Data Management & Governance
Ensure data quality, consistency, and integrity across all environments.Implement and enforce data governance standards, including metadatamanagement, lineage tracking, and data cataloging (e.g., Azure Purview).
Manage security, compliance, and access controls using Azure Active Directory(AAD) and RBAC principles.
Automate data validation, auditing, and monitoring workflows using modernDevOps practices.
3. Cloud & DevOps Integration
Manage and optimize Azure Data Lake Storage (ADLS Gen2) for cost andperformance.
Leverage Infrastructure as Code (IaC) tools such as Terraform, Bicep, or ARMtemplates for environment provisioning.
Implement CI / CD pipelines for data workflows using Azure DevOps / GitHubActions.
Optimize compute and storage costs while maintaining scalability and resilience.4. Collaboration & Leadership
Partner with data scientists, BI developers, and product teams to deliver reliabledata solutions.
Mentor junior data engineers on best practices, coding standards, and Azureservices.
Participate in architectural reviews and contribute to data platform designdecisions.
Communicate technical insights and trade-offs to non-technical stakeholderseffectively.
Required Skills :
Core Azure Services :
Azure Data Factory (ADF) – pipeline orchestration, triggers, linked services,integration runtime.
Azure Databricks – PySpark, Delta Lake, notebooks, job orchestration.Azure Synapse Analytics – dedicated and serverless SQL pools, data modeling,query optimization.
Azure Data Lake Storage (ADLS Gen2) – hierarchical namespace, lifecyclemanagement.
Azure Functions / Logic Apps – event-driven data workflows.Azure Event Hubs / Kafka / Stream Analytics – real-time data ingestion.Programming & Tools :
Strong in Python or Scala for data transformations.
Proficient in SQL (T-SQL, Spark SQL, or Synapse SQL).
Experience with PySpark for distributed data processing.
Knowledge of Git, CI / CD, and DevOps best practices.
Familiarity with Power BI integration or other reporting tools is a plus.Data Architecture & Modeling :
Expertise in data warehouse design, data lakehouse architecture, and datapipelines.
Understanding of data partitioning, indexing, and query optimizationtechniques.
Experience with dimensional modeling, slowly changing dimensions, and facttables.
Other :
Experience with API-based data ingestion and RESTful integrations.Exposure to machine learning data pipelines or feature stores is a plus.Familiarity with cost optimization and performance tuning in Azure.Soft Skills
Excellent analytical and problem-solving abilities.
Strong communication and documentation skills.
Ability to work in an agile, fast-paced environment.
Proven experience collaborating in cross-functional teams.
Self-driven with a passion for data engineering and cloud technologies.