LEAD DATA ENGINEER

We’re Hiring : LEAD DATA ENGINEER

Location : HYDERABAD(HYBRID)

Experience : 7+ Years

⏳ Notice Period : Immediate to 30 Days

Responsibilities

Design, build, and optimize scalable data pipelines on Azure Data Lake, Azure Databricks, and Azure Synapse.

Develop ETL / ELT workflows using PySpark, Python, and Databricks notebooks.

Implement and manage data ingestion pipelines using ADF (Azure Data Factory).

Work with Delta Lake, structured streaming, and advanced data transformation frameworks.

Optimize data processing performance and troubleshoot pipeline issues.

Collaborate with data architects, analysts, and business stakeholders to deliver high-quality datasets.

Implement data quality, data validation, and schema evolution mechanisms.

Ensure best practices for CI / CD, version control, and automation using Git, Azure DevOps.

Manage data storage, partitioning, and governance within the Azure ecosystem.

Work with SQL-based systems such as Azure SQL, PostgreSQL, or SQL Server.

Required Skills

Strong hands-on experience with Azure Databricks, PySpark, and Python.

Solid understanding of Azure Data Lake Storage (ADLS) and Azure Data Factory (ADF).

Experience with Delta Lake, Databricks workflows / jobs.

Strong SQL skills and exposure to PostgreSQL / other relational databases.

Understanding of data modeling, ETL frameworks, and distributed data processing.

Experience with Azure DevOps, Git, CI / CD pipelines.

Familiarity with Azure Synapse, Spark optimization & performance tuning.