Experience - 8+ years
Role - Data Engineer(Tech Lead / Lead Consultant)
Location - Bangalore , Hyderabad , Chennai
Responsibilities :
Key Responsibility Areas :
- Design, build, and manage scalable and reliable data pipelines using Python or PySpark and Azure services .
- Lead a team of data engineers; mentor junior team members and enforce best practices in data engineering.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
- Develop and maintain data models, ETL / ELT workflows, and data orchestration pipelines.
- Optimize performance of data solutions, ensuring efficient data access, transformation, and storage.
- Oversee the implementation and maintenance of Azure Data Factory pipelines for automated data movement.
- Ensure proper data governance, security, and compliance across all data platforms.
- Participate in architectural design and decision-making for enterprise-scale data platforms.
Key Skills & Qualifications :
Must-Have Skills :
Strong proficiency in Python or PySpark for data engineering and ETL development.Advanced SQL skills for data transformation and query optimization.Deep experience with :Azure Data Lake Storage (Gen2)Azure Data Factory (ADF)Azure DatabricksExperience with performance tuning and optimizing big data solutions.Strong understanding of data warehousing concepts, data modelling, and data governance.