Job Title : Data Engineer
Experience Required : 4+ Years
Location : Pune
Job Description :
We are looking for a skilled Data Engineer with strong expertise in Python, PySpark, and Databricks to join our team. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines, ensuring the quality and reliability of data across various sources and destinations.
Key Responsibilities :
- Analyze and understand source data structures, formats, and business requirements.
- Design and develop efficient data pipelines on Databricks using PySpark to extract, transform, and load (ETL) data into target Delta tables .
- Optimize and automate data workflows for improved performance and scalability.
- Ensure data accuracy, consistency, and integrity across systems.
- Collaborate with cross-functional teams including data analysts, data scientists, and business stakeholders to support data-driven initiatives.
- Implement best coding practices and ensure the quality and maintainability of the data engineering solutions.
- Monitor and troubleshoot production data issues, ensuring timely resolution.
Required Skills & Qualifications :
Minimum 4 years of hands-on experience as a Data Engineer.Strong proficiency in Python programming and PySpark .Experience with Databricks and working with Delta Lake tables .Solid understanding of data modeling , ETL processes , and data warehousing concepts .Experience in analyzing and working with large datasets from multiple data sources.Good understanding of SQL and performance optimization techniques.Strong analytical and problem-solving skills with attention to detail.Excellent communication and collaboration abilities.Nice to Have :
Experience with Azure Data Lake , AWS , or GCP .Familiarity with CI / CD for data pipelines.Knowledge of data governance and data quality frameworks .