Lead Data Engineer - Scala / Python

Job Title : Lead Data Engineer

Location : Bangalore / Pune

Employment Type : Full-time

Experience : 5- 8 years

Notice Period : Up to 30 days

Job Description :

We are seeking a highly skilled Lead Data Engineer to join our growing data team. The ideal candidate will take ownership of designing, building, and maintaining scalable data pipelines and architectures that support business intelligence, analytics, and advanced AI / ML initiatives. This role requires strong technical expertise, leadership abilities, and a deep understanding of modern data Responsibilities :

• Lead the design, development, and maintenance of end-to-end data pipelines and ETL processes.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver reliable solutions.
• Architect and implement high-performance, scalable, and fault-tolerant data systems across cloud and on- premise environments.
• Ensure data quality, consistency, and governance throughout the data lifecycle.
• Mentor and guide junior data engineers, fostering best practices in coding, testing, and deployment.
• Evaluate and integrate new data technologies and tools to enhance the teams efficiency and performance.
• Optimize data workflows for performance, cost, and reliability.
• Participate in cross-functional initiatives, providing technical leadership on data strategy, architecture, and Skills & Qualifications :
• 5- 8 years of experience in data engineering, ETL development, and big data technologies.
• Strong programming skills in Python, Java, or Scala.
• Experience with cloud platforms (AWS, Azure, or GCP) and their data services (Redshift, BigQuery, Snowflake, Databricks, etc.).
• Hands-on experience with data pipeline frameworks like Apache Spark, Airflow, Kafka, or Flink.
• Expertise in relational and NoSQL databases, data lakes, and warehousing solutions.
• Knowledge of CI / CD processes, containerization (Docker / Kubernetes), and version control (Git).
• Strong analytical, problem-solving, and debugging skills.
• Excellent communication and team leadership :
• Experience with real-time streaming and batch processing pipelines.
• Familiarity with ML / AI workflows and model deployment pipelines.
• Prior experience leading a team of data engineers or managing data projects Join Us :
• Work on cutting-edge data platforms supporting analytics and AI initiatives.
• Opportunity to lead a talented team and influence the organizations data strategy.
• Competitive compensation and benefits package.
• Dynamic work environment with growth and learning opportunities.

(ref : hirist.tech)