Lead Data Pipeline Architect

Position : Senior Data Engineer

Location : Bengaluru

Experience : 5+ years

Job Summary :

We are seeking an experienced Senior Data Engineer with a strong background in PySpark, Spark, and Big Data technologies. The ideal candidate will have over five years of hands-on experience designing, building, and optimizing large-scale data processing systems and pipelines.

Key Responsibilities :

• Design, develop, and maintain scalable data pipelines and ETL processes using PySpark and Spark.
• Perform data wrangling, transformation, and integration across diverse data sources.
• Optimize the performance and reliability of data processing workflows.
• Implement data quality frameworks to ensure data accuracy and integrity.
• Develop and maintain data architectures and data models to support business needs.
• Collaborate with cross-functional teams including data analysts, software engineers, and business stakeholders to understand data requirements.
• Stay updated with the latest advancements in Big Data tools, frameworks, and best practices.
• Mentor and guide junior data engineers in technical and professional growth.

Required Skills and Qualifications :

Preferred Qualifications :