Sr Data Engineering Lead (AWS / Azure)

Attention All Data Folks!!!

We are Hiring Sr Data Engineering Lead (Azure / AWS) in Indore, MP (Hybrid Model)  Below is the JD for the reference Sr Data Engineering Lead Indore, MP (Hybrid 3 Days a Week) Full Time Exp Level-  8-15+ years   About the Role Ccube is seeking a highly skilled Data Engineering Lead to architect, build, and scale our enterprise-grade data infrastructure.

This role demands a strong technical foundation in distributed data systems, deep hands-on experience with modern big data and cloud technologies, and proven leadership in building and mentoring high-performing data engineering teams.

The ideal candidate will shape the data engineering roadmap, drive innovation, and ensure robust, scalable, and efficient data solutions across the organization.

Key Responsibilities Data Strategy & Architecture   Define the end-to-end data engineering strategy and technical vision aligned with Ccube’s business goals.

• Architect scalable, high-performance, and cost-efficient data platforms on AWS, Azure, or GCP .
• Design and optimize data lakehouse medallion architectures integrating batch and streaming pipelines (Spark, Kafka, Delta Lake, Iceberg, Hudi, etc.).
• Build reusable frameworks for data ingestion, transformation, and orchestration across heterogeneous systems.
• Data Engineering Execution   Lead the development and optimization of ETL / ELT pipelines using PySpark, Scala, SQL, and Airflow or equivalent orchestration tools.
• Oversee the implementation of real-time streaming solutions leveraging Kafka, Kinesis, or Pub / Sub .
• Guide the team in integrating structured, semi-structured, and unstructured data sources .
• Drive adoption of DataOps and DevOps best practices — CI / CD for data pipelines, automated testing, and monitoring.
• Cloud & Infrastructure   Design and manage cloud-native data solutions (AWS Glue, EMR, Redshift, Snowflake, BigQuery, Databricks, etc.).
• Optimize infrastructure for scalability, performance, and cost using Terraform, CloudFormation, or Pulumi .
• Lead initiatives for data platform modernization and cloud migration strategies.
• Governance, Security & Observability   Define and enforce standards for data quality, lineage, governance, and metadata management (e.g., Great Expectations, Apache Atlas, or Collibra).
• Implement robust data security, compliance, and privacy frameworks aligned with industry standards (GDPR, HIPAA, etc.).
• Establish observability frameworks for data pipelines — logging, monitoring, and anomaly detection.
• Leadership & Collaboration   Lead and mentor a team of senior and mid-level data engineers, fostering a culture of excellence, ownership, and innovation.
• Collaborate cross-functionally with AI / ML, Analytics, and Product Engineering teams to enable data-driven decision-making.
• Evaluate emerging technologies (e.g., VectorDBs, GraphDBs, RAG frameworks ) and drive their adoption for advanced data-driven use cases.
• Represent the data engineering practice in architecture reviews and executive technology forums.
• Required Qualifications Experience :   10+ years in data engineering and architecture roles, with 3+ years in technical leadership or data platform lead roles.

Technical Expertise :   Deep proficiency in Spark, PySpark, Scala, SQL, and distributed data processing.

Proven hands-on work in cloud data platforms – AWS (Glue, EMR, Redshift), GCP (Dataflow, BigQuery), or Azure (Synapse, Data Factory).

Why Join Ccube?

Build the Future – Be part of a mission-driven company shaping the next generation of AI and digital solutions Collaborative Culture – Work with kind, brilliant people who value transparency, experimentation, and integrity Career Mobility – We invest in your learning and promote from within Compensation & Equity – Competitive base + commission + stock options Professional Growth – Annual stipend for certifications, courses, and industry events Are you ready to help companies unlock the power of technology—and build something extraordinary in the process?   Feel Free to reach out to me at "rahul @ccube.com " for more details.

​ Powered by JazzHR