No longer accepting applications
Principal data engineer
AutodeskBengaluru, Karnataka, India4 days ago
Job descriptionPartner with architects to design multi-tenant, secure, and compliant (SOC2 / GDPR / CCPA) batch processing services Define reference architectures and reusable frameworks for cross-domain processing workloads Lead technical reviews, solution designs, and architectural forums across ADP Guide the evolution of Airflow APIs and SDKs to enable scalable pipeline self-service Mentor senior engineers, staff engineers, and tech leads within the Batch Processing and adjacent teams Partner with the Batch Ingestion and Data Security teams to deliver unified ingestion + processing flows Drive modernization initiatives to migrate away from legacy tools Pioneer AI-augmented data engineering practices (e.g., pipeline optimization agents, anomaly detection) Ensure scalability, cost-efficiency, and reliability for thousands of production pipelines across Autodesk Influence company-wide data engineering strategy by contributing thought leadership and whitepapers 8+ years of experience in software / data engineering, with at least 3 years in big data platform leadership roles Expert in distributed data processing (Spark, Py Spark, Ray, Flink) Deep experience with workflow orchestration (Airflow, Dagster, Prefect) Strong hands-on expertise in Lakehouse technologies (Iceberg, Delta, Hudi) and cloud platforms (AWS / Azure / GCP) Proven track record in architecting secure, multi-tenant, and compliant data platforms Skilled in SQL / No SQL databases, Snowflake, Glue, and modern metadata / catalog tools Strong problem-solving, communication, and cross-geo collaboration skills Experience mentoring engineers, building strong technical culture, and influencing at scale Exposure to AI / ML-driven data engineering (optimizers, anomaly detection, auto-scaling) Experience with data governance, lineage, and observability tools (Atlan, Databand, Collibra, etc.) Familiarity with streaming + batch hybrid architectures (Kappa / Lambda)
#25 WD9141
Position Overview
We are seeking a highly experienced Principal Engineer to lead the design, development, and evolution of our Batch Processing platform, which powers Autodesk’s Analytics Data Platform (ADP). This role requires deep technical expertise in distributed data systems, large-scale pipeline orchestration, and hands-on leadership in shaping next-generation data platform capabilities. You will partner closely with Engineering Managers, Architects, Product teams, and Partner Engineering to modernize our data lakehouse architecture, deliver highly reliable data pipelines, and establish technical excellence across ingestion, processing, and governance.
Responsibilities
Technical Leadership
- Define and drive the technical strategy for Batch Processing within ADP
- Design and scale Py Spark based distributed processing frameworks integrated with Airflow for orchestration
- Champion Apache Iceberg-based data lakehouse adoption (versioning, branching, WAP, indexing)
- Introduce performance optimization patterns for data pipelines (e.g., partition pruning, caching, resource tuning)
- Establish patterns for metadata-driven, intent-based data processing aligned with AI-assisted pipelines
Architecture & Delivery
Mentorship & Collaboration
Innovation & Impact
Minimum Qualifications
Preferred Qualifications
#LI-KS2
Create a job alert for this search
Principal Data Engineer • Bengaluru, Karnataka, India
Related jobs
![[Apply Now] Principal Data Engineer](https://cdn-logos.talent.com/v1/logo/image?feedcode=attb-in&company_name=PriceEasy%20AI)
![Principal AI Engineer [T500-19330]](https://cdn-logos.talent.com/v1/logo/image?feedcode=clickcast-appcast-li-in-cpc&company_name=Best%20Buy%20India)
- Promoted
Senior Principal Engineer - Generative AI
Kautilya Management ConsultantsBangalore Role & responsibilities : - Will work on end to end development of highly sophisticated Gen AI applications for firmwide integretion - Work as an indep...Show moreLast updated: 30+ days ago
- Promoted
Data Platform Engineer
Grizmo LabsBengaluru, Karnataka, India Design, build, and maintain scalable data platforms and pipelines to support the needs of the business.Ensure data quality and accuracy by implementing data validation and testing processes.Partici...Show moreLast updated: 18 days ago
- Promoted
Principal Engineer- Data Interfaces
ABBBengaluru, Karnataka, India This job is with ABB, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community.
Please do not contact the recruiter directly.At ABB, we help indu...Show moreLast updated: 5 days ago
- Promoted
Principal Engineer - Cloud Platform
Leap India Stack FoundationBangalore Position Purpose : The Cloud Platform Engineering Principal Platforms is a technical management position with responsibility for influencing impactful changes at the ...Show moreLast updated: 30+ days ago
- Promoted
Principal Data Engineer
Tata Consultancy ServicesBengaluru, Republic Of India, IN Minimum of 8-10 years of relevant data science, Business Intelligence, or data analytics experience – preferably with a digital media or digital advertising company.
Experience delivering products i...Show moreLast updated: 17 days ago
- Promoted
Principal Data Engineer
NeemtreeBangalore Description : Responsibilities : - Define and lead the data architecture vision and strategy, ensuring it supp...Show moreLast updated: 14 days ago
- Promoted
Principal Software Engineer - Big Data Technologies
hirevedaBangalore Key Responsibilities : - Lead technical initiatives and guide the team to develop innovative software solutions that address complex challenges.Build scalable, effici...Show moreLast updated: 30+ days ago
- Promoted
Principal Engineer
FV Bankvijayapura, India FV Bank is a fully licensed and regulated U.With a focus on innovation, security, and compliance, FV Bank is Banking the Future by providing USD banking, digital asset custody services, money marke...Show moreLast updated: 7 days ago
- Promoted
Principal Data Engineer
PriceEasy AIBengaluru, Karnataka, India Join one of the most respected and cutting-edge companies in the pricing and analytics industry! PriceEasy, a VC-backed leader, empowers convenience retailers with actionable insights through advan...Show moreLast updated: 30+ days ago
- Promoted
AI & Data Principal Engineer
Talent SocioBangalore About the Role : We are seeking a visionary AI & Data Principal Engineer to lead the architecture, development, and deployment of advanced AI and data-driven solutions.This...Show moreLast updated: 22 days ago
- Promoted
[Apply Now] Principal Data Engineer
PriceEasy AIBengaluru, Karnataka, India About PriceEasy : Join one of the most respected and cutting-edge companies in the pricing and analytics industry! PriceEasy, a VC-backed leader, empowers convenience retailers with actionable insi...Show moreLast updated: 17 days ago
- Promoted
Elanco - Principal Data Engineer
ElancoBangalore Description : At Elanco (NYSE : ELAN) it all starts with animals!.As a global leader in animal health, we are dedicated to innovation and delivering products and se...Show moreLast updated: 3 days ago
- Promoted
Principal Data Engineer
AutodeskBengaluru, Karnataka, India We are seeking a highly experienced.Autodesk’s Analytics Data Platform (ADP).This role requires deep technical expertise in distributed data systems, large-scale pipeline orchestration, and hands-o...Show moreLast updated: 7 days ago
- Promoted
Principal AI Engineer [T500-19330]
Best Buy IndiaBengaluru, Karnataka, India Lead operationalization of large-scale AI solutions, being the resident expert for all AI Engineering and AIOps challenges, partnering closely with AI Scientists.
Lead by example, inspire and guide ...Show moreLast updated: 30+ days ago
- Promoted
Principal Engineer-Devops
Dover IndiaGreater Bengaluru Area, India Dover is a diversified global manufacturer with annual revenues of $8 billion.We deliver equipment and components, in specialty systems and support services through five major operating segments : E...Show moreLast updated: 30+ days ago
- Promoted
GCP Data Engineer
JRD Systemshosur, tamil nadu, in Collaborate with data architects, analysts, and business teams to define and maintain.Roadnet) and modernized data sources.
Partner with governance and infrastructure teams to establish.Automate and...Show moreLast updated: 1 day ago
- Promoted
Principal Engineer
Wenger & WatsonBengaluru, Karnataka, India Principal Engineer — Marketing & CRM.We’re seeking a Principal Engineer to drive the evolution of our cloud-native, SaaS-based marketing and CRM platform serving banks and credit unions across the ...Show moreLast updated: 4 days ago
- Promoted
Data Engineer
Response Informaticshosur, tamil nadu, in AWS services : Must be proficient in building scalable data pipelines and managing cloud-native ETL workflows.Snowflake : Moderate understanding of Snowflake architecture.
CICD - Terraform or CloudFo...Show moreLast updated: 28 days ago