Data Pipeline ArchitectBlend • Chennai, Republic Of India, IN
Data Pipeline Architect
Blend • Chennai, Republic Of India, IN
2 days ago
Job descriptionBachelor's or master's degree in computer science, Engineering, or a related field. 8+ years of hands-on experience in data engineering , with at least 3 years working with AWS Databricks . Strong programming skills in Python for data processing and automation. Advanced proficiency in SQL for querying and transforming large datasets. Deep experience with Apache Spark / PySpark in a distributed computing environment. Solid understanding of data modelling, warehousing, and performance optimization techniques. Proficiency with AWS services such as Glue , S3 , Lambda and EMR . Experience with version control Git or Code commit Experience in any workflow orchestration like Airflow, AWS Step functions is a plus. Excellent communication and stakeholder management skills with a strategic mindset.
We are looking for an experienced Data Engineering Manager with a strong foundation in Python, SQL, and Spark , and hands-on expertise in AWS, Databricks . In this role, you will build and maintain scalable data pipelines and architecture to support analytics, data science, and business intelligence initiatives. You’ll work closely with cross-functional teams to drive data reliability, quality, and performance.
Responsibilities :
- Design, develop, and optimize scalable data pipelines using Databricks in AWS such as Glue, S3, Lambda, EMR, Databricks notebooks, workflows and jobs.
- Building data lake in WS Databricks.
- Build and maintain robust ETL / ELT workflows using Python and SQL to handle structured and semi-structured data.
- Develop distributed data processing solutions using Apache Spark or PySpark .
- Partner with data scientists and analysts to provide high-quality, accessible, and well-structured data.
- Ensure data quality, governance, security, and compliance across pipelines and data stores.
- Monitor, troubleshoot, and improve the performance of data systems and pipelines.
- Participate in code reviews and help establish engineering best practices.
- Mentor junior data engineers and support their technical development.
Requirements :
Create a job alert for this search
Data Pipeline Architect • Chennai, Republic Of India, IN
Related jobs
Data Pipeline Architect
Randstad Enterprise • Republic Of India, IN
Shift Timing : 2 : 00 Pm - 11 : 00 Pm.Experience : 2- 4 years relevant Experience only ( this is a Junior position with us ).
GCP - 2 years minimum working Experience.Worked with global stakeholders.Ran...Show more
Last updated: 30+ days ago • Promoted
Data Pipeline Architect
Tredence Inc. • Republic Of India, IN
IT experience and 7+ years' experience in SQL Server, SSIS, SQL, Python / C#, ETL, Streaming.Design and develop SQL Server stored procedures, functions, views and triggers to be used during the ETL p...Show more
Last updated: 30+ days ago • Promoted
Data Pipeline Architect
IntraEdge • Republic Of India, IN
We are seeking a highly skilled Data Engineer with strong experience in Python, PySpark, Snowflake, and AWS Glue to join our growing data team.
You will be responsible for building scalable and reli...Show more
Last updated: 30+ days ago • Promoted
GCP Data Pipeline Architect
EXL • Pune, Republic Of India, IN
The ideal candidate will have a solid background in big data technologies, data warehousing, and cloud-based data architecture, with hands-on experience using GCP-native tools.Dataflow, Dataproc, P...Show more
Last updated: 16 days ago • Promoted
Lead Data Pipeline Architect
CYGNVS • Chennai, Republic Of India, IN
Our company culture is a perfect blend of productivity and enjoyment, prioritizing the growth and development of our talented employees as part of a high-achieving team.
We are in search of exceptio...Show more
Last updated: 24 days ago • Promoted
Lead Data Pipeline Architect
Lear Corporation • Pune, Republic Of India, IN
Lear, a global automotive technology leader in Seating and E-Systems, is Making every drive better by delivering intelligent in-vehicle experiences for customers around the world.With over 100 year...Show more
Last updated: 20 days ago • Promoted
Data Pipeline Architect
Solytics Partners • Pune, Republic Of India, IN
Solytics Partners is a Global Analytics firm, recognized with multiple industry awards for innovation and excellence.Our team comprises experts with deep knowledge in risk, analytics, AI / ML, AML / FC...Show more
Last updated: 30+ days ago • Promoted
Data Pipeline Architect
Kanerika Inc • Indore, Republic Of India, IN
Hyderabad, Indore and Ahmedabad (India).Develop and maintain data pipelines using Azure Data Factory (ADF) / Databricks for data integration and ETL processes.
Design, implement, and optimize Power B...Show more
Last updated: 23 days ago • Promoted
Data Pipeline Architect
NexionPro Services • Republic Of India, IN
PySpark-based data processing workflows.Collaborate with data architects, analysts, and cross-functional teams to understand business requirements and translate them into technical solutions.Optimi...Show more
Last updated: 30+ days ago • Promoted
Data Pipeline Architect
NielsenIQ • Chennai, Republic Of India, IN
Design, build, and optimize large-scale ETL and data-processing pipelines handling GB–TB volumes.Operate within the Databricks ecosystem and drive migration of selected workloads to high-performanc...Show more
Last updated: 11 days ago • Promoted
Data Pipeline Architect
HCLTech • Chennai, Republic Of India, IN
Senior Data Engineer – Vendor Experience : 6–12 Years.We are seeking Senior Data Engineer resources to work on the migration of applications from our legacy Cloudera environment to the new Kubernete...Show more
Last updated: 11 days ago • Promoted
Data Pipeline Architect
EXL • Pune, Republic Of India, IN
Location - Pune, Bangalore, Noida, Gurgaon, Hyderabad.The ideal candidate will have strong expertise in Snowflake, Hadoop ecosystem, PySpark, and SQL, and will play a key role in enabling data-driv...Show more
Last updated: 30+ days ago • Promoted
Data Pipeline Architect
Mastek • Chennai, Republic Of India, IN
Work closely with our data science team to help build complex algorithms that provide unique insights into our data.Use agile software development processes to make iterative improvements to our ba...Show more
Last updated: 6 days ago • Promoted
Senior Data Pipeline Architect
Ascendion • Chennai, Republic Of India, IN
Job Description : Senior DBT Engineer (8–12 Years).Senior Data Engineer with strong ELT experience.Focus on building scalable, efficient, and well-governed DBT data models on modern cloud platforms.Show more
Last updated: 2 days ago • Promoted
Data Pipeline Architect
Lemongrass • Republic Of India, IN
Lemongrass is a software-enabled services provider, synonymous with SAP on Cloud, focused on delivering superior, highly automated Managed Services to Enterprise customers.Our customers span multip...Show more
Last updated: 30+ days ago • Promoted
Lead Data Pipeline Architect
LanceSoft, Inc. • Republic Of India, IN
US working hours (Night Shift form india).This team is looking to hire a motivated Sr.Data Engineer who will be working closely with team of highly motivated data engineers to build a state-of-the-...Show more
Last updated: 4 days ago • Promoted
Lead Azure Data Pipeline Architect
All European Careers • Republic Of India, IN
For an international project in Chennai, we are urgently looking for a Full Remote Senior Azure Data Engineer, who will build data pipeline for enterprise search applications using ADF and Databric...Show more
Last updated: 9 days ago • Promoted
Principal Data Pipeline Architect
Publicis Production • Republic Of India, IN
We are seeking a proactive and self-motivated Senior Data Engineer with a proven track record in building scalable cloud-based data solutions across multiple cloud platforms to support our work in ...Show more
Last updated: 2 days ago • Promoted