Senior Cloud Data Pipeline Architect
Nityo InfotechBengaluru, Republic Of India, IN7 days ago
Job descriptionStrong hands-on experience with AWS Glue , Glue Studio , and Glue Catalog . Expertise in PySpark and distributed data processing. Proficiency in Python , data structures, and Spark optimization techniques. Strong understanding of S3, IAM, Athena, Redshift, DynamoDB, Kinesis , etc. Experience with ETL / ELT pipeline design , debugging, and performance tuning. Good understanding of data warehousing concepts and SQL. Experience working in Agile environments and version control (Git).
Job Summary :
We are looking for an experienced Senior Data Engineer with strong expertise in AWS Glue, PySpark, and cloud-based data engineering. The ideal candidate will design, develop, and optimize scalable data pipelines and ETL workflows on AWS, ensuring high performance, reliability, and data quality.
Key Responsibilities
- Design, build, and maintain scalable ETL pipelines using AWS Glue , PySpark , and Lambda .
- Develop and optimize PySpark scripts for large-scale data processing and transformation.
- Implement data ingestion from multiple sources (S3, RDS, APIs, Streaming sources).
- Work with AWS services such as S3, Glue, Athena, EMR, Redshift, IAM, CloudWatch .
- Ensure data quality, validation, and governance across all pipelines.
- Optimize performance of Glue jobs, Spark clusters, and query execution.
- Collaborate with data architects, analysts, and cross-functional teams to understand requirements.
- Participate in code reviews, best-practice enforcement, and CI / CD pipeline implementation.
- Troubleshoot production issues, monitor pipeline performance, and deliver timely fixes.
- Manage data workflows, job orchestration, and scheduling using AWS Glue Workflows / Step Functions / Airflow.
Required Skills
Create a job alert for this search
Cloud Data Architect • Bengaluru, Republic Of India, IN
Related jobs
- Promoted
Enterprise Data Architect
DBiz.aihosur, tamil nadu, in You will ensure that data is modeled, structured and integrated across the organization to support analytics and AI / ML initiatives at scale.
You will work closely with business stakeholders and tech...Show moreLast updated: 11 days ago
- Promoted
Data Engineer(Azure / AWS, Python / Pyspark, SQL)
Sail Analyticshosur, tamil nadu, in Architect, develop, test and maintain scalable data warehouses and data pipelines.Expertise in SQL, PySpark / Python and Azure(ADB, ADF) or AWS(Glue, Lambda, Redshift).
Bachelor's degree or equivalent...Show moreLast updated: 30+ days ago
- Promoted
Data Architect
Tech Mahindrahosur, tamil nadu, in Strong experience architecting and delivering.Proficiency in Azure services including.Azure Synapse Analytics, Azure Data Factory, Azure Data Lake Storage, Azure Databricks.Python, Spark, or relate...Show moreLast updated: 1 day ago
- Promoted
Data Architect
BrillioBengaluru, Karnataka, India Architecting, designing and implementing Modern Data Platforms and Solutions using Microsoft Azure Cloud and understands reference architectures.
Deep understanding Data Migration Strategies with 2+...Show moreLast updated: 1 day ago
- Promoted
- New!
Cloud Consultant
Mastekhosur, tamil nadu, in We need an experienced AWS Data Engineer / Performance Optimization Consultant to help us analyze, tune, and optimize our existing AWS Glue ETL, Step Functions workflows and Amazon Redshift data wa...Show moreLast updated: 12 hours ago
- Promoted
Senior Data Engineer (AWS / Databricks)
Accoladehosur, tamil nadu, in The multifamily real estate industry is undergoing a massive transformation, and Accolade is at the forefront.We are building the industry's first AI-native Operations Centralization Platform, desi...Show moreLast updated: 30+ days ago
- Promoted
Azure Cloud Architect
Persistent Systemsbangalore, karnataka, in We are seeking an experienced Azure Data Cloud Architect to design, implement, and optimize cutting-edge data solutions on Microsoft Azure.
You will collaborate with cross-functional teams to transl...Show moreLast updated: 22 days ago
- Promoted
Azure Cloud Data Engineer
Intelebeehosur, tamil nadu, in Data Factory, Synapse, and Databricks.Python, PySpark, and Advanced SQL.Lakehouse / Medallion architecture.Customer Interaction & Technical Documentation.
Interact with clients and business stakehol...Show moreLast updated: 15 days ago
- Promoted
Cloud Architect
Hexaware Technologieshosur, tamil nadu, in Total IT Experience 12 to 18 years.Must experience in Cloud Architect.Must experience in Azure (Func, Blob storage, API Mgmt, Logic apps).
Can be from any technical programming background like.NET C...Show moreLast updated: 1 day ago
- Promoted
Cloud Architect
The Goodyear Tire & Rubber Companyhosur, tamil nadu, in Strong experience designing and building cloud-native applications.Proficiency in modern backend and frontend frameworks (e.
Deep knowledge of DevSecOps, CI / CD pipelines, and infrastructure-as-code....Show moreLast updated: 13 days ago
- Promoted
GCP Data Architect
Tata Consultancy Servicesbangalore district, karnataka, in Job Title : - GCP Data Architect.Design and Implement Data Architectures : Architect and build scalable, end-to-end data solutions on GCP, encompassing data ingestion, transformation, storage, and co...Show moreLast updated: 22 days ago
- Promoted
Senior Data Engineer
Guidanz Inchosur, tamil nadu, in BI Connector is the industry leading solution for integrating Oracle Fusion Cloud data into modern BI platforms like Power BI, Tableau, and Data Warehouse, without complex ETL.Our Data Architecture...Show moreLast updated: 1 day ago
- Promoted
SAP Basis Cloud architect Advisory
HCLTechhosur, tamil nadu, in CAA (Cloud Architect Advisory).This is SAP Cloud Architect position which is elevated role as per career progression / path for SAP Basis consultant.
Architect’s primary role / responsibility is to pl...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
SAIVA AIhosur, tamil nadu, in We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 30+ days ago
- Promoted
Cloud Architect
iVedha Inc.hosur, tamil nadu, in Seeking a highly experienced Cloud Architect to design and oversee robust, scalable, and secure.Architect end-to-end cloud solutions (public, private, hybrid) with a focus on reliability, security,...Show moreLast updated: 30+ days ago
- Promoted
Principal / Senior Data Architect
Aayshosur, tamil nadu, in Position : Principal / Senior Data Architect.You will act as a key member of the consulting team helping Clients to re-invent their corporate finance function by leveraging advanced analytics.You wil...Show moreLast updated: 30+ days ago
- Promoted
Databricks Data Architect
Tata Consultancy Serviceshosur, tamil nadu, in Experience with machine learning frameworks and pipelines, particularly in Databricks.Familiarity with containerized environments (e.
Docker, Kubernetes) for deploying data workflows.Should be able ...Show moreLast updated: 1 day ago
- Promoted
Data Architect
Persistent SystemsBengaluru, Karnataka, India We are looking for an experienced and talented Data Architect to join our growing data engineering team.The ideal candidate will have a strong background in working with Python, Pyspark, AWS / Databr...Show moreLast updated: 30+ days ago