Data Pipeline Architect
Veca Consulting Pvt LtdBengaluru, Republic Of India, IN6 days ago
Job descriptionProficiency in Hadoop ecosystems such as Spark, HDFS, Hive, Iceberg, Spark SQL. Extensive experience with Apache Kafka, Apache Flink, and other relevant streaming technologies. Proven ability to design and implement automated data pipelines and materialized views. Proficiency in Python, Unix or similar languages. Good understanding of SQL oracle, SQL server or similar languages. Ops & CI / CD : Monitoring (Prometheus / Grafana), logging, pipelines (Jenkins / GitHub Actions). Core Engineering : Data structures / algorithms, testing (JUnit / pytest), Git, clean code. 5+ years of directly applicable experience BS in Computer Science, Engineering, or equivalent experience.
Role Name : Data Engineer
Experience : 3+ Years
Location : Bangalore(No relocation)
Notice Period : 20-30 days(who are currently serving)
Mode : Hybrid
Type : Fulltime
Job Description :
As a Data Engineer on our team, you will work on our Hadoop-based data warehouse, contributing to scalable and reliable big data solutions for analytics and business insights. This is a hands-on role focused on building, optimizing, and maintaining large data pipelines and warehouse infrastructure.
Key Responsibilities
- Design, develop, and maintain robust data pipelines in Hadoop and related ecosystems, ensuring data reliability, scalability, and performance.
- Implement data ETL processes for batch and streaming analytics requirements.
- Optimize and troubleshoot distributed systems for ingestion, storage, and processing.
- Collaborate with data engineers, analysts, and platform engineers to align solutions with business needs.
- Ensure data security, integrity, and compliance throughout the infrastructure.
- Maintain documentation and contribute to architecture reviews.
- Participate in incident response and operational excellence initiatives for the data warehouse.
- Continuously learn mindset and apply new Hadoop ecosystem tools and data technologies.
Required Skills and Experience
Create a job alert for this search
Data Pipeline Architect • Bengaluru, Republic Of India, IN
Related jobs
)
- Promoted
Data Architect
Tredence Inc.Bangalore Urban, Karnataka, India IT experience and 7+ years' experience in SQL Server, SSIS, SQL, Python / C#, ETL, Streaming.Design and develop SQL Server stored procedures, functions, views and triggers to be used during the ETL p...Show moreLast updated: 1 day ago
- Promoted
Data Architect
Tech Mahindrahosur, tamil nadu, in Strong experience architecting and delivering.Proficiency in Azure services including.Azure Synapse Analytics, Azure Data Factory, Azure Data Lake Storage, Azure Databricks.Python, Spark, or relate...Show moreLast updated: 16 days ago
- Promoted
- New!
ETL Architect
Tata Consultancy Serviceshosur, tamil nadu, in Notice period- 0 to 6 o Days or Serving NP.Developing, deploying, and monitoring end to end data integration pipelines with extensive knowledge of evaluation metrics and best practices.Data Warehou...Show moreLast updated: 3 hours ago
- Promoted
Data Pipeline Architect
MastekBengaluru, Republic Of India, IN Work closely with our data science team to help build complex algorithms that provide unique insights into our data.Use agile software development processes to make iterative improvements to our ba...Show moreLast updated: 30+ days ago
- Promoted
Data Architect
BrillioBengaluru, Karnataka, India We are looking for a highly skilled.The ideal candidate will combine strong technical capabilities with architectural oversight, ensuring efficient, scalable, and governed AWS data ecosystems.Exper...Show moreLast updated: 16 days ago
- Promoted
Databricks Data Architect
Tata Consultancy ServicesBengaluru, Karnataka, India Experience with machine learning frameworks and pipelines, particularly in Databricks.Familiarity with containerized environments (e.
Docker, Kubernetes) for deploying data workflows.Should be able ...Show moreLast updated: 16 days ago
- Promoted
Data Pipeline Architect
ANSRBengaluru, Republic Of India, IN To Care for People on Life's Journey®.Together with our American Eagle regional partners, we offer thousands of flights daily to more than 350 destinations in more than 60 countries.American Airlin...Show moreLast updated: 13 days ago
- Promoted
Senior Data Architect - Snowflake
Reflections Info Systemshosur, tamil nadu, in We are looking for 10 + year experienced Data Architect with strong background in Snowflake, demonstrating leadership in technical design, architecture, and implementation of complex data solutions...Show moreLast updated: 1 day ago
- Promoted
Data Architect
AndelaBengaluru, IN Andela exists to connect brilliance and opportunity.Since 2014, we have been dedicated to breaking down global barriers and accelerating the future of work for both technologists and organizations ...Show moreLast updated: 2 days ago
- Promoted
Associate Director, Data Architecture (Snowflake • Databricks • AWS • CPG / FMCG • Enterprise Data)
Sky Systems, Inc. (SkySys)hosur, tamil nadu, in Associate Director, Data Architecture.Full-Time Contract (40hrs / week).Months+ (with a possibility of Contract-to-Hire).
Marketing, Product, Finance, Supply Chain, and global business teams.This role...Show moreLast updated: 1 day ago
- Promoted
AWS Data Architect
ACL DigitalBengaluru, IN AWS (S3, Redshift, Glue, Lake Formation, IAM).Proficient in data modeling, performance tuning, and security best practices.
.AWS Certified Solutions Architect preferred.Show moreLast updated: 7 days ago
- Promoted
GCP Data Architect
Lykora Consulting Services Private LimitedBangalore, IN GCP Architecture, Data Platform Design, DevOps, Security, Cloud Infrastructure.Leads architecture and delivery of enterprise data platforms on GCP, ensuring scalability, compliance, and performance...Show moreLast updated: 1 day ago
- Promoted
Data Architect
Genisys GroupBengaluru, Karnataka, India The Data Lead / Data Architect will own the vision, design, and delivery of cutting-edge enterprise data platforms and modern analytics solutions across public clouds and hybrid infrastructures.This ...Show moreLast updated: 3 days ago
- Promoted
- New!
Azure + Mongo- Data Architect
People Prime Worldwidehosur, tamil nadu, in Our client is a global technology consulting and digital solutions company that enables enterprises to reimagine business models and accelerate innovation through digital technologies.Powered by mo...Show moreLast updated: 3 hours ago
- Promoted
Data Pipeline Architect
SigmoidBengaluru, Republic Of India, IN We are seeking a highly skilled and motivated Data Engineer to join our dynamic team.You will be responsible for designing, implementing, and maintaining data pipelines to support our organization'...Show moreLast updated: 14 days ago
- Promoted
Data Architect (Adobe AEP)
RibbitZ LLCBengaluru, IN Define overall AEP data architecture, schemas (XDM), and identity strategy.Design and oversee ingestion pipelines (batch, streaming, APIs).
Ensure data quality, scalability, and governance across da...Show moreLast updated: 1 day ago
- Promoted
Sr. Azure Data Architect & Presales Solution
Programmers.iohosur, tamil nadu, in Job Title : Azure Data Architect.Location : Hyderabad, Pune, Jaipur.Experience required : 12+ years.We are seeking a highly experienced.
The ideal candidate should bring strong expertise in SQL, ETL / EL...Show moreLast updated: 9 days ago
- Promoted
Senior Solutions Architect (Data)
Hillview Consulting Solutionshosur, tamil nadu, in If candidate is in Mumbai this would be onsite in Andheri East, Mumbai, Maharashtra.We’re looking for a senior, hands-on.
You’ll own architecture for ETL / ELT, data warehousing, analytics pipelines, ...Show moreLast updated: 2 days ago