Principal Data Pipeline Engineer
XebiaBengaluru, Republic Of India, IN2 days ago
Job descriptionBachelor’s degree (B.Tech / BE) or equivalent in Computer Science, Engineering, or related. 6-10 years of relevant experience in data engineering / big data / analytics platforms. Strong expertise in distributed data processing frameworks such as Apache Spark (Scala or Java) and / or Python. Solid experience with SQL and query optimisation over large volumes of data. Experience with big data ecosystems : Hadoop distributions (Cloudera / Hortonworks), file formats (Parquet / Avro / ORC), streaming frameworks (Kafka, Spark Streaming). Familiarity with data lake / warehouse design, data modelling, metadata management, data quality frameworks. Exposure to cloud data engineering (Azure, AWS, GCP) or hybrid architectures is desirable. Good software engineering practices : code quality, unit / integration testing, version control (Git), CI / CD pipelines, containerisation (Docker / Kubernetes) is a plus. Excellent communication and collaboration skills, ability to work independently and mentor others.
Job Title : Senior Data Engineer
Job Location : Bengaluru
Exp Range : 6-10 years
Notice Period : immediate - 15 days
We are looking for Data Engineer with hands on experience on Python, Pyspark, Hadoop Hive, HDFS, Oozie, Yarn
Key Responsibilities
- Design, develop and maintain large-scale, high-performance data pipelines (batch and streaming) to support analytics and data-driven decision-making for retail business.
- Collaborate with cross-functional teams (business stakeholders, analysts, platform / infrastructure teams) to understand data needs, ingest diverse data sources, and deliver data solutions.
- Work on building and evolving a data lake / data warehouse / data platform architecture (including ingestion, transformation, storage, metadata, governance).
- Implement optimal data models, query tuning, performance optimization, ensuring data quality, reliability, and scalability.
- Adopt DevOps / CI-CD practices for data workflows (e.G., automated deployments, monitoring, scheduling).
- Mentor and guide junior engineers, review code / designs, and contribute to architectural decisions and best practices in data engineering.
- Stay current with new data technologies (cloud data services, streaming frameworks, orchestration) and recommend improvements.
Required Skills & Experience
Create a job alert for this search
Data Pipeline Engineer • Bengaluru, Republic Of India, IN
Related jobs
- Promoted
Backend and Data Pipeline Engineer
JRD Systemsbangalore, karnataka, in Job Role : Backend and Data Pipeline Engineer - Python.We’re investing in technology to develop new products that help our customers drive their growth and transformation agenda.These include new da...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer(Azure / AWS, Python / Pyspark, SQL)
Sail Analyticshosur, tamil nadu, in Architect, develop, test and maintain scalable data warehouses and data pipelines.Expertise in SQL, PySpark / Python and Azure(ADB, ADF) or AWS(Glue, Lambda, Redshift).
Bachelor's degree or equivalent...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Acuity Knowledge PartnersBengaluru, Karnataka, India The position requires advanced proficiency in Python and SQL (covering both core programming and data manipulation), hands-on experience with cloud-native platform - Azure, and strong knowledge of ...Show moreLast updated: 2 days ago
- Promoted
Data Engineer
Recrohosur, tamil nadu, in Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake.
AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago
- Promoted
Python Data Engineer
iVoyanthosur, tamil nadu, in One of our clients is looking for an experienced Python Data Engineer to join their team.Strong Python Experience (web services, background jobs.
we use Fast API).Data processing and reporting usin...Show moreLast updated: 13 days ago
- Promoted
Senior Data Engineer (AWS / Databricks)
Accoladehosur, tamil nadu, in The multifamily real estate industry is undergoing a massive transformation, and Accolade is at the forefront.We are building the industry's first AI-native Operations Centralization Platform, desi...Show moreLast updated: 30+ days ago
- Promoted
- New!
Senior Data Engineer
SII Group Indiahosur, tamil nadu, in Databricks, GCP, and Data Pipeline Development.The ideal candidate will have a deep understanding of.Design, develop, and optimize scalable.
Cloud Build, BigQuery, Firestore.Collaborate with cross-f...Show moreLast updated: 8 hours ago
- Promoted
Principal Data Engineer
PriceEasy AIBengaluru, Karnataka, India Join one of the most respected and cutting-edge companies in the pricing and analytics industry! PriceEasy, a VC-backed leader, empowers convenience retailers with actionable insights through advan...Show moreLast updated: 30+ days ago
- Promoted
- New!
AWS Lead Data engineer
Tata Consultancy Serviceshosur, tamil nadu, in In this key leadership role, you will lead the development of foundational components for a Lakehouse architecture on AWS and drive the migration of existing data processing workflows to the new La...Show moreLast updated: 8 hours ago
- Promoted
AWS Data Engineer
Tata Consultancy Serviceshosur, tamil nadu, in TCS has always been in the spotlight for being adept in the next big technologies.What we can offer you is a space to explore varied technologies and quench your techie soul.Required Skill Set : AWS...Show moreLast updated: 11 days ago
- Promoted
Principal Data Engineer
K2 Partnering SolutionsBengaluru, Karnataka, India Job Title : Principal Data / App Engineer, Celonis.Work Location : Bengaluru, Karnataka 560058.Job Type : Full-time / Permanent with the Client directly.
Our Client’s Center for Process Excellence team is ...Show moreLast updated: 30+ days ago
- Promoted
- New!
Lead Data Engineer – Python & GCP
People Prime Worldwidehosur, tamil nadu, in Lead Data Engineer – Python & GCP.You will design, develop, and manage scalable.Design, build, and maintain scalable.Dataflow, BigQuery, Cloud Composer, Cloud Functions, Cloud Run, GCS,.Architect e...Show moreLast updated: 8 hours ago
- Promoted
Data Engineer II
ClearDemandhosur, tamil nadu, in Building on the foundation of the SDE-I role, the DE- II position takes on a greater level of responsibility and leadership.
You'll play a crucial role in driving the evolution and efficiency of our...Show moreLast updated: 2 days ago
- Promoted
Principal Data Engineer
AutodeskBengaluru, Karnataka, India We are seeking a highly experienced.Autodesk’s Analytics Data Platform (ADP).This role requires deep technical expertise in distributed data systems, large-scale pipeline orchestration, and hands-o...Show moreLast updated: 13 days ago
- Promoted
Principal Data Engineer
CodeMyMobileBangalore, IN Experience Required - 7 to 10 Years.Are you a Data Engineer who cares about clean engineering, autonomy, and solving real data challenges? If this sounds like you, we’d love to connect!.Email your ...Show moreLast updated: 2 days ago
- Promoted
Senior Data Engineer – ETL & Pipeline Development (7 to 12 yrs)
AIMLEAPhosur, tamil nadu, in Senior Data Engineer – ETL & Pipeline Development.Remote (Work from Home) / Bangalore / India.Tech / MCA / Computer Science / IT.
IT / Data / AI / LegalTech / Enterprise Solutions.Pandas, Airflow, o...Show moreLast updated: 2 days ago
- Promoted
Data Engineer
IntraEdgehosur, tamil nadu, in Python, PySpark, AWS services (Glue, Lambda), and Snowflake.The ideal candidate will design, build, and maintain scalable data pipelines, ensure efficient data integration, and enable advanced anal...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Response Informaticshosur, tamil nadu, in We’re Hiring | Data Engineering Experts (Hyderabad / Remote).We’re looking for passionate and experienced.Data Engineering professionals.
If you love building scalable data pipelines, optimizing clo...Show moreLast updated: 30+ days ago