Principal Data Pipeline Engineer
CoditasPune, Republic Of India, IN15 days ago
Job descriptionProvide research, high-level design and estimates for data transformation and data integration from source applications to end-use Investigate alternatives for data storing and processing to ensure implementation of the most streamlined solutions. Develop comprehensive data products and implement them on cloud or in-house servers for production use. Minimum 6-10 years of progressive experience building solutions in Big Data environments. Have a strong ability to build robust and resilient data pipelines which are scalable, fault tolerant and reliable in terms of data movement. Hands-on experience of Apache Spark with Python for batch and stream processing. Should have knowledge of experience in batch and stream data processing Exposure working on projects across multiple domains. Strong hands on capabilities on SQL and NoSQL technologies. Minimum 3+ years of experience with AWS services like S3, DMS, Redshift, Glue, Lambda, Kinesis, MSK etc. is must have or similar services of either Azure / GCP. Strong analytical / quantitative skills and comfortable working with huge sets of data. Excellent written and verbal communication skills.
Roles and Responsibilities
- Understand Business requirements and implement scalable solutions.
- Translate complex technical and functional requirements into detailed designs
- Develop a highly scalable, reliable, and high-performance data processing pipeline to
extract, transform and load data from various systems to the Enterprise Data warehouse / Data Lake / Data Mesh
Technical Skills
Create a job alert for this search
Data Pipeline Engineer • Pune, Republic Of India, IN
Related jobs
- Promoted
Data Engineer
DigitalzoneNagpur, IN As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics.
Develop and maintain scalable ETL / ELT pipelines using Py...Show moreLast updated: 1 day ago
- Promoted
Data Engineer
Response Informaticsnagpur, maharashtra, in AWS services : Must be proficient in building scalable data pipelines and managing cloud-native ETL workflows.Snowflake : Moderate understanding of Snowflake architecture.
CICD - Terraform or CloudFo...Show moreLast updated: 30+ days ago
- Promoted
Principal Data Engineer
CodeMyMobileNagpur, IN Experience Required - 7 to 10 Years.Are you a Data Engineer who cares about clean engineering, autonomy, and solving real data challenges? If this sounds like you, we’d love to connect!.Email your ...Show moreLast updated: 1 day ago
- Promoted
Backend and Data Pipeline Engineer
JRD SystemsNagpur, IN Job Role : Backend and Data Pipeline Engineer - Python.We’re investing in technology to develop new products that help our customers drive their growth and transformation agenda.These include new da...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer(Azure / AWS, Python / Pyspark, SQL)
Sail Analyticsnagpur, maharashtra, in Architect, develop, test and maintain scalable data warehouses and data pipelines.Expertise in SQL, PySpark / Python and Azure(ADB, ADF) or AWS(Glue, Lambda, Redshift).
Bachelor's degree or equivalent...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer (AWS / Databricks)
Accoladenagpur, maharashtra, in The multifamily real estate industry is undergoing a massive transformation, and Accolade is at the forefront.We are building the industry's first AI-native Operations Centralization Platform, desi...Show moreLast updated: 30+ days ago
- Promoted
GCP Data Engineer
JRD Systemsnagpur, maharashtra, in Collaborate with data architects, analysts, and business teams to define and maintain.Roadnet) and modernized data sources.
Partner with governance and infrastructure teams to establish.Automate and...Show moreLast updated: 6 days ago
- Promoted
Data Engineer
RecroNagpur, IN Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake.
AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Canopus Infosystems - A CMMI Level 3 CompanyNagpur, IN Python expertise and hands-on experience in handling large datasets, data cleaning, analysis, and visualization.The ideal candidate should be capable of building data pipelines, performing web scra...Show moreLast updated: 30+ days ago
- Promoted
AWS Data Engineer
TerraGiGNagpur, IN Design, development, and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR.
Writing reusable, testable, and efficient code.Integration of data storage ...Show moreLast updated: 12 days ago
- Promoted
Python Data Engineer
iVoyantNagpur, IN One of our clients is looking for an experienced Python Data Engineer to join their team.Strong Python Experience (web services, background jobs.
we use Fast API).Data processing and reporting usin...Show moreLast updated: 12 days ago
- Promoted
Python Data Engineer
Dexian Indianagpur, maharashtra, in Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights.
Constructing infrastructure for efficient ETL processes from vari...Show moreLast updated: 1 day ago
- Promoted
Senior Data Engineer – ETL & Pipeline Development (7 to 12 yrs)
AIMLEAPNagpur, IN Senior Data Engineer – ETL & Pipeline Development.Remote (Work from Home) / Bangalore / India.Tech / MCA / Computer Science / IT.
IT / Data / AI / LegalTech / Enterprise Solutions.Pandas, Airflow, o...Show moreLast updated: 1 day ago
- Promoted
Data Engineer
Dexian Indianagpur, maharashtra, in Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights.
Constructing infrastructure for efficient ETL processes from vari...Show moreLast updated: 1 day ago
- Promoted
Data Engineer II
ClearDemandNagpur, IN Building on the foundation of the SDE-I role, the DE- II position takes on a greater level of responsibility and leadership.
You'll play a crucial role in driving the evolution and efficiency of our...Show moreLast updated: 1 day ago
- Promoted
AWS Data Engineer
Tata Consultancy Servicesnagpur, maharashtra, in Aws data engineer having strong experience of Python.Technical / Behavioral Competency.Proficient in Python, with experience in deploying Python packages and OOP, Experience in ingesting data from di...Show moreLast updated: 9 days ago
- Promoted
Azure Data Engineer
LTIMindtreenagpur, maharashtra, in Role Senior Data Engineer 8 years of experience.Build reusable utilities templates and automation pipelines.Design scalable data engineering frameworks standards and best practices.Provide architec...Show moreLast updated: 30+ days ago
- Promoted
Freelance Data Engineer
upGradNagpur, IN We are seeking a highly skilled and motivated.The ideal candidate will be responsible for designing, developing, and optimizing large-scale data pipelines and data warehouse solutions, utilizing a ...Show moreLast updated: 21 days ago