Python Data Pipeline Engineer
Sonata SoftwareHyderabad, Republic Of India, IN6 hours ago
Job descriptionPython : Strong experience in writing clean, modular Python code using Pandas, Boto3, and JSON handling. AWS Services : Proficiency in Lambda, S3, CloudWatch, and IAM configurations. Snowflake : Experience with stages, copy into commands, and stored procedures. Error Handling : Expertise in exception management, logging, and retry mechanisms. Version Control : Familiarity with Git or similar tools. Exposure to AWS Step Functions, SNS / SQS, or Glue for orchestration. Experience with data quality checks and audit logging frameworks. Familiarity with CI / CD for Lambda deployments (e.G., GitHub Actions, Jenkins). Knowledge of Snowflake Python Connector or Snowpark. Data Engineering : knowledge of ETL workflows, file-based ingestion, and data validation patterns. Knowledge on data movement between S3 → Lambda → Snowflake to ensure performance and cost efficiency.
Python Developer
Experience : 4+ Years
Location : Hyderabad
Mode : Work From Office
Note : F2F interview is mandatory.
Job Summary :
We are seeking a skilled Python Developer with strong experience in data engineering using AWS Lambda, S3, and Snowflake. The candidate will design and develop automated data ingestion workflows that fetch files from S3, perform multiple validation checks, and load validated data into Snowflake stages and tables.
Key Responsibilities :
- Develop AWS Lambda functions in Python to fetch data files from S3 buckets and load them into Snowflake.
- Implement data validation logic within Lambda functions, in Python
- Design reusable and scalable data ingestion pipelines for incremental and batch loads.
- Manage Snowflake stages, file ingestion, and data load processes using Python and Snowflake SDKs.
- Develop retry logic, logging, and exception handling for resilient Lambda executions.
- Collaborate with architects and QA teams to define data validation frameworks and automation scripts.
- Monitor and debug data pipelines using CloudWatch and Snowflake query logs.
Required Skills :
Good to Have :
Thanks & Regards
Tanaya | Sonata Software Ltd.
Talent Acquisition Specialist
Email : mohana.sritanaya@sonata-software.com
www.Sonata-software.Com
Create a job alert for this search
Data Pipeline Engineer • Hyderabad, Republic Of India, IN
Related jobs
- Promoted
Data Engineer – ETL & Pipeline Development (5 to 10 yrs)
AIMLEAPHyderabad, IN Data Engineer – ETL & Pipeline Development.Remote (Work from Home) / Bangalore / India.Tech / MCA / Computer Science / IT.
IT / Data / AI / LegalTech / Enterprise Solutions.ETL workflows — not just ...Show moreLast updated: 2 days ago
- Promoted
Data Pipeline Engineer
People Tech Group IncHyderabad, Republic Of India, IN We are looking for a skilled and experienced Data Engineer with over 5 years of experience in data engineering and data migration projects.
The ideal candidate should possess strong expertise in SQL...Show moreLast updated: 30+ days ago
- Promoted
Data Pipeline Engineer
ValueLabsHyderabad, Republic Of India, IN Design, develop, and maintain • •ETL / ELT data pipelines • • to ingest data from various sources, including databases and flat files, into our data warehouse.
Utilize AWS services such as • •AWS Lambda • •...Show moreLast updated: 30+ days ago
- Promoted
Python Data Engineer
iVoyantHyderabad, IN One of our clients is looking for an experienced Python Data Engineer to join their team.Strong Python Experience (web services, background jobs.
we use Fast API).Data processing and reporting usin...Show moreLast updated: 18 days ago
- Promoted
Data engineer Python with aws
Tata Consultancy Serviceshyderabad, telangana, in TCS Hiring !!Virtual Drive • • •.TCS - Hyderabad (preferred) / India Locations.Please read Job description before Applying.
NOTE : If the skills / profile matches and interested, please reply to this ema...Show moreLast updated: 1 day ago
- Promoted
Python Developer
LTIMindtreehyderabad, telangana, in Experience : 5 years to 12 years.Notice : Immediate to 30 Days only.Mandatory skills : Python Data Engineer, AWS and services.
Strong programming skills in Python, including libraries like Pandas, NumP...Show moreLast updated: 21 days ago
- Promoted
Data Pipeline Developer
MphasisHyderabad, Republic Of India, IN We are seeking an Python Developer with strong expertise in Python and Databases with 5-7 Yrs exp & hands-on experience in Azure cloud technologies.
The role will focus on migrating processes from t...Show moreLast updated: 30+ days ago
- Promoted
Data Pipeline Engineer
Integrated Personnel Services LimitedHyderabad, Republic Of India, IN Design, develop, and maintain scalable.Work closely with business stakeholders, analysts, and data scientists to understand data requirements and deliver reliable solutions.Optimize ETL workflows f...Show moreLast updated: 19 days ago
- Promoted
AWS Data Pipeline Engineer
Innova SolutionsHyderabad, Republic Of India, IN INNOVA Hiring Immediate Joiner For.AWS Data Engineer with SCD -1 / 2 ,AWS, SQL, Pyspark, Python @ Hyderabad, Chennai, Noida.
JD : The role focuses on ETL development, AWS Cloud technologies (Glue, Athe...Show moreLast updated: 7 days ago
- Promoted
Lead Data Pipeline Engineer
Saras AnalyticsHyderabad, Republic Of India, IN We are an ecommerce focused end to end data analytics firm assisting enterprises & brands in data driven decision making to maximize business value.
Our suite of work spans extraction, transformatio...Show moreLast updated: 30+ days ago
- Promoted
Python Data Engineer
Dexian Indiahyderabad, telangana, in Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights.
Constructing infrastructure for efficient ETL processes from vari...Show moreLast updated: 7 days ago
- Promoted
Backend and Data Pipeline Engineer
JRD SystemsHyderabad, IN Job Role : Backend and Data Pipeline Engineer - Python.We’re investing in technology to develop new products that help our customers drive their growth and transformation agenda.These include new da...Show moreLast updated: 30+ days ago
- Promoted
Lead Data Pipeline Engineer
StraiveHyderabad, Republic Of India, IN The ideal candidate is a strong software engineer with hands-on experience in Spark (3.You'll be responsible for designing and implementing ETL / ELT solutions, collaborating with teams to deliver da...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Pipeline Engineer
People Tech Group IncHyderabad, Republic Of India, IN We are seeking a highly skilled Senior Data Engineer with 5+ years of experience in data engineering and migration projects.
The ideal candidate will have strong expertise in SQL, Python, data model...Show moreLast updated: 30+ days ago
- Promoted
Data Pipeline Engineer
Prudent Technologies and Consulting, Inc.Hyderabad, Republic Of India, IN We are seeking a skilled Data Engineer with strong experience in Python, Snowflake, and AWS.The ideal candidate will be responsible for building and optimizing scalable data pipelines, integrating ...Show moreLast updated: 30+ days ago
- Promoted
Data Pipeline Engineer
Tata Consultancy ServicesHyderabad, Republic Of India, IN TCS Hiring !!Virtual Drive • • •.November • • • • 2 PM to 5 PM Python with AWS.Please read Job description before Applying.
NOTE : If the skills / profile matches and interested, please reply to this email ...Show moreLast updated: 30+ days ago
- Promoted
Data Pipeline Engineer
ExperianHyderabad, Republic Of India, IN Experian Consumer Services – Careers That Define “What’s the Next Big (Data) Thing” for Consumers?.What could be more exciting – personally and professionally – than being part of a “disruptive” bu...Show moreLast updated: 7 days ago
- Promoted
Lead Data Pipeline Engineer
People Tech Group IncHyderabad, Republic Of India, IN Experience : 5+Years (Work from Office).We are looking for a skilled and experienced Data Engineer with over 5 years of experience in data engineering and data migration projects.The ideal candidate...Show moreLast updated: 30+ days ago