Senior Data Engineer - Python / PySpark

Job Title : Senior Data Engineer - Azure | ADF | Databricks | PySpark | AWS

Location : Bangalore, Hyderabad, Chennai (Hybrid Mode)

Experience Required : 5+ Years (Budget around 25 LPA)

Employment Type : Full-Time

Industry : IT Services & Consulting / Data & Analytics

Notice Period : Immediate to 30 days preferred

Job Description :

We are looking for a Senior Data Engineer who is passionate about designing and developing scalable data pipelines, optimizing data architecture, and working with advanced big data tools and cloud platforms. This is a great opportunity to be a key player in transforming data into meaningful insights by leveraging modern data engineering practices on Azure, AWS, and Databricks.

You will be working with cross-functional teams including data scientists, analysts, and software engineers to deliver robust data solutions. The ideal candidate will be technically strong in Azure Data Factory, PySpark, Databricks, and AWS services, and will have experience in building end-to-end ETL workflows and driving business impact through data.

Key Responsibilities :

• Design, build, and maintain scalable and reliable data pipelines and ETL workflows
• Implement data ingestion and transformation using Azure Data Factory (ADF) and Azure Databricks (PySpark)
• Work across multiple data platforms including Azure, AWS, Snowflake, and Redshift
• Collaborate with data scientists and business teams to understand data needs and deliver solutions
• Optimize data storage, processing, and retrieval for performance and cost-effectiveness
• Develop data quality checks and monitoring frameworks for pipeline health
• Ensure data governance, security, and compliance with industry standards
• Lead code reviews, set data engineering standards, and mentor junior team members
• Propose and evaluate new tools and technologies for continuous improvement

Must-Have Skills :

Nice to Have :

Technical Environment :

Soft Skills :

Education & Certifications

Key Result Areas (KRAs) :

Key Performance Indicators (KPIs) :

ref : hirist.tech)