Data Engineer
StraiveHyderabad, India1 day ago
Job description5–8 years industry experience as a Data Engineer. Strong hands-on experience with Apache Spark 3.x (development, tuning, debugging). Proficient in Scala programming for Spark ETL / ELT development (datasets, RDDs when required). Experience with streaming frameworks like Spark Structured Streaming, Kafka. Proficient in Python programming. Advanced SQL skills (complex queries, window functions, CTEs, query optimization). Experience building production-grade ETL / ELT workflows, testing and monitoring them. Experience with orchestration tools like Apache Airflow Strong software engineering fundamentals : version control (Git), modular code, code reviews, CI / CD pipelines. Experience with cloud object storage and common data formats (Parquet / ORC / Avro). Strong problem solving, communication and collaboration skills. Experience with AWS Glue, EMR, Athena, Redshift, S3 will be a plus.
Key Responsibilities :
- Design, build and maintain scalable batch and (optionally) streaming data pipelines using Apache Spark 3.x, Scala, Python & SQL
- Implement core ETL / ELT logic in Scala and Python; author efficient Spark DataFrame / Dataset jobs.
- Write and optimize complex SQL for ingestion, transformation and consumption layers.
- Tune Spark jobs for performance and cost : partitioning, join strategies, broadcast, memory / tuning, shuffle reduction.
- Ensure code quality via unit tests, integration tests, CI / CD and code reviews.
- Work with data modeling, schema evolution and data quality checks to ensure reliable outputs.
- Collaborate with platform / DevOps teams to deploy and monitor pipelines (retries, logging, alerting).
- Troubleshoot production issues and perform root-cause analysis.
- Mentor and guide junior engineers, share best practices and drive improvements to the data platform.
Qualifications :
Create a job alert for this search
Data Engineer • Hyderabad, India
Related jobs
- Promoted
Data Engineer – Databricks Platform
Amicon Hub ServicesHyderabad, Telangana, India Delta Lake, Spark, PySpark, SQL).SQL Server, MongoDB, InfluxDB).Kafka, Azure Event Hubs, or similar).Excellent problem-solving skills and the ability to work in a fast-paced environment.Familiar wi...Show moreLast updated: 25 days ago
- Promoted
Data Engineer (OnPrem)
BlendHyderabad, Telangana, India Senior Data Engineers / Data Engineers.Python / Scala, and modern data engineering tools to help design and implement an end-to-end data architecture for a leading enterprise client.This role is part o...Show moreLast updated: 13 days ago
- Promoted
- New!
Data Engineer
One Tapp Consultinghyderabad, telangana, in Design and Development : Design, build, and maintain robust, scalable, and optimized ETL / ELT.Python and standard data warehousing principles.
Data Platform Management : Implement and manage data proce...Show moreLast updated: 20 hours ago
- Promoted
Data Engineer
DigitalzoneHyderabad, IN As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics.
Develop and maintain scalable ETL / ELT pipelines using Py...Show moreLast updated: 15 days ago
- Promoted
Data Engineer
RecroHyderabad, IN Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake.
AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Prudent Technologies and Consulting, Inc.Hyderabad, Telangana, India Job Summary : Seeking an experienced Senior Data Engineer with over 9 years of industry experience to architect, build, and manage advanced data pipelines and platforms in a cloud environment.The i...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
EgenHyderabad, Telangana, India Job Overview : We are looking for a skilled and motivated Lead Data Engineer with strong experience in Python programming and Google Cloud Platform (GCP) to join our data engineering team.The ideal...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
KPI PartnersHyderabad, Telangana, India KPI Partners is seeking a talented Data Engineer with expertise in STIBO (STEP) development to join our dynamic team.The ideal candidate will be responsible for designing, developing, and implement...Show moreLast updated: 12 days ago
- Promoted
Data Engineer
Technogen India Pvt. Ltd.Hyderabad, Telangana, India The Impact : Design, build, and measure complex ELT jobs to process disparate data sources and form a high integrity, high quality, clean data asset.
Executes and provides feedback for data modeling ...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
HCLTechHyderabad, Telangana, India Data Engineer - Databricks Job Description.The Data Engineer - Databricks will be responsible for building and optimizing our data pipelines, architectures, and data sets.He will work closely with ...Show moreLast updated: 12 days ago
- Promoted
Data Engineer - Snowflake
Prudent Technologies and Consulting, Inc.hyderabad, telangana, in We are seeking a skilled Data Engineer with strong experience in Python, Snowflake, and AWS.The ideal candidate will be responsible for building and optimizing scalable data pipelines, integrating ...Show moreLast updated: 1 day ago
- Promoted
AWS Data Engineer
Atyeti Inchyderabad, telangana, in Looking for Data Engineer who will be responsible for design, build and maintenance of data pipelines running on Airflow, Spark on the AWS Cloud platform.
Build and maintain all facets of Data Pipel...Show moreLast updated: 23 hours ago
- Promoted
Data Engineer
Idyllic Serviceshyderabad, telangana, in We are looking for an enthusiastic GCP Data Engineer to join our dynamic and growing project team.In this role, you will have the opportunity to develop your skills while contributing to large-scal...Show moreLast updated: 23 hours ago
- Promoted
Data Platform Engineer
Prudent Technologies and Consulting, Inc.Hyderabad, Republic Of India, IN Seeking an experienced Senior Data Engineer with over 9 years of industry experience to architect, build, and manage advanced data pipelines and platforms in a cloud environment.The ideal candidate...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
StraiveHyderabad, Telangana, India Design, build and maintain scalable.Implement core ETL / ELT logic in Scala and Python; author efficient Spark DataFrame / Dataset jobs.
Write and optimize complex SQL for ingestion, transformation and ...Show moreLast updated: 1 day ago
- Promoted
Data Engineer
Azilen Technologiessecunderabad, telangana, in Manage and create design schema, SQL query tuning, and code review.Min 4+ years of professional experience in the field of data engineering with knowledge of the data platform and DWH development.D...Show moreLast updated: 23 hours ago
- Promoted
Lead Data Engineer
ValueMomentumHyderabad, Republic Of India, IN We are seeking a Senior Databricks Data Engineer with experience in designing and developing data pipelines using Azure Databricks, Data Factory, and Datalake.
The role involves managing large volum...Show moreLast updated: 27 days ago
- Promoted
Data Engineer
Insight GlobalHyderabad, IN GCP DATA ENGINEER - Contract (Long term).Data Engineer with hands-on support for Google Looker.Strong experience in data modeling and building data marts.
Proficiency in ETL / ELT pipeline development...Show moreLast updated: 30+ days ago