Big Data Engineer (GCP, Hadoop, PySpark)
ConfidentialHyderabad / Secunderabad, Telangana30+ days ago
Job descriptionBachelor's or Master's degree in Computer Science, Engineering, or related field. 3+ years of experience in big data engineering or related roles. Strong hands-on experience with Google Cloud Platform (GCP) services for big data processing. Proficiency in Hadoop ecosystem tools : HDFS, MapReduce, Hive, HBase, etc. Expert-level knowledge of PySpark for data processing and analytics. Experience with data warehousing concepts and tools such as BigQuery . Good understanding of ETL processes, data modeling, and pipeline orchestration. Programming proficiency in Python and scripting. Familiarity with containerization (Docker) and CI / CD pipelines. Strong analytical and problem-solving skills. Experience with streaming data platforms like Kafka or Pub / Sub . Knowledge of data governance and compliance standards (GDPR, HIPAA). Familiarity with ML workflows and integration with big data platforms. Experience with Terraform or other infrastructure-as-code tools. Certification in GCP Data Engineer or equivalent.
Key Responsibilities :
- Design, develop, and optimize big data pipelines and ETL workflows using PySpark , Hadoop (HDFS, MapReduce, Hive, HBase) .
- Develop and maintain data ingestion, transformation, and integration processes on Google Cloud Platform services such as BigQuery , Dataflow , Dataproc , and Cloud Storage .
- Ensure data quality, security, and governance across all pipelines.
- Monitor and troubleshoot performance issues in data pipelines and storage systems.
- Collaborate with data scientists and analysts to understand data needs and deliver clean, processed datasets.
- Implement batch and real-time data processing solutions.
- Write efficient, reusable, and maintainable code in Python and PySpark.
- Automate deployment and orchestration using tools like Airflow , Cloud Composer , or similar.
- Stay current with emerging big data technologies and recommend improvements.
Qualifications and Requirements :
Desirable Skills :
Skills Required
Gdpr, Hipaa, Pyspark, Python, Hadoop
Create a job alert for this search
Gcp Data Engineer • Hyderabad / Secunderabad, Telangana
Related jobs
- Promoted
Data Engineer – Databricks Platform
Amicon Hub ServicesHyderabad, Telangana, India Delta Lake, Spark, PySpark, SQL).SQL Server, MongoDB, InfluxDB).Kafka, Azure Event Hubs, or similar).Excellent problem-solving skills and the ability to work in a fast-paced environment.Familiar wi...Show moreLast updated: 2 days ago
- Promoted
Senior Data Engineer (AWS,Python, Spark / databricks, SQL)
SID Information TechnologiesHyderabad, Telangana, India Role : Senior Data Engineer (Python, Spark / Databricks, SQL, AWS) Experience : 6–12 years Location : Hyderabad Work Mode : Hybrid (3 days / week in-office) Join Time : Immediate Must-Have Techn...Show moreLast updated: 2 days ago
- Promoted
Big Data Engineer - Scala / PySpark
Sheryl strategic solutions Pvt. LTD.Hyderabad Description : Position : Big Data Engineer Location : Hyderabad, India (Hybrid, 2-3 times per week on any weekdays) Duratio...Show moreLast updated: 21 days ago
- Promoted
Data Engineer - Python / Spark
VariteHyderabad About The Job : - Develops technical tools and programming to cleanse, organize and transform data and to maintain, protect and update data structures and integrity ...Show moreLast updated: 30+ days ago
- Promoted
Big Data Engineer - Hadoop / Spark
ImpacteersHydrebad Position Overview : We are looking for a skilled and detail-oriented Big Data Engineer to design, develop, and maintain scalable data pipelines and architectures.The ...Show moreLast updated: 30+ days ago
- Promoted
Big Data Engineer - Python / PySpark
People Prime World WideHyderabad Key Responsibilities : - Design, develop, and maintain scalable ETL / ELT pipelines using Python and PySpark.Work with large-scale datasets across di...Show moreLast updated: 30+ days ago
- Promoted
Big Data Developer(Hive, Pyspark)
Tata Consultancy Servicessecunderabad, India Big Data Developer(Hive, Pyspark).Years of Experience : 4 to 10 yrs.Date of Interview : 25th Oct, 2025.Mode of Interview : Virtual Drive( Cisco or WebEx).
Hands-on experience of Hadoop, Python, PySpark...Show moreLast updated: 5 days ago
- Promoted
Estuate - GCP Data Engineer - Big Data
Estuate Software Pvt LtdHyderabad Job Title : Data Engineer / Integration Engineer Job Summary : We are seeking a highly skilled Data Engineer / Integration Engineer to join our team.The i...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer - Snowflake DB
YO IT CONSULTINGHyderabad Job Description : Title : Senior Data Engineer (Snowflake) Experience : 3 to 6 years &...Show moreLast updated: 30+ days ago
- Promoted
Senior Engineer, Big Data Engineer
ConfidentialHyderabad / Secunderabad, Telangana We are a Digital Product Engineering company that is scaling in a big way! We build products, services, and experiences that inspire, excite, and delight.
We work at scale across all devices and dig...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer – Big Data & AWS (Python + PySpark)
CoforgeHyderabad, Telangana, India We are hiring for the position of Senior Data Engineer – Big Data & AWS (Python + PySpark) at Coforge Ltd.Job Location : Greater Noida & Hyderabad Only.
For queries, WhatsApp : 9667427662.If you are a...Show moreLast updated: 22 days ago
- Promoted
Tao Digital - Big Data Engineer - Scala / Python
Tao Digital India Private LimitedHyderabad Immediate Joiners Job Description : Highly skilled Big Data Engineer with expertise in distributed systems, and advanced pro...Show moreLast updated: 30+ days ago
- Promoted
Big Data Engineer - Python / SQL / ETL
CURATALHyderabad Key Responsibilities : - Design, develop, and support robust ETL pipelines to extract, transform, and load data into analytical products that driv...Show moreLast updated: 30+ days ago
- Promoted
Softility - Big Data Engineer - Apache Spark
SoftilityHyderabad Big Data Engineer : - Design, build, and optimize robust and scalable data pipelines for both batch and real-time data i...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer - Python / Spark
HIMFLAX INFORMATION TECHNOLOGIES PRIVATE LIMITEDHyderabad Key Responsibilities : Data Architecture & Pipeline Development : - Design, implement, and mainta...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
StraiveHyderabad, Telangana, India Job Summary We're looking for a Senior Data Engineer with 5-8 years of experience to build and maintain scalable, production-grade data pipelines.
The ideal candidate is a strong software engineer...Show moreLast updated: 30+ days ago
- Promoted
Big Data Engineer - SQL / Python
ANB IT Solutions Pvt LtdHyderabad Job Description : - Minimum of 7+ years of progressive experience in Data Engineering, with a proven track record in pi...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer - Snowflake DB
PRUDENT GLOBALTECH SOLUTIONS PRIVATE LIMITEDHyderabad Description : the job : Job Description : We are s...Show moreLast updated: 21 days ago