Job Summary :
We are immediately hiring a skilled and motivated GCP Cloud Data Engineer for one of our esteemed clients. The ideal candidate should have hands-on experience with Python, Apache Airflow, Google BigQuery, SQL, Apache Spark, and Pub / Sub. You will be responsible for designing, developing, and maintaining robust data pipelines and cloud-based data solutions on Google Cloud Platform (GCP).
Key Responsibilities :
- Design, build, and optimize scalable and efficient data pipelines using Python and Airflow.
- Develop and maintain ETL / ELT workflows to process data across various sources and sinks.
- Work extensively with Google BigQuery for data transformation and analysis.
- Integrate real-time and batch data processing solutions using Pub / Sub and Spark.
- Write complex and optimized SQL queries for large-scale datasets.
- Collaborate with data scientists, analysts, and platform teams to deliver end-to-end data solutions.
- Ensure data quality, performance, and governance across pipelines.
- Monitor, troubleshoot, and optimize data workflows in production environments.
Required Technical Skills :
Programming :
Proficient in Python for data engineering tasks and workflow orchestration.Orchestration Tools :
Experience with Apache Airflow.Cloud Platform :
Strong experience with Google Cloud Platform (GCP) services, particularly :
BigQueryPub / SubData Processing :
Hands-on experience with Apache Spark.Database Skills :
Advanced SQL skills for querying and managing large datasets.Preferred Qualifications :
GCP Data Engineer certification is a plus.Experience with CI / CD pipelines and DevOps practices in data engineering.Familiarity with other GCP services like Cloud Storage, Dataflow, or Dataproc.Strong problem-solving skills and ability to work in a fast-paced, agile environment.(ref : hirist.tech)