Data Engineer - Google Cloud Platform

Job Title : Data Engineer (GCP)

Location : Pune, Mumbai, Bangalore, Gurgaon, Panchkula, Mohali

Experience : 3+ years

Notice period : Immediate joiner

Job Description :

We are looking for a skilled and passionate Data Engineer with hands-on experience working on Google Cloud Platform (GCP). The ideal candidate will have expertise in building and maintaining data pipelines, utilizing GCP services, and implementing best practices for data ingestion, storage, and processing. If you have strong technical proficiency in Python or Java and enjoy solving complex challenges, we would love to hear from you!

Key Responsibilities :

GCP Services :

- Design, develop, and manage data pipelines leveraging GCP services such as Google Cloud Storage (GCS), PubSub, Dataflow or DataProc, BigQuery, and Airflow/Composer.

- Work on Python (preferred) or Java-based solutions to build robust, scalable, and efficient data pipelines.

ETL on GCP Cloud :

- Develop and maintain ETL pipelines using Python/Java, ensuring data is processed efficiently and accurately.

- Write efficient scripts, adhere to best practices for data ingestion, transformation, and storage on GCP.

- Troubleshoot and solve data pipeline challenges, ensuring high performance and reliability.

Data Ingestion (Batch and Streaming) :

- Implement batch and streaming data ingestion workflows on GCP.

- Design, optimize, and monitor data pipelines for both batch processing and real-time streaming, ensuring smooth and seamless data flow.

Database Expertise :

- Knowledge of relational (SQL) and non-relational (NoSQL) databases both on-premise and in the cloud.

- Understanding the differences between SQL and NoSQL databases and experience working with at least two types of NoSQL databases.

- Expertise in working with databases on GCP, including BigQuery and others.

Data Warehouse Concepts :

- Apply your knowledge of data warehouse concepts to assist in designing efficient storage and processing solutions for large datasets.

- Familiarity with the principles of data warehousing, from data modeling to ETL processes, is required at a beginner to intermediate level.

Technical Skills Required :

GCP Services : GCS, PubSub, Dataflow, DataProc, BigQuery, Airflow/Composer | Programming : Python, Java | ETL Development : Data pipelines, Scripting | Data Ingestion : Batch, Streaming | Databases : SQL, NoSQL | Data Warehousing : Data Warehouse Concepts

(ref:hirist.tech)