This job offer is not available in your country.

Data Engineer - Google Cloud Platform

TECHOAKS IT SOLUTIONS PRIVATE LIMITEDBangalore

30+ days ago

Job description

Exp : 5+ Yrs

NP : Immediate , 15 Days

Job Description :

Required skills (maximum of 6) :

GCP - Data Engineer
Experience in GCP Projects (Big Query, Data Proc and other GCP Components)
Experience in Python
Experience in Scala

Roles and Responsibilities :

1. Data Pipeline Development :

Design, develop, and maintain robust batch and streaming data pipelines using GCP services like BigQuery, Dataflow, Pub / Sub, and DataProc.

Develop reusable and modular workflows using Cloud Composer (Apache Airflow).

2. Data Transformation and Integration :

Work with business teams to understand data requirements and build transformation logic using Python, PySpark, and SQL.

Implement complex business rules and data quality checks to ensure accurate and reliable outputs.

3. Performance Optimization :

Optimize BigQuery queries for cost and performance using partitioning, clustering, and materialized views.

Tune pipelines and processing logic for large-scale datasets and real-time streaming use cases.

4. Data Management & Infrastructure :

Manage and monitor data storage in Google Cloud Storage (GCS).

Support schema evolution, version control, and metadata handling for structured and semi-structured data.

5. Collaboration & Deployment :

Collaborate with QA, DevOps, Product, and Compliance teams to align data solutions with business KPIs and regulatory standards (e.g., HIPAA, PCI DSS).

Use Google Cloud SDK, Git, and CI / CD tools to automate deployments and reduce manual efforts.

6. Compliance & Monitoring :

Build alerts, retries, and SLAs in Composer DAGs to monitor data pipeline health.

Ensure data compliance and governance practices are followed throughout the lifecycle.

(ref : hirist.tech)

Create a job alert for this search

Data Engineer Google Cloud Platform • Bangalore