GCP Data Engineer

Job description : Data Engineer with Python and GCP

Experience Level : 5 to 9 Years

Loc : Chennai

Must Have Skillset

SQL (4+ Years)

Python or PySpark (4+ Years)

GCP services (3+ Years)

BigQuery

Dataflow or Dataproc

Pub / Sub

Scheduled Query

Cloud Functions

Monitoring Tools Dashboard

Apache Kafka

Terraform scripting (2+ Year)

Airflow / Astronomer / Cloud Composer (2+ Year)

Good to have Skillset

Data fusion

Tekton pipeline / CI-CD Pipeline

GCP :

Cloud Secrete Manager

Soft Skills

Strong all-round communication skills (written / presentation, one-to-one / group) & Excellent analytical, problem-solving and communication skills

Additional Notes :

When hiring for a Data Engineer with GCP experience, it is important to look for candidates with a strong understanding of both data warehousing and data processing technologies.

Roles & Responsibilities

• Design, develop, and maintain scalable data pipelines and architectures in Google Cloud Platform (GCP) using Python and SQL.
• Lead the implementation of data integration solutions utilizing GCP services such as BigQuery, Dataflow, and Pub / Sub.
• Collaborate with cross-functional teams to gather requirements and translate them into technical specifications for data-related projects.
• Implement monitoring and optimization processes to ensure data quality and performance across data workflows.
• Utilize Apache Kafka for real-time data processing and ensure data availability and reliability.
• Develop and manage CI / CD pipelines using Terraform and Tekton for automated deployment.
• Mentor and guide junior data engineers in best practices for data engineering and GCP technologies.
• Create and maintain documentation for data engineering processes, architecture, and workflows.
• Conduct regular performance testing and troubleshooting of data systems to identify and resolve issues swiftly.
• Stay updated with the latest trends and advancements in data engineering and GCP services to drive continuous improvement.

Our ideal candidate

Share your CV for naveena@intellistaff.in