Job Summary & Responsibilities :
Roles and Responsibilities :
- Proven experience as a Data Engineer with a focus on GCP.
- Strong proficiency in Python programming for data processing and automation.
- Experience with GCP data services such as Apache Airflow, BigQuery, Dataflow, and Data prep.
- Knowledge of data modeling and database design principles.
- Hands-on experience with ETL processes preferably DBT and data integration.
- Understanding of data governance, quality, and security practices.
- Good experience on SQL and advanced SQL.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills.
- Design and implement scalable data pipelines using GCP services such as BigQuery, Dataflow, Pub / Sub, Cloud Storage, and Composer (Airflow).
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality data solutions.
- Build and maintain ETL / ELT workflows to ingest, transform, and load structured and unstructured data from various sources.
- Optimize performance and cost-efficiency of data pipelines and BigQuery datasets.
- Ensure data quality, governance, and security best practices are followed across all data workflows.
- Monitor data jobs and troubleshoot issues in real-time to ensure high availability.
- Participate in architecture reviews, code reviews, and performance Skills & Qualifications :
- 35+ years of experience in data engineering with at least 2 years of hands-on experience in Google Cloud Platform (GCP).
- Proficiency in SQL and Python or Java for data pipeline development.
- Strong experience with BigQuery, Dataflow, Cloud Storage, Pub / Sub, and Cloud Composer (Apache Airflow).
- Solid understanding of data warehousing, streaming data processing, and data modeling concepts.
- Familiarity with CI / CD pipelines, Terraform / IaC, and Git-based workflows.
- Experience working with large-scale datasets and performance Qualifications :
- GCP Professional Data Engineer Certification.
- Knowledge of Apache Beam, Kafka, or other streaming technologies.
- Familiarity with data security and privacy regulations (e.g., GDPR, HIPAA).
- Experience integrating with BI tools such as Looker, Tableau, or Power BI.
ref : hirist.tech)