This job offer is not available in your country.

Data Engineer - BigQuery, Spark SQL, Microsoft Fabric - 10+ Years - Immediate joiners only

Blue Cloud Softech Solutions LimitedGandhinagar, IN

2 days ago

Job description

Role- Data Engineer

Work Mode- Remote

Mandatory Skills - BigQuery, Spark SQL, Microsoft Fabric and who have worked on Big Data as this is a complex project.

Profile Summary

Data Engineer with over 10 years of experience designing and building large-scale data platforms across cloud and on-prem ecosystems. Proven track record of developing high performance ETL pipelines, optimizing data infrastructure, and enabling analytics teams through reliable, well-structured, and secure data systems. Deep expertise in Microsoft Fabric and GCP, with hands-on experience across data ingestion, transformation, modeling, and delivery. Comfortable working in fast-moving environments that require ownership, adaptability, and strong collaboration with analytics and product teams.

Core Skills

Microsoft Fabric

Designed and implemented pipelines and dataflows (Gen2) to orchestrate data ingestion from multiple structured and semi-structured sources.
Developed notebooks using Python and Spark SQL for data wrangling, feature engineering, and model-ready transformations.
Experience in Fabric & Azure DevOps for automated deployment of pipelines, datasets, and reports.
Defined security and access control strategies in Fabric, ensuring compliance with enterprise data governance standards.
Built and optimized Direct Lake models for low-latency Power BI reporting.
Developed advanced DAX measures and tabular models to support complex analytical dashboards.

Google Cloud Platform (GCP)

Experience using BigQuery, improving performance and reducing query costs through partitioning and clustering.

Automated workflows using Cloud Functions to handle event-based data transformations and notifications.

Integrated real-time streaming data through Pub / Sub, enabling near real-time analytics pipelines.

Technical Skills

Languages : Python, SQL, Spark SQL

Tools : Microsoft Fabric, Power BI, Azure DevOps, Git, GCP Console, Dataflow, Dataproc

Concepts : Data Modeling (Kimball, Data Vault), ETL / ELT, CI / CD, Data Governance, Security, Performance Optimization

Experience Highlights

Led migration of on-prem ETL workflows to Microsoft Fabric

Built an end-to-end analytics solution combining BigQuery with Power BI

Designed a CI / CD pipeline in Fabric, automating dataset promotion from dev to production with integrated testing and validation.

Partnered with data scientists to deliver feature stores and ML-ready datasets using Fabric Notebooks and Spark SQL.

Implemented Fabric security model to support row-level access and data masking across multiple business domains.

Created a unified metadata catalog and data quality monitoring framework, improving data discoverability and trust.

Education

Bachelor’s degree in Computer Science (or related field)

Create a job alert for this search

Data Engineer • Gandhinagar, IN