Role- Data Engineer
Work Mode- Remote
Mandatory Skills - BigQuery, Spark SQL, Microsoft Fabric and who have worked on Big Data as this is a complex project.
Profile Summary
Data Engineer with over 10 years of experience designing and building large-scale data platforms across cloud and on-prem ecosystems. Proven track record of developing high performance ETL pipelines, optimizing data infrastructure, and enabling analytics teams through reliable, well-structured, and secure data systems. Deep expertise in Microsoft Fabric and GCP, with hands-on experience across data ingestion, transformation, modeling, and delivery. Comfortable working in fast-moving environments that require ownership, adaptability, and strong collaboration with analytics and product teams.
Core Skills
Microsoft Fabric
- Designed and implemented pipelines and dataflows (Gen2) to orchestrate data ingestion from multiple structured and semi-structured sources.
- Developed notebooks using Python and Spark SQL for data wrangling, feature engineering, and model-ready transformations.
- Experience in Fabric & Azure DevOps for automated deployment of pipelines, datasets, and reports.
- Defined security and access control strategies in Fabric, ensuring compliance with enterprise data governance standards.
- Built and optimized Direct Lake models for low-latency Power BI reporting.
- Developed advanced DAX measures and tabular models to support complex analytical dashboards.
Google Cloud Platform (GCP)
Experience using BigQuery, improving performance and reducing query costs through partitioning and clustering.Automated workflows using Cloud Functions to handle event-based data transformations and notifications.Integrated real-time streaming data through Pub / Sub, enabling near real-time analytics pipelines.Technical Skills
Languages : Python, SQL, Spark SQLTools : Microsoft Fabric, Power BI, Azure DevOps, Git, GCP Console, Dataflow, DataprocConcepts : Data Modeling (Kimball, Data Vault), ETL / ELT, CI / CD, Data Governance, Security, Performance OptimizationExperience Highlights
Led migration of on-prem ETL workflows to Microsoft FabricBuilt an end-to-end analytics solution combining BigQuery with Power BIDesigned a CI / CD pipeline in Fabric, automating dataset promotion from dev to production with integrated testing and validation.Partnered with data scientists to deliver feature stores and ML-ready datasets using Fabric Notebooks and Spark SQL.Implemented Fabric security model to support row-level access and data masking across multiple business domains.Created a unified metadata catalog and data quality monitoring framework, improving data discoverability and trust.Education
Bachelor’s degree in Computer Science (or related field)