Senior Data EngineerInsight Global • dombivli, maharashtra, in
Senior Data Engineer
Insight Global • dombivli, maharashtra, in
2 days ago
Job descriptionStrong experience building production-grade ETL / ELT pipelines processing terabyte-scale data Knowledge of data integration patterns (full refresh, incremental load, change data capture) Experience with data transformation techniques (normalization, denormalization, aggregation) Understanding of data quality frameworks and validation strategies Proficiency with schema evolution handling changes without breaking downstream systems Experience with data lineage tracking from source to consumption Expert-level SQL skills including advanced analytics functions and query optimization Understanding of database performance tuning (indexing, partitioning, query plans) Experience with relational databases (PostgreSQL, MySQL, SQL Server) for source system integration Familiarity with NoSQL databases (Firestore, Bigtable) for specialized use cases Knowledge of data warehousing concepts (fact tables, dimension tables, slowly changing dimensions) Strong Python proficiency for data pipeline development and scripting Experience with Apache Beam SDK for Dataflow pipeline development Proficiency with Pandas, NumPy for data manipulation and analysis Understanding of object-oriented programming and software engineering best practices Experience with Git for version control and collaborative development Basic Shell scripting for automation and operational tasks Understanding of row-level security implementation patterns in BigQuery Experience with PHI / PII data handling and healthcare compliance requirements (HIPAA preferred) Knowledge of data masking and de-identification techniques Understanding of audit logging and compliance reporting requirements Familiarity with least privilege principles and data access controls Google Cloud Professional Data Engineer certification Healthcare industry experience with understanding of clinical and administrative data Experience with Google Cloud Storage lifecycle policies and storage class optimization Knowledge of Cloud Spanner for transactional workloads Familiarity with Cloud DLP API for automated data classification Experience with dbt (data build tool) for analytics engineering Understanding of data mesh or data fabric architectural patterns Background in DevOps practices and CI / CD for data pipelines Experience with Terraform for infrastructure as code Knowledge of data visualization tools (Looker, Tableau, Power BI) Familiarity with machine learning workflows on GCP (Vertex AI) Experience with Docker and Kubernetes (GKE) for containerized workloads
Summary
The Senior Data Engineer is responsible for building and optimizing ETL / ELT pipelines that process terabytes of data daily across 186 data assets, implementing BigQuery datasets with enterprise-scale performance optimization, and creating the data quality monitoring and transparency dashboards that enable data owner self-service.
Required Qualifications
Google Cloud Platform Data Engineering
- 5+ years of data engineering experience with at least 2+ years focused on Google Cloud Platform
- Strong proficiency with BigQuery including :
- Advanced SQL for analytical queries (window functions, CTEs, complex joins)
- Partitioning and clustering strategies for performance optimization
- Materialized views, authorized views, and query optimization techniques
- Cost optimization through efficient query design and storage management
- Understanding of BigQuery architecture (slot allocation, shuffle operations, distributed execution)
- Hands-on experience with Google Cloud Dataflow and Apache Beam :
- Pipeline development in Python or Java
- Batch and streaming data processing patterns
- Performance tuning and resource optimization
- Error handling and pipeline monitoring
- Proficiency with Cloud Composer (Apache Airflow) :
- DAG development and dependency management
- Airflow operators (BigQueryOperator, DataflowOperator, custom operators)
- Workflow orchestration for complex multi-step processes
- Monitoring and troubleshooting failed workflows
ETL / ELT & Data Integration
SQL & Database Technologies
Programming & Scripting
Data Security & Compliance
Preferred Qualifications
Create a job alert for this search
Senior Data Engineer • dombivli, maharashtra, in
Related jobs
NASDAQ : CMPR) is the world leader in mass customisation.Our unmatched technology, production, and supply chain operations allow us to offer products that can be personalised by an individual custom...Show more
Senior Data Engineer
Celebal Technologies • Navi Mumbai, Maharashtra, India
We are seeking a highly skilled Data Engineer with deep expertise in Apache Kafka integration with Databricks, structured streaming, and large-scale data pipeline design using the Medallion Archite...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
Incept Labs • dombivli, maharashtra, in
Incept Labs is an AI research lab based in San Francisco, California.Our small team of scientists, engineers, and builders is passionate about developing domain-specific, next-generation AI solutio...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer - Snowflake / AWS
Resourcetree • Mumbai
About the Role : A "Senior Data Engineer" is mid-level professional leading the design, build and evolution of the inhouse data platforms.You lead the const...Show more
Last updated: 30+ days ago • Promoted
Collinson Group - Senior Data Engineer
Collinson Loyalty & Benefits Private Limited” • Mumbai
Description : - Design, develop and maintain data pipelines for collecting, transforming and loading data into various data stores.
Build and maintain data warehousing...Show more
Last updated: 17 days ago • Promoted
Senior Data Engineer
Amicon Hub Services • mumbai city, maharashtra, in
Required Skills & Qualifications.Delta Lake, Spark, PySpark, SQL).SQL Server, MongoDB, InfluxDB).Kafka, Azure Event Hubs, or similar).
Excellent problem-solving skills and the ability to work in a f...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer – Snowflake & Azure
Cube Hub Inc. • Mumbai, IN
We are hiring for our US Client.Candidate need to work on US Central time zone.Candidate need to use their Laptop for work .
No work computer provided.Monthly pay in US Dollars ; No Benefits ;.Must...Show more
Last updated: 14 hours ago • Promoted • New!
Senior Data Engineer
Quantiphi • Mumbai, Maharashtra, India
Quantiphi is an award-winning Data Science and Machine Learning Software and Services Company focused on helping organizations translate the big promise of Machine Learning technologies into quanti...Show more
Last updated: 10 days ago • Promoted
Senior Data Engineer
Insight Global • thane, maharashtra, in
The Senior Data Engineer is responsible for building and optimizing ETL / ELT pipelines that process terabytes of data daily across 186 data assets, implementing BigQuery datasets with enterprise-sca...Show more
Last updated: 2 days ago • Promoted
Senior Data Engineer-Remote
Cimpress India • dombivli, maharashtra, in
Remote
Last updated: 30+ days ago • Promoted
Senior Data Engineer
Straive • Mumbai, Maharashtra, India
Analyse relevant internally and externally sourced data (raw data) to generate BI and Advanced.Analytics datasets based on your stakeholders’ requirements.
Design data pipelines to curate sourced da...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
Saras Analytics • navi mumbai, maharashtra, in
We are an ecommerce focused end to end data analytics firm assisting enterprises & brands in data driven decision making to maximize business value.
Our suite of work spans extraction, transformatio...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
Mastek • thane, maharashtra, in
We are looking for a Senior Data Engineer with good exposure to Cloud Services in AWS(IAM, RDS, Lambda, DMS) along with Snowflake / Postgresql, ETL and Streaming Services like Kafka.Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
SGS & Co • navi mumbai, maharashtra, in
Position Title : Senior Data Engineer.Experience Required : 8 to 12 Years.We are looking for a highly skilled and experienced Data Engineer with strong expertise in.
The ideal candidate will play a ke...Show more
Last updated: 10 days ago • Promoted
Senior Data Engineer
HealthyMe AI • Kalyan-Dombivli, IN
We are seeking a Senior Data Engineer with deep expertise in building data pipelines and integrating healthcare data.You will report to the Head of AI and Engineering and be responsible for designi...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
SAIVA AI • Thane, IN
We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
eClerx • thane, India
We are looking for an experienced Senior Data Engineer of hands-on experience in designing, building, and managing complex data architectures and BI solutions.
The ideal candidate will have deep exp...Show more
Last updated: 7 hours ago • Promoted • New!
Senior Data Engineer
Dexian India • Kalyan-Dombivli, IN
Minimum 8+ years of hands-on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines.
High proficiency in Python, ...Show more
Last updated: 2 days ago • Promoted
Senior Data Engineer
RapidBrains • Mumbai, IN
Job Title : Senior Data Engineer.We are looking for a Senior Data Engineer with deep expertise in Azure Data Engineering to design, build, and optimize large-scale data pipelines.The ideal candidate...Show more
Last updated: 1 day ago • Promoted