Senior Data EngineerInsight Global • hosur, tamil nadu, in
Senior Data Engineer
Insight Global • hosur, tamil nadu, in
2 days ago
Job descriptionStrong experience building production-grade ETL / ELT pipelines processing terabyte-scale data Knowledge of data integration patterns (full refresh, incremental load, change data capture) Experience with data transformation techniques (normalization, denormalization, aggregation) Understanding of data quality frameworks and validation strategies Proficiency with schema evolution handling changes without breaking downstream systems Experience with data lineage tracking from source to consumption Expert-level SQL skills including advanced analytics functions and query optimization Understanding of database performance tuning (indexing, partitioning, query plans) Experience with relational databases (PostgreSQL, MySQL, SQL Server) for source system integration Familiarity with NoSQL databases (Firestore, Bigtable) for specialized use cases Knowledge of data warehousing concepts (fact tables, dimension tables, slowly changing dimensions) Strong Python proficiency for data pipeline development and scripting Experience with Apache Beam SDK for Dataflow pipeline development Proficiency with Pandas, NumPy for data manipulation and analysis Understanding of object-oriented programming and software engineering best practices Experience with Git for version control and collaborative development Basic Shell scripting for automation and operational tasks Understanding of row-level security implementation patterns in BigQuery Experience with PHI / PII data handling and healthcare compliance requirements (HIPAA preferred) Knowledge of data masking and de-identification techniques Understanding of audit logging and compliance reporting requirements Familiarity with least privilege principles and data access controls Google Cloud Professional Data Engineer certification Healthcare industry experience with understanding of clinical and administrative data Experience with Google Cloud Storage lifecycle policies and storage class optimization Knowledge of Cloud Spanner for transactional workloads Familiarity with Cloud DLP API for automated data classification Experience with dbt (data build tool) for analytics engineering Understanding of data mesh or data fabric architectural patterns Background in DevOps practices and CI / CD for data pipelines Experience with Terraform for infrastructure as code Knowledge of data visualization tools (Looker, Tableau, Power BI) Familiarity with machine learning workflows on GCP (Vertex AI) Experience with Docker and Kubernetes (GKE) for containerized workloads
Summary
The Senior Data Engineer is responsible for building and optimizing ETL / ELT pipelines that process terabytes of data daily across 186 data assets, implementing BigQuery datasets with enterprise-scale performance optimization, and creating the data quality monitoring and transparency dashboards that enable data owner self-service.
Required Qualifications
Google Cloud Platform Data Engineering
- 5+ years of data engineering experience with at least 2+ years focused on Google Cloud Platform
- Strong proficiency with BigQuery including :
- Advanced SQL for analytical queries (window functions, CTEs, complex joins)
- Partitioning and clustering strategies for performance optimization
- Materialized views, authorized views, and query optimization techniques
- Cost optimization through efficient query design and storage management
- Understanding of BigQuery architecture (slot allocation, shuffle operations, distributed execution)
- Hands-on experience with Google Cloud Dataflow and Apache Beam :
- Pipeline development in Python or Java
- Batch and streaming data processing patterns
- Performance tuning and resource optimization
- Error handling and pipeline monitoring
- Proficiency with Cloud Composer (Apache Airflow) :
- DAG development and dependency management
- Airflow operators (BigQueryOperator, DataflowOperator, custom operators)
- Workflow orchestration for complex multi-step processes
- Monitoring and troubleshooting failed workflows
ETL / ELT & Data Integration
SQL & Database Technologies
Programming & Scripting
Data Security & Compliance
Preferred Qualifications
Create a job alert for this search
Senior Data Engineer • hosur, tamil nadu, in
Related jobs
Senior Data Engineer
Insight Global • bangalore district, karnataka, in
The Senior Data Engineer is responsible for building and optimizing ETL / ELT pipelines that process terabytes of data daily across 186 data assets, implementing BigQuery datasets with enterprise-sca...Show more
Last updated: 2 days ago • Promoted
Senior Data Engineer
Straive • hosur, tamil nadu, in
The ideal candidate is a strong software engineer with hands-on experience in Spark (3.You'll be responsible for designing and implementing ETL / ELT solutions, collaborating with teams to deliver da...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
Dexian India • Bangalore, IN
Minimum 8+ years of hands-on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines.
High proficiency in Python, ...Show more
Last updated: 2 days ago • Promoted
Senior Data Engineer
Saras Analytics • hosur, tamil nadu, in
We are an ecommerce focused end to end data analytics firm assisting enterprises & brands in data driven decision making to maximize business value.
Our suite of work spans extraction, transformatio...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
Societe Generale Global Solution Centre • Bengaluru, Karnataka, India
With 7 - 10 years of experience, you will design and maintain scalable data pipelines, ensure compliance with global data regulations (including GDPR), and empower auditors / inspectors with high-qua...Show more
Last updated: 23 days ago • Promoted
Senior Data Engineer
Mastek • bangalore, karnataka, in
We are looking for a Senior Data Engineer with good exposure to Cloud Services in AWS(IAM, RDS, Lambda, DMS) along with Snowflake / Postgresql, ETL and Streaming Services like Kafka.Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
SGS & Co • hosur, tamil nadu, in
Position Title : Senior Data Engineer.Experience Required : 8 to 12 Years.We are looking for a highly skilled and experienced Data Engineer with strong expertise in.
The ideal candidate will play a ke...Show more
Last updated: 10 days ago • Promoted
Senior Data Analytics Engineer
Insight Global • hosur, tamil nadu, in
We’re looking for a creative and driven Senior Analytics Engineer to join our high-performing Data Analytics team in a fast-paced Healthcare IT environment.
This is a hands-on role for someone who’s...Show more
Last updated: 10 days ago • Promoted
Senior Data Engineer
Incept Labs • hosur, tamil nadu, in
Incept Labs is an AI research lab based in San Francisco, California.Our small team of scientists, engineers, and builders is passionate about developing domain-specific, next-generation AI solutio...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
Sonata Software • Bengaluru, Karnataka, India
Experience building data pipelines.Highly proficient in python, Data frames.Experience with AWS Lambda, Events (SQS), API calls, Kinesis, DynamoDB.
Highly proficient in SQL Query.Snowflake or experi...Show more
Last updated: 29 days ago • Promoted
Senior Data Engineer
ACL Digital • Bengaluru, Karnataka, India
Tech Lead design, build, and maintain the infrastructure for an organization's data, ensuring it is reliable, accessible, and optimized for use by data scientists and analysts.Key responsibilities ...Show more
Last updated: 8 days ago • Promoted
Senior Data Engineer - Databricks
Tredence Inc. • Bengaluru, Karnataka, India
Tredence is a global data science solutions provider founded in 2013 by Shub Bhowmick, Sumit Mehra, and Shashank Dubey focused on solving the last-mile problem in AI.
Headquartered in San Jose, Cali...Show more
Last updated: 5 days ago • Promoted
Senior Data Engineer
SAIVA AI • hosur, tamil nadu, in
We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
HealthyMe AI • hosur, tamil nadu, in
We are seeking a Senior Data Engineer with deep expertise in building data pipelines and integrating healthcare data.You will report to the Head of AI and Engineering and be responsible for designi...Show more
Last updated: 30+ days ago • Promoted
SENIOR DATA ENGINER
Tredence Inc. • Bengaluru, Karnataka, India
Primary Roles and Responsibilities : .Developing Modern Data Warehouse solutions using Databricks and Azure Stack.Ability to provide solutions that are forward-thinking in data engineering and analyt...Show more
Last updated: 19 days ago • Promoted
Senior Data Engineer
Pretium Enterprise Services, India • Bengaluru, Karnataka, India
Would be a lead responsible for developing data products & solutions for the organization.IT with experience in Data warehouse, ETL, BI projects is required.
Would collaborate within Agile teams to ...Show more
Last updated: 7 days ago • Promoted
Senior Data Engineer
Persistent Systems • hosur, tamil nadu, in
Data Engineer with Pyspark, SQL, ETL Data Engineer.Job Type : Full Time Employment.Candidate should have 5+ years of experience in Data Engineering.
Must have experience in Pyspark, SQL, ETL, data pi...Show more
Last updated: 1 hour ago • Promoted • New!
Senior Data Engineer
RapidBrains • hosur, tamil nadu, in
Job Title : Senior Data Engineer.We are looking for a Senior Data Engineer with deep expertise in Azure Data Engineering to design, build, and optimize large-scale data pipelines.The ideal candidate...Show more
Last updated: 1 day ago • Promoted