Senior Data EngineerInsight Global • vijayawada, andhra pradesh, in
Senior Data Engineer
Insight Global • vijayawada, andhra pradesh, in
1 day ago
Job descriptionStrong experience building production-grade ETL / ELT pipelines processing terabyte-scale data Knowledge of data integration patterns (full refresh, incremental load, change data capture) Experience with data transformation techniques (normalization, denormalization, aggregation) Understanding of data quality frameworks and validation strategies Proficiency with schema evolution handling changes without breaking downstream systems Experience with data lineage tracking from source to consumption Expert-level SQL skills including advanced analytics functions and query optimization Understanding of database performance tuning (indexing, partitioning, query plans) Experience with relational databases (PostgreSQL, MySQL, SQL Server) for source system integration Familiarity with NoSQL databases (Firestore, Bigtable) for specialized use cases Knowledge of data warehousing concepts (fact tables, dimension tables, slowly changing dimensions) Strong Python proficiency for data pipeline development and scripting Experience with Apache Beam SDK for Dataflow pipeline development Proficiency with Pandas, NumPy for data manipulation and analysis Understanding of object-oriented programming and software engineering best practices Experience with Git for version control and collaborative development Basic Shell scripting for automation and operational tasks Understanding of row-level security implementation patterns in BigQuery Experience with PHI / PII data handling and healthcare compliance requirements (HIPAA preferred) Knowledge of data masking and de-identification techniques Understanding of audit logging and compliance reporting requirements Familiarity with least privilege principles and data access controls Google Cloud Professional Data Engineer certification Healthcare industry experience with understanding of clinical and administrative data Experience with Google Cloud Storage lifecycle policies and storage class optimization Knowledge of Cloud Spanner for transactional workloads Familiarity with Cloud DLP API for automated data classification Experience with dbt (data build tool) for analytics engineering Understanding of data mesh or data fabric architectural patterns Background in DevOps practices and CI / CD for data pipelines Experience with Terraform for infrastructure as code Knowledge of data visualization tools (Looker, Tableau, Power BI) Familiarity with machine learning workflows on GCP (Vertex AI) Experience with Docker and Kubernetes (GKE) for containerized workloads
Summary
The Senior Data Engineer is responsible for building and optimizing ETL / ELT pipelines that process terabytes of data daily across 186 data assets, implementing BigQuery datasets with enterprise-scale performance optimization, and creating the data quality monitoring and transparency dashboards that enable data owner self-service.
Required Qualifications
Google Cloud Platform Data Engineering
- 5+ years of data engineering experience with at least 2+ years focused on Google Cloud Platform
- Strong proficiency with BigQuery including :
- Advanced SQL for analytical queries (window functions, CTEs, complex joins)
- Partitioning and clustering strategies for performance optimization
- Materialized views, authorized views, and query optimization techniques
- Cost optimization through efficient query design and storage management
- Understanding of BigQuery architecture (slot allocation, shuffle operations, distributed execution)
- Hands-on experience with Google Cloud Dataflow and Apache Beam :
- Pipeline development in Python or Java
- Batch and streaming data processing patterns
- Performance tuning and resource optimization
- Error handling and pipeline monitoring
- Proficiency with Cloud Composer (Apache Airflow) :
- DAG development and dependency management
- Airflow operators (BigQueryOperator, DataflowOperator, custom operators)
- Workflow orchestration for complex multi-step processes
- Monitoring and troubleshooting failed workflows
ETL / ELT & Data Integration
SQL & Database Technologies
Programming & Scripting
Data Security & Compliance
Preferred Qualifications
Create a job alert for this search
Senior Data Engineer • vijayawada, andhra pradesh, in
Related jobs
Sr. Data Engineer
HNR Tech • Eluru, IN
We are seeking an experienced Senior Data Engineer to design, build, and optimize our modern data ecosystem.The ideal candidate will have deep expertise in data architecture, pipeline orchestration...Show more
Last updated: 2 days ago • Promoted
Senior Data Engineer
Intellias • Vijayawada, IN
Apache Flink / Apache Spark (Streaming).Data Engineer or similar role, with hands-on expertise in large-scale, production-grade data pipelines.
Kafka + Flink / Spark Streaming).Python for data engin...Show more
Last updated: 9 days ago • Promoted
Senior Data Engineer
Dexian India • Eluru, IN
Minimum 8+ years of hands-on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines.
High proficiency in Python, ...Show more
Last updated: 1 day ago • Promoted
Senior Data Engineer
HealthyMe AI • Eluru, IN
We are seeking a Senior Data Engineer with deep expertise in building data pipelines and integrating healthcare data.You will report to the Head of AI and Engineering and be responsible for designi...Show more
Last updated: 30+ days ago • Promoted
Senior AWS Data Engineer
CYAN360 • Eluru, IN
Position : Senior AWS Data Engineer.Work Timings : 2 : 30 PM to 11 : 30 PM IST.Need someone who can join immediately or in 15 days • • •.
Design, develop, and deploy end-to-end data pipelines on AWS cloud in...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
Mastek • eluru, andhra pradesh, in
We are looking for a Senior Data Engineer with good exposure to Cloud Services in AWS(IAM, RDS, Lambda, DMS) along with Snowflake / Postgresql, ETL and Streaming Services like Kafka.Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
Intellify Solutions • Eluru, IN
Total years of experience : 10+ years.Required : Bachelor’s degree or relevant experience.Required : Prior experience as a software engineer or data engineer in a.
Required : Cloud Data Warehouse experi...Show more
Last updated: 16 hours ago • Promoted • New!
Senior Data Engineer
Saras Analytics • eluru, andhra pradesh, in
We are an ecommerce focused end to end data analytics firm assisting enterprises & brands in data driven decision making to maximize business value.
Our suite of work spans extraction, transformatio...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
Insight Global • eluru, andhra pradesh, in
The Senior Data Engineer is responsible for building and optimizing ETL / ELT pipelines that process terabytes of data daily across 186 data assets, implementing BigQuery datasets with enterprise-sca...Show more
Last updated: 1 day ago • Promoted
Senior Data Engineer
SGS & Co • eluru, andhra pradesh, in
Position Title : Senior Data Engineer.Experience Required : 8 to 12 Years.We are looking for a highly skilled and experienced Data Engineer with strong expertise in.
The ideal candidate will play a ke...Show more
Last updated: 9 days ago • Promoted
Senior Data Engineer
Cloudwick • vijayawada, andhra pradesh, in
The ideal fit for the role will have a strong background in handling large volumes of data with Apache Spark to build , enhance bespoke systems to harness data.
The key focus of the position is to b...Show more
Last updated: 9 hours ago • Promoted • New!
Senior Data Engineer
SAIVA AI • Eluru, IN
We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show more
Last updated: 30+ days ago • Promoted
Data Engineer
Digitalzone • Vijayawada, IN
As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics.
Develop and maintain scalable ETL / ELT pipelines using Py...Show more
Last updated: 9 days ago • Promoted
Senior Data Platform Engineer
Black Dog Labs • Vijayawada, IN
Remote (collaboration across time zones), India or LATAM preferred.Proficient English communication.Data Engineering / Backend Engineering / DevOps.
We’re looking for a hands-on Senior Data Platform...Show more
Last updated: 30+ days ago • Promoted
Data Engineer
Insight Global • Eluru, IN
GCP DATA ENGINEER - Contract (Long term).Data Engineer with hands-on support for Google Looker.Strong experience in data modeling and building data marts.
Proficiency in ETL / ELT pipeline development...Show more
Last updated: 30+ days ago • Promoted
Data Engineer
Recro • Guntur, IN
Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake.
AI Data Enablement : Collaborate wi...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer - Data Acquisition
InfoBeans • Guntur, IN
We are seeking a highly skilled.Senior Data Engineer – Data Acquisition (ODS).The ideal candidate will have extensive hands-on experience in building and optimizing data ingestion and transformatio...Show more
Last updated: 2 days ago • Promoted
Senior Data Engineer
Straive • eluru, andhra pradesh, in
The ideal candidate is a strong software engineer with hands-on experience in Spark (3.You'll be responsible for designing and implementing ETL / ELT solutions, collaborating with teams to deliver da...Show more
Last updated: 30+ days ago • Promoted