No longer accepting applications
Senior Data Engineer
Insight GlobalAnantapur, IN30+ days ago
Job descriptionStrong experience building production-grade ETL / ELT pipelines processing terabyte-scale data Knowledge of data integration patterns (full refresh, incremental load, change data capture) Experience with data transformation techniques (normalization, denormalization, aggregation) Understanding of data quality frameworks and validation strategies Proficiency with schema evolution handling changes without breaking downstream systems Experience with data lineage tracking from source to consumption Expert-level SQL skills including advanced analytics functions and query optimization Understanding of database performance tuning (indexing, partitioning, query plans) Experience with relational databases (PostgreSQL, MySQL, SQL Server) for source system integration Familiarity with NoSQL databases (Firestore, Bigtable) for specialized use cases Knowledge of data warehousing concepts (fact tables, dimension tables, slowly changing dimensions) Strong Python proficiency for data pipeline development and scripting Experience with Apache Beam SDK for Dataflow pipeline development Proficiency with Pandas, NumPy for data manipulation and analysis Understanding of object-oriented programming and software engineering best practices Experience with Git for version control and collaborative development Basic Shell scripting for automation and operational tasks Understanding of row-level security implementation patterns in BigQuery Experience with PHI / PII data handling and healthcare compliance requirements (HIPAA preferred) Knowledge of data masking and de-identification techniques Understanding of audit logging and compliance reporting requirements Familiarity with least privilege principles and data access controls Google Cloud Professional Data Engineer certification Healthcare industry experience with understanding of clinical and administrative data Experience with Google Cloud Storage lifecycle policies and storage class optimization Knowledge of Cloud Spanner for transactional workloads Familiarity with Cloud DLP API for automated data classification Experience with dbt (data build tool) for analytics engineering Understanding of data mesh or data fabric architectural patterns Background in DevOps practices and CI / CD for data pipelines Experience with Terraform for infrastructure as code Knowledge of data visualization tools (Looker, Tableau, Power BI) Familiarity with machine learning workflows on GCP (Vertex AI) Experience with Docker and Kubernetes (GKE) for containerized workloads
Summary
The Senior Data Engineer is responsible for building and optimizing ETL / ELT pipelines that process terabytes of data daily across 186 data assets, implementing BigQuery datasets with enterprise-scale performance optimization, and creating the data quality monitoring and transparency dashboards that enable data owner self-service.
Required Qualifications
Google Cloud Platform Data Engineering
- 5+ years of data engineering experience with at least 2+ years focused on Google Cloud Platform
- Strong proficiency with BigQuery including :
- Advanced SQL for analytical queries (window functions, CTEs, complex joins)
- Partitioning and clustering strategies for performance optimization
- Materialized views, authorized views, and query optimization techniques
- Cost optimization through efficient query design and storage management
- Understanding of BigQuery architecture (slot allocation, shuffle operations, distributed execution)
- Hands-on experience with Google Cloud Dataflow and Apache Beam :
- Pipeline development in Python or Java
- Batch and streaming data processing patterns
- Performance tuning and resource optimization
- Error handling and pipeline monitoring
- Proficiency with Cloud Composer (Apache Airflow) :
- DAG development and dependency management
- Airflow operators (BigQueryOperator, DataflowOperator, custom operators)
- Workflow orchestration for complex multi-step processes
- Monitoring and troubleshooting failed workflows
ETL / ELT & Data Integration
SQL & Database Technologies
Programming & Scripting
Data Security & Compliance
Preferred Qualifications
Create a job alert for this search
Senior Data Engineer • Anantapur, IN
Related jobs
%20LIMITED)
- Promoted
Senior Data Engineer
Tata Consultancy ServicesAnantapur, IN Required Technical Skill Set : Senior Data Engineer.Desired Experience Range : 7 - 12 yrs.Notice Period : Immediate to 90Days only.
We are currently planning to do a Virtual.Primary Skills (Java, Pytho...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer - Data Acquisition
InfoBeansAnantapur, IN We are seeking a highly skilled.Senior Data Engineer – Data Acquisition (ODS).The ideal candidate will have extensive hands-on experience in building and optimizing data ingestion and transformatio...Show moreLast updated: 2 days ago
- Promoted
Senior Data Engineer
Dexian IndiaAnantapur, IN Minimum 8+ years of hands-on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines.
High proficiency in Python, ...Show moreLast updated: 1 day ago
- Promoted
Senior Data Engineer
HealthyMe AIAnantapur, IN We are seeking a Senior Data Engineer with deep expertise in building data pipelines and integrating healthcare data.You will report to the Head of AI and Engineering and be responsible for designi...Show moreLast updated: 30+ days ago
- Promoted
Senior AWS Data Engineer
CYAN360Anantapur, IN Position : Senior AWS Data Engineer.Work Timings : 2 : 30 PM to 11 : 30 PM IST.Need someone who can join immediately or in 15 days • • •.
Design, develop, and deploy end-to-end data pipelines on AWS cloud in...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Platform Engineer
Black Dog LabsAnantapur, IN Remote (collaboration across time zones), India or LATAM preferred.Proficient English communication.Data Engineering / Backend Engineering / DevOps.
We’re looking for a hands-on Senior Data Platform...Show moreLast updated: 30+ days ago
- Promoted
- New!
Senior Data Engineer
KOGTA FINANCIAL (INDIA) LIMITEDAnantapur, IN ETL & Data Warehouse Developer.As a key member of our data engineering team, you will be responsible for designing, developing, and optimizing ETL pipelines and scalable data warehouse solutions on...Show moreLast updated: 9 hours ago
- Promoted
Senior Data Engineer
Mastekananthapur, andhra pradesh, in We are looking for a Senior Data Engineer with good exposure to Cloud Services in AWS(IAM, RDS, Lambda, DMS) along with Snowflake / Postgresql, ETL and Streaming Services like Kafka.Show moreLast updated: 30+ days ago
- Promoted
- New!
Senior Data Engineer
RapidBrainsAnantapur, IN Job Title : Senior Data Engineer.We are looking for a Senior Data Engineer with deep expertise in Azure Data Engineering to design, build, and optimize large-scale data pipelines.The ideal candidate...Show moreLast updated: 9 hours ago
- Promoted
- New!
Senior Data Engineer
CloudwickAnantapur, IN The ideal fit for the role will have a strong background in handling large volumes of data with Apache Spark to build , enhance bespoke systems to harness data.
The key focus of the position is to b...Show moreLast updated: 9 hours ago
- Promoted
Senior Data Engineer
Saras Analyticsananthapur, andhra pradesh, in We are an ecommerce focused end to end data analytics firm assisting enterprises & brands in data driven decision making to maximize business value.
Our suite of work spans extraction, transformatio...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
SAIVA AIAnantapur, IN We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
IntelliasAnantapur, IN Apache Flink / Apache Spark (Streaming).Data Engineer or similar role, with hands-on expertise in large-scale, production-grade data pipelines.
Kafka + Flink / Spark Streaming).Python for data engin...Show moreLast updated: 9 days ago
- Promoted
Senior Data Engineer
Insight Globalananthapur, andhra pradesh, in The Senior Data Engineer is responsible for building and optimizing ETL / ELT pipelines that process terabytes of data daily across 186 data assets, implementing BigQuery datasets with enterprise-sca...Show moreLast updated: 1 day ago
- Promoted
Sr. Data Engineer
HNR TechAnantapur, IN We are seeking an experienced Senior Data Engineer to design, build, and optimize our modern data ecosystem.The ideal candidate will have deep expertise in data architecture, pipeline orchestration...Show moreLast updated: 2 days ago
- Promoted
Data Engineer
People Prime WorldwideAnantapur, IN Our Client is a global IT services company headquartered in Southborough, Massachusetts, USA.Founded in 1996, with a revenue of $1.
B, with 35,000+ associates worldwide, specializes in digital engin...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
DigitalzoneAnantapur, IN As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics.
Develop and maintain scalable ETL / ELT pipelines using Py...Show moreLast updated: 9 days ago
- Promoted
Data Engineer
Insight GlobalAnantapur, IN GCP DATA ENGINEER - Contract (Long term).Data Engineer with hands-on support for Google Looker.Strong experience in data modeling and building data marts.
Proficiency in ETL / ELT pipeline development...Show moreLast updated: 30+ days ago