No longer accepting applications
Senior Data Pipeline Architect
BrightEdgeRepublic Of India, IN30+ days ago
Job description4+ years of experience, ideally with background in Computer Science , working in software product companies. Extensive Python Expertise : Extensive experience in developing robust, production-grade applications with Python. Data Collection & Integration : Proven experience collecting data from various sources (REST APIs, OAuth, GraphQL, Kafka, S3, SFTP, etc.). Distributed Systems & Scalability : Strong understanding of distributed systems concepts, designing for scale, performance optimization, and fault tolerance. Cloud Platforms : Experience with major cloud providers (AWS or GCP) and their data-related services (e.G., S3, EC2, Lambda, SQS, Kafka, Cloud Storage, GKE). Database Fundamentals : Solid understanding of relational databases (SQL, schema design, indexing, query optimization). OLAP database experience is a plus (Hadoop) Monitoring & Alerting : Experience with monitoring tools (e.G., Prometheus, Grafana) and setting up effective alerts. Version Control : Proficiency with Git. Containerization (Plus) : Experience with Docker and Kubernetes. Streaming Technologies (Plus) : Experience with real-time data processing using Kafka, Flink, Spark Streaming.
Senior Software Engineer, Remote
The Software Engineer, Data Ingestion will be a critical individual contributor responsible for designing collection strategies, developing, and maintaining robust and scalable data pipelines . This role is at the heart of our data ecosystem, deliver new analytical software solution to access timely, accurate, and complete data for insights, products, and operational efficiency.
Key Responsibilities :
- Design, develop, and maintain high-performance, fault-tolerant data ingestion pipelines using Python.
- Integrate with diverse data sources (databases, APIs, streaming platforms, cloud storage, etc.).
- Implement data transformation and cleansing logic during ingestion to ensure data quality.
- Monitor and troubleshoot data ingestion pipelines, identifying and resolving issues promptly.
- Collaborate with database engineers to optimize data models for fast consumption.
- Evaluate and propose new technologies or frameworks to improve ingestion efficiency and reliability.
- Develop and implement self-healing mechanisms for data pipelines to ensure continuity.
- Define and uphold SLAs and SLOs for data freshness, completeness, and availability.
- Participate in on-call rotation as needed for critical data pipeline issues
Key Skills :
Create a job alert for this search
Data Pipeline Architect • Republic Of India, IN
Related jobs
- Promoted
Principal / Senior Data Architect
AaysNagpur, IN Position : Principal / Senior Data Architect.You will act as a key member of the consulting team helping Clients to re-invent their corporate finance function by leveraging advanced analytics.You wil...Show moreLast updated: 30+ days ago
- Promoted
Solution & Data Architect - Azure Applications
InfotechtionNagpur, IN Solution & Data Architect - Azure Applications .Location : India (Remote) | Employment Type : Full-time .Senior - 8-11+ Years of experience.
We are seeking an accomplished Solution and Data Architect ...Show moreLast updated: 3 days ago
- Promoted
- New!
Senior Data Architect
MacroHireNagpur, IN Job Profile- Sr Data Architect.Type of job- Contract for 1 year (extendable).Data Architect will focus on efficiency, security, and performance.
Responsibilities include monitoring, optimization, tr...Show moreLast updated: 8 hours ago
- Promoted
GCP Data Architect
Tata Consultancy Servicesnagpur, maharashtra, in Design and Implement Data Architectures : Architect and build scalable, end-to-end data solutions on GCP, encompassing data ingestion, transformation, storage, and consumption.Develop Data Pipelines...Show moreLast updated: 24 days ago
- Promoted
Senior Data Engineer
Guidanz IncNagpur, IN BI Connector is the industry leading solution for integrating Oracle Fusion Cloud data into modern BI platforms like Power BI, Tableau, and Data Warehouse, without complex ETL.Our Data Architecture...Show moreLast updated: 3 days ago
- Promoted
Enterprise Data Architect
DBiz.aiNagpur, IN You will ensure that data is modeled, structured and integrated across the organization to support analytics and AI / ML initiatives at scale.
You will work closely with business stakeholders and tech...Show moreLast updated: 14 days ago
- Promoted
Senior Azure Data Engineer (Architect) - Pan India
Genpactnagpur, maharashtra, in We are seeking an experienced Azure Data & Analytics Architect with a strong background in delivery and excellent communication skills.
The ideal candidate will have experience and a proven track re...Show moreLast updated: 16 days ago
- Promoted
Data Pipeline Architect
IntraEdgeRepublic Of India, IN We are seeking a highly skilled Data Engineer with strong experience in Python, PySpark, Snowflake, and AWS Glue to join our growing data team.
You will be responsible for building scalable and reli...Show moreLast updated: 30+ days ago
- Promoted
Lead Data Pipeline Architect
CYGNVSChennai, Republic Of India, IN Our company culture is a perfect blend of productivity and enjoyment, prioritizing the growth and development of our talented employees as part of a high-achieving team.
We are in search of exceptio...Show moreLast updated: 3 days ago
- Promoted
Data Pipeline Architect
Solytics PartnersPune, Republic Of India, IN Solytics Partners is a Global Analytics firm, recognized with multiple industry awards for innovation and excellence.Our team comprises experts with deep knowledge in risk, analytics, AI / ML, AML / FC...Show moreLast updated: 16 days ago
- Promoted
Data Pipeline Architect
BrightEdgeRepublic Of India, IN BrightEdge is a global leader in enterprise SEO and content performance solutions, driving AI-powered digital marketing success for the world’s top brands.
Our culture is product-first, innovation-d...Show moreLast updated: 30+ days ago
- Promoted
Data Architect
RecroNagpur, IN Central Data Office – Data Architect.SMEs, engineers, and developers to design and govern enterprise-level data architectures, ensuring alignment with corporate data principles and standards.This p...Show moreLast updated: 3 days ago
- Promoted
AI Data Architect
Tata Consultancy Servicesnagpur, maharashtra, in We are seeking an inventive Data Architect for AI with 8–15 years of experience to lead the strategic design and implementation of enterprise-scale AI solutions.
ETL / ELT, Big Data cloud data solutio...Show moreLast updated: 20 days ago
- Promoted
- New!
Data Pipeline Architect
thinkbridgePune, Republic Of India, IN We are a global digital product development firm that helps growth-stage companies gain the technology sophistication and maturity of leading modern digital businesses.
We differentiate ourselves by...Show moreLast updated: 8 hours ago
- Promoted
Data Architect
CareerUS SolutionsNagpur, IN The Data Architect is responsible for.This includes ensuring that data is.The role focuses on developing.Collaborate with software engineers, analysts, and stakeholders to understand data needs and...Show moreLast updated: 2 days ago
- Promoted
- New!
Azure Data Architect
Programmers.ionagpur, maharashtra, in We offer a vibrant and collaborative work environment, cutting-edge tools and technologies, and ample opportunities for professional growth.
Job Title : Azure Data Architect.Experience required : 15+ ...Show moreLast updated: 2 hours ago
- Promoted
Data Architect
Tata Consultancy Servicesnagpur, maharashtra, in Role- Snowflake Data Architect.Cloud data warehouse technologies on Snowflake and AWS, Azure or GCP.At least one End-to-end Snowflake implementation is a must covering all aspects including archite...Show moreLast updated: 30+ days ago
- Promoted
Data Pipeline Architect
Tata Consultancy ServicesChennai, Republic Of India, IN Minimum 3+ Years of experience Hadoop, Hive SQL’s, Spark, Bigdata Eco System Tools.Preferable to have good technical knowledge on Cloud computing, AWS or Azure Cloud Services.Strong conceptual and ...Show moreLast updated: 30+ days ago