Brightedge - Senior Software Engineer - Python

Senior Software Engineer, Remote.

The Software Engineer, Data Ingestion will be a critical individual contributor responsible for designing collection strategies, developing, and maintaining robust and scalable data pipelines.

This role is at the heart of our data ecosystem, deliver new analytical software solution to access timely, accurate, and complete data for insights, products, and operational efficiency.

Key Responsibilities :

• Design, develop, and maintain high-performance, fault-tolerant data ingestion pipelines using Python.
• Integrate with diverse data sources (databases, APIs, streaming platforms, cloud storage, etc.
• Implement data transformation and cleansing logic during ingestion to ensure data quality.
• Monitor and troubleshoot data ingestion pipelines, identifying and resolving issues promptly.
• Collaborate with database engineers to optimize data models for fast consumption.
• Evaluate and propose new technologies or frameworks to improve ingestion efficiency and reliability.
• Develop and implement self-healing mechanisms for data pipelines to ensure continuity.
• Define and uphold SLAs and SLOs for data freshness, completeness, and availability.
• Participate in on-call rotation as needed for critical data pipeline issues.

Key Skills :

Database Fundamentals : Solid understanding of relational databases (SQL, schema design, indexing, query optimization).

Version Control : Proficiency with Git.

Containerization (Plus) :

Streaming Technologies (Plus) :

(ref : hirist.tech)