Senior Python Data Engineer - ETL

About the Role :

We are seeking an experienced Senior Python Data Engineer with a strong background in building scalable data pipelines, cloud-based data engineering solutions, and advanced reporting systems.

The ideal candidate will be proficient in Python-based development, AWS cloud services, and handling complex time-series data workflows.

This role is hands-on, requiring expertise in data ingestion, transformation, storage, and visualization to support business-critical analytics and reporting Responsibilities :

• Design, develop, and maintain data pipelines using Python and AWS services to process large-scale structured and unstructured datasets.
• Implement and optimize ETL processes for ingesting data from web APIs, IoT devices, and enterprise systems.
• Build scalable and fault-tolerant solutions leveraging AWS IoT SiteWise, Lambda, Glue, and EC2 for real-time and batch processing.
• Manage time-series datasets, ensuring efficient storage, retrieval, aggregation, and analysis for operational and analytical purposes.
• Develop and automate large-scale Excel and CSV-based reports, ensuring accuracy, performance, and usability.
• Integrate Grafana dashboards for real-time monitoring, visualization, and insights.
• Collaborate with data scientists, product managers, and business teams to deliver data-driven solutions aligned with organizational goals.
• Ensure best practices in data governance, quality, and security across all engineering Skills Required & Infrastructure (AWS) :
• Strong expertise in AWS services : IoT SiteWise, Lambda (Python-based), Glue, EC2.
• Experience with AWS data integration and automation using Boto3 Development :
• Advanced Python coding skills with a focus on data engineering.
• Strong experience in time-series data handling and data transformation pipelines.
• Knowledge of PVLIB for solar data modeling and performance analysis (preferred, not mandatory).
• Proven experience in ETL pipelines and integrating with multiple web APIs.
• Hands-on experience in automated Excel report generation at & Reporting :
• Proficiency in Grafana for interactive dashboarding and visualization.
• Expertise in generating CSV & Excel reports, including scheduling and automation.
• Experience in building hourly / daily data aggregation frameworks for Qualifications :
• Strong problem-solving skills with an ability to work independently in a remote setting.
• Experience working in IoT-driven data ecosystems.
• Familiarity with data warehouse concepts and distributed data systems.
• Strong understanding of performance optimization and cost-effective AWS architecture

(ref : hirist.tech)