Senior Python Data Engineer

About the Project :

Join a dynamic engineering team working on a high-impact tax reporting platform for the 2025 fiscal season. The core goal is to modernize and significantly accelerate the generation of Excel-based reports, reducing report generation time from minutes to seconds. This involves ingesting data from multiple upstream systems, transforming it using efficient data processing libraries, and serving it through APIs. The team emphasizes scalability, maintainability, and developer productivity, incorporating spec-driven development powered by AI tools.

You’ll be contributing to the backend architecture and data pipeline powering this transformation, helping to evolve a high-performance system that's central to the client’s reporting domain.

Responsibilities :

• Design, build, and maintain high-performance data processing pipelines using Python libraries (Pandas required; Polars nice-to-have).
• Develop and expose RESTful APIs using FastAPI or similar frameworks.
• Consume and process normalized Parquet files from multiple upstream sources to generate dynamic Excel reports.
• Contribute to a spec-driven development workflow (using GitHub Copilot, Claude, etc.) to scaffold and generate API / data pipeline code.
• Optimize report generation logic for speed and scalability, currently targeting sub-20 second response times.
• Integrate with messaging and storage mechanisms (e.g., Service Bus, Storage Accounts).
• Collaborate on infrastructure-as-code automation using Bicep similar tools (Terraform, CDK).
• Participate in design discussions for future migration to Snowflake and / or a data lake architecture.
• Contribute to CI / CD pipelines using GitHub Actions.

Required Skills and Experience :

Nice-to-Have / Preferred Qualifications :

Soft Skills & Cultural Fit :

Work Environment / Time Zone Expectations :