Senior Python Data Engineer
iVoyantaligarh, uttar pradesh, in4 days ago
Job descriptionStrong proficiency in Python for data processing with hands-on expertise in Pandas. Ability to quickly learn new frameworks such as Polars if needed. Experience building backend services or APIs using frameworks like FastAPI. Solid understanding of data modeling principles (Star Schema) and handling normalized datasets. Familiarity with enterprise messaging patterns and data integration from various sources (API-based and file-based). Experience working with GitHub and CI / CD pipelines (GitHub Actions or similar). Infrastructure-as-Code experience with Bicep or comparable tools (Terraform, AWS CDK). Comfort with spec-driven development and leveraging AI tools like GitHub Copilot for scaffolding. Experience with Polars (not required). Experience with orchestration tools like Apache Airflow or ETL frameworks like DBT. Exposure to Snowflake (streams, tasks, stored procedures). Experience working with DuckDB and / or DocTV / OpenTI XLarge for report generation. Knowledge of Angular or frontend plugins for Excel is a plus but not required. Familiarity with async workflows and distributed processing concepts. Strong ownership mindset, proactive in identifying and solving problems. Collaborative team player, able to work cross-functionally with architects, engineers, and stakeholders. Comfortable working in a distributed, multicultural team across multiple time zones (Europe, India, U.S.). Adaptable and open to learning emerging tools and technologies. Global team with flexibility on candidate location. Current team members are distributed across the US, Latin America, Europe, and India. Candidates from other regions will be considered, provided there is reasonable overlap with core collaboration hours.
About the Project :
Join a dynamic engineering team working on a high-impact tax reporting platform for the 2025 fiscal season. The core goal is to modernize and significantly accelerate the generation of Excel-based reports, reducing report generation time from minutes to seconds. This involves ingesting data from multiple upstream systems, transforming it using efficient data processing libraries, and serving it through APIs. The team emphasizes scalability, maintainability, and developer productivity, incorporating spec-driven development powered by AI tools.
You’ll be contributing to the backend architecture and data pipeline powering this transformation, helping to evolve a high-performance system that's central to the client’s reporting domain.
Responsibilities :
- Design, build, and maintain high-performance data processing pipelines using Python libraries (Pandas required; Polars nice-to-have).
- Develop and expose RESTful APIs using FastAPI or similar frameworks.
- Consume and process normalized Parquet files from multiple upstream sources to generate dynamic Excel reports.
- Contribute to a spec-driven development workflow (using GitHub Copilot, Claude, etc.) to scaffold and generate API / data pipeline code.
- Optimize report generation logic for speed and scalability, currently targeting sub-20 second response times.
- Integrate with messaging and storage mechanisms (e.g., Service Bus, Storage Accounts).
- Collaborate on infrastructure-as-code automation using Bicep similar tools (Terraform, CDK).
- Participate in design discussions for future migration to Snowflake and / or a data lake architecture.
- Contribute to CI / CD pipelines using GitHub Actions.
Required Skills and Experience :
Nice-to-Have / Preferred Qualifications :
Soft Skills & Cultural Fit :
Work Environment / Time Zone Expectations :
Create a job alert for this search
Data Engineer Python • aligarh, uttar pradesh, in
Related jobs
- Promoted
Data Engineer
RecroAligarh, IN Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake.
AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
SAIVA AIAligarh, IN We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
MindlanceAligarh, IN We’re looking for a strong, hands-on Sr Data Engineer who can independently drive client conversations, collaborate with SMEs, and deliver high-quality data solutions in a cloud-native environment....Show moreLast updated: 1 day ago
- Promoted
Senior Data Engineer
IntelliasAligarh, IN Apache Flink / Apache Spark (Streaming).Data Engineer or similar role, with hands-on expertise in large-scale, production-grade data pipelines.
Kafka + Flink / Spark Streaming).Python for data engin...Show moreLast updated: 17 days ago
- Promoted
- New!
AWS Data Engineer
COVET IT INCAligarh, IN Please go through the below requirements and let me know your interest and forward your resume along with your contact information to raja@covetitinc.
We are seeking a highly skilled AWS Data Engine...Show moreLast updated: 2 hours ago
- Promoted
Senior Oracle Data Engineer
Strattek ConsultingAligarh, IN This role centers on architecting robust data pipelines across on-prem and cloud Oracle ecosystems, automating ingestion from varied sources including APIs.
Exposure to Oracle FDI (Fusion Data Intel...Show moreLast updated: 3 days ago
- Promoted
Senior Data Engineer
Intellify SolutionsAligarh, IN Total years of experience : 10+ years.Required : Bachelor’s degree or relevant experience.Required : Prior experience as a software engineer or data engineer in a.
Required : Cloud Data Warehouse experi...Show moreLast updated: 8 days ago
- Promoted
Senior AWS Data Engineer
CYAN360Aligarh, IN Position : Senior AWS Data Engineer.Work Timings : 2 : 30 PM to 11 : 30 PM IST.Need someone who can join immediately or in 15 days • • •.
Design, develop, and deploy end-to-end data pipelines on AWS cloud in...Show moreLast updated: 30+ days ago
- Promoted
AWS Data Engineer
Vista Applied Solutions Group IncAligarh, IN Job Summary for AWS Data Engineer : .Job Qualification and Responsibilities for AWS Data Engineer : .Advanced SQL (performance tuning, joins, windows functions).
Hands-on with AWS : Glue, Lambda, Redshif...Show moreLast updated: 3 days ago
- Promoted
Data Engineer
AceolutionAligarh, IN Data Engineer – Python Expert(Freelance Role).We are looking for a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power our large language model (LLM) developmen...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
IntraEdgeAligarh, IN Python, PySpark, AWS services (Glue, Lambda), and Snowflake.The ideal candidate will design, build, and maintain scalable data pipelines, ensure efficient data integration, and enable advanced anal...Show moreLast updated: 30+ days ago
- Promoted
Senior Python Developer
Aarav Solutionsaligarh, uttar pradesh, in Aarav Solutions is a global Product Engineering and IT Consulting services provider.Our goal is to enable organizations in their digital transformation journey, leveraging our domain knowledge and ...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
DigitalzoneAligarh, IN As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics.
Develop and maintain scalable ETL / ELT pipelines using Py...Show moreLast updated: 17 days ago
- Promoted
Senior Data Platform Engineer
Black Dog LabsAligarh, IN Remote (collaboration across time zones), India or LATAM preferred.Proficient English communication.Data Engineering / Backend Engineering / DevOps.
We’re looking for a hands-on Senior Data Platform...Show moreLast updated: 30+ days ago
- Promoted
AWS Data Engineer
Tata Consultancy Servicesaligarh, uttar pradesh, in TCS is Hiring AWS Data Engineer Bangalore location.Strong hands-on experience in Python programming and PySpark.Experience using AWS services (RedShift, Glue, EMR, S3 & Lambda).Experience working w...Show moreLast updated: 26 days ago
- Promoted
Senior Data Engineer
DonyatiAligarh, IN We are seeking a highly skilled Senior Data Engineer to join our team in building a modern data platform on AWS.You will play a key role in transitioning from legacy systems to a scalable, cloud-na...Show moreLast updated: 3 days ago
- Promoted
AI Engineer (Backend) - Python
Edstem TechnologiesAligarh, IN We are building the next generation of AI-powered applications that deliver actionable financial and marketing insights to our clients.
As an AI Engineer focused on backend development, you'll be in...Show moreLast updated: 3 days ago
- Promoted
Data Engineer
MastekAligarh, IN Deep hands-on experience with Unity Catalog — creating and managing catalogs, schemas, and tables.Experience automating data onboarding and metadata registration via Unity Catalog APIs or Databrick...Show moreLast updated: 11 days ago