Data Engineer (Python)

Location : Ahmedabad, Pune, Bangalore, Thiruvananthapuram

What You Will Do

Work with the team to understand requirements and design / develop algorithms and engine rules using Python.

Use BigQuery as the backend, utilizing both generic SQL / DB entities and BigQuery-specific operations.

Write Bash and PowerShell scripts.

Develop APIs using the Flask framework and implement data ingestion pipelines to handle large volumes of data from various upstream sources.

Design and implement unit tests to ensure code reliability and maintainability.

Analyze, debug, track, and resolve system issues effectively.

Collaborate with the DevSecOps team and use automation tools like Jenkins in scalable production environments.

Apply modern software development practices such as CI / CD, Infrastructure-as-Code, serverless computing, and microservices architecture.

Create Looker Studio dashboards based on the BigQuery backend, ranging from simple to complex configurations involving multiple tables and views.

What Experience You Need

7+ years of experience writing, debugging, and troubleshooting code in Python.

7+ years of experience developing applications using web frameworks such as Flask.

3+ years of experience deploying and releasing software using Jenkins and CI / CD pipelines.

3+ years of experience in data handling, management, and ETL processes.

Experience with data extraction using API calls.

Experience with big data processing solutions using BigQuery, Pub / Sub, GCS, and others.

Proficiency in using source code control systems like GitHub.

What Could Set You Apart

Experience with Angular to enhance project contributions.

Familiarity with JavaScript, JSON, XML, and PowerShell.

Experience with cloud providers such as GCP and AWS, particularly BigQuery.

Knowledge of the Pandas library for data processing, manipulation, and analysis.

Exposure to AI / ML technologies.

Familiarity with Looker Studio or similar data integration tools.

Prior experience developing Operational or Security Metrics.

Knowledge of the Security domain and controls.

Experience working with ServiceNow CMDB / ITAM.

Experience with Agile methodologies (e.G., Scrum, XP).

Excellent communication skills—able to articulate complex technical concepts effectively.

Ability to handle multiple tasks, meet deadlines, and manage internal client expectations.

Self-motivated with a proactive approach and minimal supervision.

Skills

Python, Flask Framework, CI / CD, Big Data