Senior Python Developer
DBiz.aiaurangabad, maharashtra, in1 hour ago
Job descriptionStrong coding skills in Python . Good understanding of how web pages, HTTP, and crawlers work. Hands-on experience with Docker , Postman , and PowerShell . Basic understanding of HTML , AngularJS , RabbitMQ , and Celery . Experience with Git and Bitbucket for version control and team collaboration. Familiarity with XPath , CSS selectors , and regular expressions for content extraction. Exposure to Scrapy , Puppeteer / Playwright , or other crawling frameworks. Understanding of Elasticsearch , HAProxy , or Curator for large-scale data handling.
Responsibilities :
- Develop and maintain efficient web crawlers with a focus on speed, performance, and scalability .
- Write clean, reusable, and optimized Python code for large-scale data extraction.
- Analyze and understand website structures, implement logic to handle dynamic and JS-rendered pages .
- Troubleshoot and debug crawl issues, optimize request handling, and improve crawler reliability.
- Collaborate with backend and data teams to ensure smooth data flow and integration.
- Monitor crawler performance, logs, and resolve runtime errors.
- Participate in code reviews, documentation, and continuous process improvements.
Skills Required :
Preferred :
Create a job alert for this search
Senior Python Developer • aurangabad, maharashtra, in