Talent.com
Web Scraping Solutions Engineer

Web Scraping Solutions Engineer

AIMLEAPRepublic Of India, IN
3 days ago
Job description

Python Developer – Web Scraping & Automation (WFH)

Experience : 3-5 Years

Location : Remote

Mode of Engagement : Full-time

No of Positions : 3

Educational Qualifications : Bachelor’s degree in Computer Science / Information Technology

Industry : IT / Software Development

Notice Period : Immediate Joiners Preferred

What We Are Looking For :

  • Strong hands-on experience in Python for large-scale web scraping and automation.
  • Expert in web scraping using Scrapy, BeautifulSoup, Selenium, Playwright, and Requests.
  • Excellent understanding of HTML, CSS, DOM structure , and browser-rendered content.
  • Experience in backend / API development using FastAPI, Django, or Flask .
  • Ability to handle proxies, user-agent rotation, cookies, sessions, and anti-bot mechanisms.
  • Strong debugging, error-handling, and optimization skills.
  • Ability to independently deliver reliable and scalable data extraction solutions.

Key Responsibilities :

  • Develop, optimize, and maintain web scraping scripts and automation pipelines in Python.
  • Extract data from static, dynamic, and JavaScript-heavy websites using browser automation.
  • Manage pagination, authentication of workflows, cookies, sessions , and rendering challenges.
  • Build and maintain REST APIs to support data access and processing.
  • Implement end-to-end pipelines for data cleaning, transformation, and storage (CSV, JSON, DB).
  • Work with SQL / NoSQL databases to store and manage scraped datasets.
  • Use Docker for containerization and deploy scraping tools to cloud environments.
  • Troubleshoot scraping failures, automate recovery, and ensure high data accuracy.
  • Maintain logs, monitoring, and documentation for scraping workflows.

    • Qualifications :

  • Strong programming experience in Python .
  • Proficiency with Scrapy, Beautiful Soup, Selenium, Playwright, and Requests .
  • Understanding HTTP, APIs, headers, cookies, sessions, and web protocols.
  • Experience with backend frameworks : FastAPI / Django / Flask .
  • Experience working with Docker and basic CI / CD workflows is good to have .
  • Knowledge of relational and non-relational databases.
  • Strong debugging, optimization, and troubleshooting abilities.
    • Create a job alert for this search

    Solution Engineer • Republic Of India, IN