Note : This is a remote opportunity
Requirements :
- Proven experience of 3 to 7 years as a Web Scraping Specialist or similar role, with a track record of successful web scraping projects.
- Expertise in handling dynamic content, user-agent rotation, bypass CAPTCHAs, rate limits, and utilization of proxy services.
- Knowledge on browser fingerprinting
- Has leadership experience.
- Proficiency in programming languages commonly used for web scraping, such as Python, BeautifulSoup, Scrapy, or Selenium.
- Strong knowledge of HTML, CSS, XPath, and other web technologies relevant to web scraping and Coding.
- Knowledge and experience in best of class data storage and retrieval of large volume of scraped data.
- Understanding of web scraping best practices, including handling dynamic content, user-agent rotation, and IP address management.
- Attention to detail and the ability to handle and process large volumes of data accurately.
- Familiarity with data cleansing techniques and data validation processes.
- Good communication skills and the ability to collaborate effectively with cross-functional teams.
- Knowledge of web scraping ethics, legal considerations, and compliance with website terms of service.
- Strong problem-solving skills and the ability to adapt to changing web environments
Preferred Qualifications :
Bachelor's degree in Computer Science, Data Science, Information Technology, or related fields.Experience with cloud-based solutions and distributed web scraping systems.Familiarity with APIs and data extraction from non-public sources.Knowledge of machine learning techniques for data extraction and natural language processing is desired but not mandatoryPrior experience in handling large-scale data projects and working with big data frameworks.Understanding of various data formats such as JSON, XML, CSV, etc.Experience with version control systems like Git.(ref : hirist.tech)