Senior Web Scraping Engineer
Zomunkbhopal, madhya pradesh, in22 hours ago
Job description5 - 7+ years of building production-grade web scrapers Strong experience with both API-based scraping and browser automation (Playwright / Puppeteer) Familiarity with proxies / IP rotation and basic anti-bot handling Comfortable with auth flows (cookies, sessions, tokens) Ability to design pipelines : ingestion → clean → store → cache Can work independently and own the scraper system end-to-end Familiarity with Cloudflare's ecosystem. Python, Node.js (TypeScipt only), or Go; any backend language is fine, TypeScript is preferred. You enjoy scrapers that work reliably, and predictably. You think in terms of systems, not quick patches. You can work remotely and manage your own time. You appreciate straightforward communication and minimal bureaucracy.
About us
We're building a product that relies heavily on collecting structured data from a number of known websites. We need someone experienced who can own this part of the system end-to-end; from writing scrapers to making sure they scale and stay reliable.
This is a senior role. We're looking for someone who has already dealt with the real-world problems of web scraping and can work independently.
What you'll do
- Build and maintain scrapers for 24-30 websites (starting with 6).
- Handle authenticated sites, sessions, cookies, tokens, etc.
- Work with search endpoints and dynamic content.
- Architect the data ingestion pipeline, caching, retries, and queues.
- Make the scrapers stable and resilient even as websites change.
- Collaborate closely with our backend team to define and use APIs.
- Handle scaling, rate limits, and performance.
What we're looking for (hard requirements)
Nice to have (not required)
Who will do well here
Compensation
Base : ₹3,00,000 - 4,00,000 per month (36 - 48 LPA). Equity / options are available if you choose to take a lower cash component.
Create a job alert for this search
Senior Web Engineer • bhopal, madhya pradesh, in