Senior Web Scraping Engineer

About us

We're building a product that relies heavily on collecting structured data from a number of known websites. We need someone experienced who can own this part of the system end-to-end; from writing scrapers to making sure they scale and stay reliable.

This is a senior role. We're looking for someone who has already dealt with the real-world problems of web scraping and can work independently.

What you'll do

Build and maintain scrapers for 24-30 websites (starting with 6).

Handle authenticated sites, sessions, cookies, tokens, etc.

Work with search endpoints and dynamic content.

Architect the data ingestion pipeline, caching, retries, and queues.

Make the scrapers stable and resilient even as websites change.

Collaborate closely with our backend team to define and use APIs.

Handle scaling, rate limits, and performance.

What we're looking for (hard requirements)

5 - 7+ years of building production-grade web scrapers

Strong experience with both API-based scraping and browser automation (Playwright / Puppeteer)

Familiarity with proxies / IP rotation and basic anti-bot handling

Comfortable with auth flows (cookies, sessions, tokens)

Ability to design pipelines : ingestion → clean → store → cache

Can work independently and own the scraper system end-to-end

Nice to have (not required)

Familiarity with Cloudflare's ecosystem.

Python, Node.js (TypeScipt only), or Go; any backend language is fine, TypeScript is preferred.

Who will do well here

You enjoy scrapers that work reliably, and predictably.

You think in terms of systems, not quick patches.

You can work remotely and manage your own time.

You appreciate straightforward communication and minimal bureaucracy.

Compensation

Base : ₹3,00,000 - 4,00,000 per month (36 - 48 LPA). Equity / options are available if you choose to take a lower cash component.