YipitData - Data Engineer - Web Scraping

About Us :

YipitData is the leading market research and analytics firm for the disruptive economy and most recently raised $475M from The Carlyle Group at a valuation of over $1B.

Every day, our proprietary technology analyzes billions of alternative data points to uncover actionable insights across sectors like software, AI, cloud, e-commerce, ridesharing, and payments.

Our data and research teams transform raw data into strategic intelligence, delivering accurate, timely, and deeply contextualized analysis that our customersranging from the worlds top investment funds to Fortune 500 companiesdepend on to drive high-stakes decisions.

From sourcing and licensing novel datasets to rigorous analysis and expert narrative framing, our teams ensure clients get not just data, but clarity and confidence.

We operate globally with offices in the US (NYC, Austin, Miami, Mountain View), APAC (Hong Kong, Shanghai, Beijing, Guangzhou, Singapore), and India.

Our award-winning, people-centric culturerecognized by Inc.

as a Best Workplace for three consecutive yearsemphasizes transparency, ownership, and continuous mastery.

What Its Like to Work at YipitData :

YipitData isnt a place for coasting its a launchpad for ambitious, impact-driven professionals.

From day one, youll take the lead on meaningful work, accelerate your growth, and gain exposure that shapes careers.

Why Top Talent Chooses YipitData :

Ownership That Matters : Youll lead high-impact projects with real business outcomes.

Rapid Growth : We compress years of learning into months.

Merit Over Titles : Trust and responsibility are earned through execution, not tenure.

Velocity with Purpose : We move fast, support each other, and aim highalways with purpose and intention.

If your ambition is matched by your work ethicand you're hungry for a place where growth, impact, and ownership are the normYipitData might be the opportunity youve been waiting for.

About The Role :

We are seeking a Web Scraping Engineer to join our growing engineering team.

In this hands-on role, youll take ownership of designing, building, and maintaining robust web scrapers that power critical reports and customer experiences across our organization.

You will work on complex, high-impact scraping challenges and collaborate closely with cross-functional teams to ensure our data ingestion processes are resilient, efficient, and scalable, while delivering high-quality data to our products and stakeholders.

As Our Web Scraping Engineer You Will :

Refactor and Maintain Web Scrapers.

• Overhaul existing scraping scripts to improve reliability, maintainability, and efficiency.
• Implement best coding practices (clean code, modular architecture, code reviews, etc.) to ensure quality and sustainability.
• Implement Advanced Scraping Techniques.
• Utilize sophisticated fingerprinting methods (cookies, headers, user-agent rotation, proxies) to avoid detection and blocking.
• Handle dynamic content, navigate complex DOM structures, and manage session / cookie lifecycles effectively.
• Collaborate with Cross-Functional Teams.
• Work closely with analysts and other stakeholders to gather requirements, align on targets, and ensure data quality.
• Provide support, documentation, and best practices to internal stakeholders to ensure effective use of our web scraped data in critical reporting workflows.
• Monitor and Troubleshoot.
• Develop robust monitoring solutions, alerting frameworks to quickly identify and address failures.
• Continuously evaluate scraper performance, proactively diagnosing bottlenecks and scaling issues.
• Drive Continuous Improvement.
• Propose new tooling, methodologies, and technologies to enhance our scraping capabilities and processes.
• Stay up to date with industry trends, evolving bot-detection tactics, and novel approaches to web data extraction.
• This is a fully-remote opportunity based in India.
• Standard work hours are from 11am to 8pm IST, but there is flexibility here.

You Are Likely To Succeed If :

(ref : hirist.tech)