Develop and maintain efficient web crawlers with a focus on speed, performance, and scalability .
Write clean, reusable, and optimized Python code for large-scale data extraction.
Analyze and understand website structures, implement logic to handle dynamic and JS-rendered pages .
Troubleshoot and debug crawl issues, optimize request handling, and improve crawler reliability.
Collaborate with backend and data teams to ensure smooth data flow and integration.
Monitor crawler performance, logs, and resolve runtime errors.
Participate in code reviews, documentation, and continuous process improvements.
Skills Required :
Strong coding skills in Python .
Good understanding of how web pages, HTTP, and crawlers work.
Hands-on experience with Docker , Postman , and PowerShell .
Basic understanding of HTML , AngularJS , RabbitMQ , and Celery .
Experience with Git and Bitbucket for version control and team collaboration.
Familiarity with XPath , CSS selectors , and regular expressions for content extraction.
Preferred :
Exposure to Scrapy , Puppeteer / Playwright , or other crawling frameworks.
Understanding of Elasticsearch , HAProxy , or Curator for large-scale data handling.
Create a job alert for this search
Web Developer • bhopal, India
Related jobs
Full Stack Developer
WeConnect • bhopal, madhya pradesh, in
We are seeking an experienced Full-Stack Developer to join a dynamic and globally recognized academic institution on a 9-month contractual engagement.
The role involves developing and maintaining we...Show more
Last updated: 3 days ago • Promoted
Python / JavaScript Developer – Web Scraping ( 4 to 7 Years)
AIMLEAP • bhopal, madhya pradesh, in
Python / JavaScript Developer – Web Scraping - 4 to 7 years.Bachelor's degree in Computer Science, Information Technology.
Strong expertise in web scraping with hands-on experience in large-scale data...Show more
Last updated: 30+ days ago • Promoted
Blockchain Developer
StackSmith Consultancy • bhopal, madhya pradesh, in
Omara – Blockchain Development Scope (RFP Summary).Omara is building a next-generation carbon credit platform that operates fully on-chain.
The blockchain layer must support : .Tokenisation of already...Show more
Last updated: 12 days ago • Promoted
Senior Software Engineer
Forage AI • bhopal, madhya pradesh, in
Role Overview (Hands‑on Player‑Coach).You’ll work closely with engineers—reviewing designs and PRs, pairing on tricky problems, and raising the bar on architecture, reliability, security, and perfo...Show more
Last updated: 8 days ago • Promoted
Sr. Python Developer
ToneOpFit • Bhopal, Madhya Pradesh, India
TONEOPFIT is dedicated to keeping you healthy through an extensive variety of nutritious diet plans and recipes tailored to individual needs and regional preferences.
Our team of nutritionists and d...Show more
Last updated: 7 days ago • Promoted
Full Stack Developer (React & Python)
Confidential • Bhopal, India
Confidential (based on experience).Note : This is a requirement for one of Uplers' client - Xpergia).What do you need for this opportunity.
Python, TypeScript, Redux, JavaScript, AWS, Azure.We are lo...Show more
Last updated: 8 days ago • Promoted
Lead Full Stack Developer
Convexicon Software Solutions • bhopal, madhya pradesh, in
We are seeking a highly accomplished Principal Full Stack Developer with over a decade of experience in architecting and delivering large-scale, enterprise-grade software solutions.The ideal candid...Show more
Last updated: 7 hours ago • Promoted • New!
Full Stack Engineer - Reactjs, Python
RiDiK (a Subsidiary of CLPS. Nasdaq : CLPS) • bhopal, madhya pradesh, in
ReactJS, TypeScript, Python and AWS.This role offers the opportunity to work on meaningful projects, collaborate with talented colleagues, and contribute to the success of a growing company.If you’...Show more
Last updated: 3 days ago • Promoted
MEAN Stack Developer
Yakuza E-Vehicles • bhopal, madhya pradesh, in
Experience Required : 3–4 Years.Are you a passionate full-stack developer with hands-on experience in the.Develop, test, and deploy full-stack web applications using the.
Collaborate with UI / UX desig...Show more
Last updated: 7 hours ago • Promoted • New!
Python Developer
VAYUZ Technologies • bhopal, madhya pradesh, in
Develop and maintain scalable backend services using.Build responsive, high-performing front-end interfaces using.Show more
Last updated: 7 hours ago • Promoted • New!
PHP Developer
UniTech Software Solutions • bhopal, madhya pradesh, in
Company : UniTech Software Solutions, Pune.Industry : Software ( ERP Solutions for Packaging and Construction Industries).
Senior PHP ( Work From Office).At UniTech Software Solutions, we empower the ...Show more
Last updated: 7 hours ago • Promoted • New!
Python Developer
Confidential • Bhopal, India
ToneOpFit is dedicated to keeping you healthy through a comprehensive range of nutritious diet plans and recipes.Our diverse team of nutritionists and diet experts provides customized meal plans ta...Show more
Last updated: 27 days ago • Promoted
Content Developer - Payment systems
Talentgigs • bhopal, madhya pradesh, in
Role : Content Developer - Payment systems.Proven experience in content writing for the FinTech industry, with direct exposure to International Payments, SWIFT, or complex financial services messagi...Show more
Last updated: 7 hours ago • Promoted • New!
Sr. Python Developer
Confidential • Bhopal, India
TONEOPFIT is dedicated to keeping you healthy through an extensive variety of nutritious diet plans and recipes tailored to individual needs and regional preferences.
Our team of nutritionists and d...Show more
Last updated: 5 days ago • Promoted
Clay & n8n Expert
HubCredo • bhopal, madhya pradesh, in
Employment Type : Full-Time, 8 hours / day (Contractual).Salary : ₹20,000 – ₹30,000 / month.This is not a coding / software development role.
Please apply only if you are experienced in Clay, n8n, and no-co...Show more
Last updated: 2 days ago • Promoted
Lead Python Developer
Confidential • Bhopal, India
Confidential (based on experience).Full Time Contract for 6 Months(40 hrs a week / 160 hrs a month).Note : This is a requirement for one of Uplers' client - Global leader in data integrity).What do yo...Show more
Last updated: 22 days ago • Promoted
website developer
Lawshield Technologies Private Limited • bhopal, madhya pradesh, in
India LawShield is a technology-driven platform redefining legal assistance in India by combining AI-powered automation with human legal expertise.
Founded by professionals from Artificial Intellige...Show more
Last updated: 7 hours ago • Promoted • New!
Shopify Website Developer
JOKRS • bhopal, madhya pradesh, in
We’re Hiring — Contractual Shopify Website Developer.JOKRS (Premium Streetwear for Gen Alpha).We move fast, design bold, and aim to deliver a world-class e-commerce experience.Shopify development a...Show more