Develop and maintain efficient web crawlers with a focus on speed, performance, and scalability .
Write clean, reusable, and optimized Python code for large-scale data extraction.
Analyze and understand website structures, implement logic to handle dynamic and JS-rendered pages .
Troubleshoot and debug crawl issues, optimize request handling, and improve crawler reliability.
Collaborate with backend and data teams to ensure smooth data flow and integration.
Monitor crawler performance, logs, and resolve runtime errors.
Participate in code reviews, documentation, and continuous process improvements.
Skills Required :
Strong coding skills in Python .
Good understanding of how web pages, HTTP, and crawlers work.
Hands-on experience with Docker , Postman , and PowerShell .
Basic understanding of HTML , AngularJS , RabbitMQ , and Celery .
Experience with Git and Bitbucket for version control and team collaboration.
Familiarity with XPath , CSS selectors , and regular expressions for content extraction.
Preferred :
Exposure to Scrapy , Puppeteer / Playwright , or other crawling frameworks.
Understanding of Elasticsearch , HAProxy , or Curator for large-scale data handling.
Create a job alert for this search
Web Developer • belgaum, karnataka, in
Related jobs
Python Developer
PRI Global • belgaum, karnataka, in
Job Title : Python Developer with Snowflake.Bangalore(preferred) / Hyderabad.SQL Developer, Snowflake, Python programming, Spark SQL, Scala, DBT.
Spark SQL, Scala, Python programming.Strong knowledge...Show more
Last updated: 2 hours ago • Promoted • New!
Python Developer – GenAI & Agentic AI
PROADSW3 Relations pvt. ltd. • belgaum, karnataka, in
Hiring : Python Developer – GenAI & Agentic AI | 5–7 Years | Bangalore / Remote | Immediate Joiners.Immediate (Bench resources only).
ESI / Medical Insurance, EPFO Registration, PT Registration, Labour W...Show more
Last updated: 2 hours ago • Promoted • New!
Python Full Stack Developer
RAVSoft Solutions Inc. • belgaum, karnataka, in
RAVSoft Solutions is focused on providing innovative and high-quality software solutions to healthcare professionals.With its rich experience and expertise for 25 years, RAVSoft is one of the leadi...Show more
ThreatXIntel is a startup specializing in cybersecurity services to help businesses and organizations safeguard their digital assets from cyber threats.
Our team offers tailored, cost-effective solu...Show more
Last updated: 2 hours ago • Promoted • New!
Web Crawling Engineer
Forage AI • Belgaum, IN
The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms.
Forage AI is a pioneering A...Show more
Last updated: 13 days ago • Promoted
Python Developer
Tata Consultancy Services • belagavi, karnataka, in
Required Skillset : Python, OOP, Django, Fast API.Core Python Knowledge, which is most important for any framework.Libraries and Frameworks (eg : Django, FastAPI etc).
At least basic knowledge of depl...Show more
Last updated: 30+ days ago • Promoted
Python Developer
eAspire Technolabs Inc. • belgaum, karnataka, in
We Are Hiring : Senior Python Developer + AWS (6+ Years Experience).Onsite for first 3–4 months, then Hybrid Base Location.
We’re looking for an experienced.If you’re passionate about building scalab...Show more
Last updated: 20 days ago • Promoted
Full Stack Web Devoloper
Astroinvite • belgaum, karnataka, in
Full Stack Developer (Shopify + PHP + WordPress) – Immediate Joiner.Noida, Sector 58 (Work From Office).E-commerce / Astrology & Gemstones (AstroInvite & GemsInvite).
We’re looking for a hands-on Fu...Show more
Last updated: 2 hours ago • Promoted • New!
Python Developer (Web Crawling Exp)
DBiz.ai • belagavi, karnataka, in
Python Developer (Web Crawling Exp) - 5+years - Kochi.Develop and maintain efficient.Write clean, reusable, and optimized.
Analyze and understand website structures, implement logic to handle.Collab...Show more
Last updated: 2 hours ago • Promoted • New!
Python Automation & Web Scraping Engineer (2 to 4 yrs)
AIMLEAP • Belgaum, IN
Python Automation & Web Scraping Engineer (WFH).Bachelor’s degree in Computer Science / Information Technology.Selenium, BeautifulSoup, Requests.
Experience in backend / API development using.Strong s...Show more
Last updated: 1 day ago • Promoted
Python Lead developer
Cognizant • belgaum, karnataka, in
Should have 10+ years of Python development experience.Experience with popular Python frameworks such as Django, Flask or Pyramid.
Knowledge of data science and machine learning concepts and tools.A...Show more
Last updated: 2 hours ago • Promoted • New!
Backend Developer
Assima • Belgaum, IN
Assima is a worldwide leader in training, assistance and software adoption for Fortune 500 companies.Our patented technology is trusted by 15 of the top 20 banks, major international organizations ...Show more
Last updated: 2 days ago • Promoted
Software Engineer
Tilda Research • belgaum, karnataka, in
A passion for building scalable AI agents.Build scalable back-end services using.Optimize graph database queries and models in Neo4j.
Collaborate cross-functionally with Product, Engineering, and Cl...Show more
Last updated: 2 days ago • Promoted
Python Developer
GoML • belgaum, karnataka, in
We are in search of a passionate and skilled Python Developer to join our dynamic team.This role is ideal for someone with a profound expertise in Python and its ecosystem, focused on backend devel...Show more
Last updated: 5 days ago • Promoted
Python Backend Developer
Tata Consultancy Services • Belgaum, IN
Bengaluru / Chennai / Hyderabad / Indore / Ahmedabad / Mumbai.Required Technical Skill Set : -.Python, Unix Shell Scripting, OOPS Concepts.
Responsibility of / Expectations from the Role.Experience in atleast ...Show more
Last updated: 9 hours ago • Promoted • New!
Senior Python Developer
Aarav Solutions • belgaum, karnataka, in
Aarav Solutions is a global Product Engineering and IT Consulting services provider.Our goal is to enable organizations in their digital transformation journey, leveraging our domain knowledge and ...Show more
Last updated: 30+ days ago • Promoted
Python Developer
MyData Insights Pvt Ltd • belgaum, karnataka, in
We are seeking a Python / FastAPI Development Lead with strong expertise in FastAPI to join an exciting Reinforcement Learning (RL) Gym project.
You will design, build, and optimize scalable APIs, col...Show more
Last updated: 5 days ago • Promoted
Back End Developer
Digitalzone • Belgaum, IN
We are building a new AI-native engineering team that leverages Vibe Coding, a cutting-edge development workflow infused with AI agent pair programming (e.
Claude Code, Cursor AI, GitHub Copilot, Co...Show more