Develop and maintain efficient web crawlers with a focus on speed, performance, and scalability .
Write clean, reusable, and optimized Python code for large-scale data extraction.
Analyze and understand website structures, implement logic to handle dynamic and JS-rendered pages .
Troubleshoot and debug crawl issues, optimize request handling, and improve crawler reliability.
Collaborate with backend and data teams to ensure smooth data flow and integration.
Monitor crawler performance, logs, and resolve runtime errors.
Participate in code reviews, documentation, and continuous process improvements.
Skills Required :
Strong coding skills in Python .
Good understanding of how web pages, HTTP, and crawlers work.
Hands-on experience with Docker , Postman , and PowerShell .
Basic understanding of HTML , AngularJS , RabbitMQ , and Celery .
Experience with Git and Bitbucket for version control and team collaboration.
Familiarity with XPath , CSS selectors , and regular expressions for content extraction.
Preferred :
Exposure to Scrapy , Puppeteer / Playwright , or other crawling frameworks.
Understanding of Elasticsearch , HAProxy , or Curator for large-scale data handling.
Create a job alert for this search
Web Developer • amritsar, punjab, in
Related jobs
Senior Python Developer
Rapid Innovation • amritsar, punjab, in
Job Title : Senior Python Developer (5–6 Years Experience).We are looking for an experienced Senior Python Developer who can design, build, and optimise scalable backend systems.You will work closel...Show more
Last updated: 4 hours ago • Promoted • New!
Python Developer – Gen AI & Agentic Frameworks
RapidBrains • amritsar, punjab, in
We are looking for a skilled Python Developer with strong software engineering fundamentals and hands-on experience in building applications using Generative AI and agent-based frameworks.The ideal...Show more
Last updated: 4 hours ago • Promoted • New!
Full Stack Engineer - Reactjs, Python
RiDiK (a Subsidiary of CLPS. Nasdaq : CLPS) • amritsar, punjab, in
ReactJS, TypeScript, Python and AWS.This role offers the opportunity to work on meaningful projects, collaborate with talented colleagues, and contribute to the success of a growing company.If you’...Show more
Last updated: 1 day ago • Promoted
Web Developer
Smart Moves Consultants • amritsar, punjab, in
Design and develop high-performance, responsive web portals using React.Build scalable backend services and APIs with Node.
Integrate and optimize Snowflake for secure data storage, querying, and an...Show more
Last updated: 4 hours ago • Promoted • New!
Python Automation Developer - PAN India
Concentrix • amritsar, punjab, in
Looking for Python Expertise with minimum 6+ years of experience with having below must have skills : .Experience in automating business processes using Python.
Integrate bots with databases (SQL, Pos...Show more
Last updated: 1 day ago • Promoted
Software Developer 1
Interview Kickstart • amritsar, punjab, in
Interview Kickstart (IK) helps software engineers prepare for technical interviews at companies like Google, Meta, Netflix, and Amazon.
We offer a comprehensive career acceleration platform that inc...Show more
Last updated: 30+ days ago • Promoted
Web Crawling Engineer
Forage AI • amritsar, punjab, in
The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms.
Forage AI is a pioneering A...Show more
Last updated: 14 days ago • Promoted
Senior Python Developer
HISH IT SERVICES • amritsar, punjab, in
Role : Senior Python Developer.Location : Remote(Banglore,Chennai,Pune).Pay : 23LPA - 27 LPA(Based on Experience).Timings : A couple of hours overlap with EST, as the client is Canada-based (till 12...Show more
Last updated: 11 days ago • Promoted
Full Stack Developer
WeConnect • amritsar, punjab, in
We are seeking an experienced Full-Stack Developer to join a dynamic and globally recognized academic institution on a 9-month contractual engagement.
The role involves developing and maintaining we...Show more
Last updated: 1 day ago • Promoted
Python Developer
GoML • amritsar, punjab, in
We are in search of a passionate and skilled Python Developer to join our dynamic team.This role is ideal for someone with a profound expertise in Python and its ecosystem, focused on backend devel...Show more
Last updated: 6 days ago • Promoted
Python Full Stack Developer
Luxoft • amritsar, punjab, in
We need an experienced Python Developer to work for a leading investment bank client.Design, develop, and maintain full-stack Python applications with modern frontend frameworks.Build and optimize ...Show more
Last updated: 4 hours ago • Promoted • New!
Backend Developer
Assima • amritsar, punjab, in
Assima is a worldwide leader in training, assistance and software adoption for Fortune 500 companies.Our patented technology is trusted by 15 of the top 20 banks, major international organizations ...Show more
Last updated: 4 days ago • Promoted
AI Engineer (Backend) - Python
Edstem Technologies • amritsar, punjab, in
We are building the next generation of AI-powered applications that deliver actionable financial and marketing insights to our clients.
As an AI Engineer focused on backend development, you'll be in...Show more
Last updated: 7 days ago • Promoted
HYPERVISOR TEST ENGINEER (Foundation Level)
Piepeople Consulting Inc. • amritsar, punjab, in
Solid understanding of hypervisors, virtual machines (VMs), and core concepts like CPU, memory, and I / O allocation.Basic operating systems (especially Linux), hardware basics, and fundamental progr...Show more
Last updated: 2 days ago • Promoted
Senior Python Developer (Programming)
Jobseekerspage.com • amritsar, punjab, in
HIRING - SENIOR PYTHON DEVELOPER.Senior Python Developer (Programming).We are seeking an experienced Python Developer with strong programming skills (not scripting) to develop and optimize data con...Show more
Last updated: 5 days ago • Promoted
Web Developer (Freelance)
Sweet • amritsar, punjab, in
Sweet is the AI-native business platform built for creators — a business partner that clears the clutter, automates the back-office, and gives creators the freedom to focus on craft, while Sweet gr...Show more
Last updated: 1 day ago • Promoted
Senior Azure / Biztalk Developer
Orion Innovation • amritsar, punjab, in
Job Title : Senior Azure Biztalk Developer.Senior Azure Developer with Biztalk.Design, develop, and support enterprise-level integration solutions.
Create technical designs and architectures for comp...Show more
Last updated: 7 days ago • Promoted
Senior Python Developer
Aarav Solutions • amritsar, punjab, in
Aarav Solutions is a global Product Engineering and IT Consulting services provider.Our goal is to enable organizations in their digital transformation journey, leveraging our domain knowledge and ...Show more