Develop and maintain efficient web crawlers with a focus on speed, performance, and scalability .
Write clean, reusable, and optimized Python code for large-scale data extraction.
Analyze and understand website structures, implement logic to handle dynamic and JS-rendered pages .
Troubleshoot and debug crawl issues, optimize request handling, and improve crawler reliability.
Collaborate with backend and data teams to ensure smooth data flow and integration.
Monitor crawler performance, logs, and resolve runtime errors.
Participate in code reviews, documentation, and continuous process improvements.
Skills Required :
Strong coding skills in Python .
Good understanding of how web pages, HTTP, and crawlers work.
Hands-on experience with Docker , Postman , and PowerShell .
Basic understanding of HTML , AngularJS , RabbitMQ , and Celery .
Experience with Git and Bitbucket for version control and team collaboration.
Familiarity with XPath , CSS selectors , and regular expressions for content extraction.
Preferred :
Exposure to Scrapy , Puppeteer / Playwright , or other crawling frameworks.
Understanding of Elasticsearch , HAProxy , or Curator for large-scale data handling.
Create a job alert for this search
Senior Python Developer • belgaum, karnataka, in
Related jobs
Promoted
Senior Python Developer (Programming)
Jobseekerspage.combelagavi, karnataka, in
HIRING - SENIOR PYTHON DEVELOPER.Senior Python Developer (Programming).We are seeking an experienced Python Developer with strong programming skills (not scripting) to develop and optimize data con...Show moreLast updated: 2 days ago
Promoted
Senior Data Engineer
Live Connectionsbelagavi, karnataka, in
Hiring : Data Engineer | 9-12 Years Experience | Remote (India).Python, SQL, Big Data, Cloud, Snowflake.We are looking for an experienced.
The ideal candidate will have deep experience in.Python, SQL...Show moreLast updated: 30+ days ago
Promoted
Senior AI Developer
PioVation GmbHBelgaum, IN
Cloud Operating System and we need someone who can ship.If you like taking AI from prototype → scalable product, this is for you.
Design and ship AI / LLM features that run in production.Build RAG-sty...Show moreLast updated: 19 days ago
Promoted
Senior Data Engineer
IntelliasBelgaum, IN
Apache Flink / Apache Spark (Streaming).Data Engineer or similar role, with hands-on expertise in large-scale, production-grade data pipelines.
Kafka + Flink / Spark Streaming).Python for data engin...Show moreLast updated: 19 days ago
Promoted
Python Developer
GoMLbelagavi, karnataka, in
We are in search of a passionate and skilled Python Developer to join our dynamic team.This role is ideal for someone with a profound expertise in Python and its ecosystem, focused on backend devel...Show moreLast updated: 3 days ago
Promoted
Python Developer
Tata Consultancy Servicesbelagavi, karnataka, in
Required Skillset : Python, OOP, Django, Fast API.Core Python Knowledge, which is most important for any framework.Libraries and Frameworks (eg : Django, FastAPI etc).
At least basic knowledge of depl...Show moreLast updated: 30+ days ago
Promoted
Senior Developer
Virtusabelgaum, karnataka, in
Flexcube Application Developer Extension Development.Job SummaWe are seeking a skilled Flexcube Application Developer with handson experience in developing extensions and customizations for Oracle ...Show moreLast updated: 3 days ago
Promoted
Senior Python Data Engineer
iVoyantbelgaum, karnataka, in
Join a dynamic engineering team working on a high-impact tax reporting platform for the 2025 fiscal season.The core goal is to modernize and significantly accelerate the generation of Excel-based r...Show moreLast updated: 5 days ago
Promoted
Senior Software Engineer
Live Connectionsbelgaum, karnataka, in
We’re Hiring – Senior Engineers.Open Position 1 – Senior Engineer.Strong frontend or strong full-stack expertise.Experience in design / architecture.
Must have worked in a large e-commerce environment...Show moreLast updated: 3 days ago
Promoted
Senior Software Engineer
Forage AIbelgaum, karnataka, in
Role Overview (Hands‑on Player‑Coach).You’ll work closely with engineers—reviewing designs and PRs, pairing on tricky problems, and raising the bar on architecture, reliability, security, and perfo...Show moreLast updated: 3 days ago
Promoted
Senior Python Developer
Aarav Solutionsbelgaum, karnataka, in
Aarav Solutions is a global Product Engineering and IT Consulting services provider.Our goal is to enable organizations in their digital transformation journey, leveraging our domain knowledge and ...Show moreLast updated: 30+ days ago
Promoted
Python Developer
MyData Insights Pvt Ltdbelgaum, karnataka, in
We are seeking a Python / FastAPI Development Lead with strong expertise in FastAPI to join an exciting Reinforcement Learning (RL) Gym project.
You will design, build, and optimize scalable APIs, col...Show moreLast updated: 3 days ago
Promoted
Senior Hadoop Developer
Information Tech Consultantsbelgaum, karnataka, in
UK (Relocation Required - Sponsorship will be provided).Bachelors in Science (IT / Computer Science / Engineer).Scala / Hadoop / Big Data Developer.
The role will focus on building and enhancing data-driv...Show moreLast updated: 1 day ago
Promoted
Senior DBA Developer-Pan India(Hybrid)
DigiHelic Solutions Pvt. Ltd.belgaum, karnataka, in
Job Title-Senior DBA Developer.DBA Developer, Power Shell Scripting, T SQL, Python ( Preferred), DBA activities.At least 4 Years of relevant experience on PowerShell scripting and T- SQL and DBA is...Show moreLast updated: 3 days ago
Promoted
Senior / Lead / Principal SDET - Python automation(Storage)
DDNBelgaum, IN
This is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades.
DataDirect Networks (DDN) is a globa...Show moreLast updated: 6 days ago
Promoted
New!
Senior Python Data Engineer – ETL & Pipeline Development (5 to 12 yrs)
AIMLEAPBelgaum, IN
Senior Python Data Engineer – ETL & Pipeline Development.Remote (WFH) / Bangalore / India.Tech / MCA / Computer Science / IT.
IT / Data / AI / LegalTech / Enterprise Solutions.ETL pipelines and data...Show moreLast updated: less than 1 hour ago
Promoted
Lead Python Data Engineer _ Exp : 7+ Years
Atyeti Incbelagavi, karnataka, in
Bachelor’s or Master’s degree in Computer Science or equivalent experience.Application Developer or in similar software engineering roles.
Python; strong SQL and cloud-native development experience ...Show moreLast updated: 9 days ago
Promoted
Backend Developer
Assimabelagavi, karnataka, in
Assima is a worldwide leader in training, assistance and software adoption for Fortune 500 companies.Our patented technology is trusted by 15 of the top 20 banks, major international organizations ...Show moreLast updated: 1 day ago