Talent.com
This job offer is not available in your country.
Data Engineer - Web Scrapping & Enrichment

Data Engineer - Web Scrapping & Enrichment

Gala IntelligencePalakkad, IN
30+ days ago
Job description

We're looking for an entrepreneurial, passionate, and driven Data Engineer to join Startup Gala Intelligence backed by Navneet Tech Venture situated in Ahmedabad . As we're building our technology platform from scratch, you'll have the unique opportunity to shape our technology vision, architecture, and engineering culture right from the ground up. You’ll directly contribute to foundational development and establish best practices, while eventually building and contributing to our engineering team.

This role is ideal for someone eager to own the entire tech stack, who thrives on early-stage challenges, and loves building innovative, scalable solutions from day zero.

What You’ll Do

  • Web Scraping & Crawling : Build and maintain automated scrapers to extract structured and unstructured data from websites, APIs, and public datasets.
  • Scalable Scraping Systems : Develop multi-threaded, distributed crawlers capable of handling high-volume data collection without interruptions.
  • Data Parsing & Cleaning : Normalize scraped data, remove noise, and ensure consistency before passing to data pipelines.
  • Anti-bot & Evasion Tactics : Implement proxy rotation, captcha solving, and request throttling techniques to handle scraping restrictions.
  • Integration with Pipelines : Deliver clean, structured datasets into NoSQL stores and ETL pipelines for further enrichment and graph-based storage.
  • Data Quality & Validation : Ensure data accuracy, deduplicate records, and maintain a trust scoring system for data confidence.
  • Documentation & Maintenance : Keep scrapers updated when websites change, and document scraping logic for reproducibility.

Who You Are

Technical Skills :

  • 2+ years of experience in web scraping , crawling, or data collection.
  • Strong proficiency in Python (libraries like BeautifulSoup, Scrapy, Selenium, Playwright, Requests).
  • Familiarity with NoSQL databases (MongoDB, DynamoDB) and data serialization formats (JSON, CSV, Parquet).
  • Experience in handling large-scale scraping with proxy management and rate-limiting.
  • Basic knowledge of ETL processes and integration with data pipelines.
  • Exposure to graph databases (Neo4j) is a plus.
  • Soft Skills :

  • Detail-oriented, ensuring accuracy and reliability of collected data.
  • Strong problem-solving skills, particularly in adapting scrapers to evolving web structures.
  • Curious mindset with a drive to discover new data sources.
  • Comfortable working in a fast-paced, early-stage startup environment.
  • Who We Are & Our Culture

    Gala Intelligence , backed by Navneet Tech Ventures , is a tech-driven startup dedicated to solving one of the most pressing business challenges - fraud detection and prevention. We're building cutting-edge, real-time products designed to empower consumers and businesses to stay ahead of fraudsters, leveraging innovative technology and deep domain expertise.

    Our culture and values :

    We’re united by a single, critical mission - stopping fraud before it impacts businesses. Curiosity, innovation, and proactive action define our approach. We value transparency, collaboration, and individual ownership, creating an environment where talented people can do their best work.

  • Problem-Driven Innovation : We're deeply committed to solving real challenges that genuinely matter for our customers.
  • Rapid Action & Ownership : We encourage autonomy and accountability—own your projects, move quickly, and shape the future of Gala Intelligence.
  • Collaborative Excellence : Cross-team collaboration ensures alignment, sparks innovation, and drives us forward together.
  • Continuous Learning : Fraud evolves rapidly, and so do we. Continuous improvement, experimentation, and learning are core to our success.
  • If you're excited by the opportunity to leverage technology in the fight against fraud, and you're ready to build something impactful from day one, we want to hear from you!

    Create a job alert for this search

    Data Engineer • Palakkad, IN

    Related jobs
    • Promoted
    AI Engineer

    AI Engineer

    GumoCoimbatore, IN
    Design, build, and maintain our AI-driven knowledge graph creation pipeline.Integrate third-party APIs for web scraping (e. Tavily), and multimedia processing.Develop and refine sophisticated prompt...Show moreLast updated: 3 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Bahwan CyberTekCoimbatore, IN
    Job Title : Data Engineer – Google Cloud Platform (GCP).We are seeking a skilled and motivated Data Engineer with hands-on experience in building scalable data pipelines and cloud-native data soluti...Show moreLast updated: 16 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Insight GlobalThrissur, IN
    Notice Period : This is a ASAP hire - only apply if you can start 2 weeks after offer is extended!.Priority scheduling for candidates who message on with resume. Design, develop, and maintain robust...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Innodata Inc.Thrissur, IN
    CI / CD practices, Databricks (Spark), Python, Github and SQL.The ideal candidate should have hands-on expertise in building and automating data pipelines, managing multi-environment deployments, and...Show moreLast updated: 24 days ago
    • Promoted
    Data Engineer

    Data Engineer

    EveriseCoimbatore, IN
    Join us on our mission to elevate customer experiences for people around the world.As a member of the Everise family, you will be part of a global experience company that believes in being people-f...Show moreLast updated: 5 days ago
    • Promoted
    Data Engineer

    Data Engineer

    XebiaPalakkad, IN
    We’re Hiring : Data Engineer | Xebia.Any Xebia location (Hybrid, 3 days office per week).Immediate to 2 weeks – only apply if you can join early. Databricks, Python, SQL, and Postgres.The ideal candi...Show moreLast updated: 4 days ago
    • Promoted
    Backend Developer / Data Scraping / Kubernetes

    Backend Developer / Data Scraping / Kubernetes

    Daply | Scaling Digital PublishingThrissur, IN
    IN ORDER TO BE CONSIDERED FOR THIS ROLE, YOU MUST SUBMIT AN APPLICATION HERE : .Web Scraping / Captcha Control.Show moreLast updated: 5 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Kumaran SystemsCoimbatore, IN
    Strong hands-on experience in Databricks.Proven expertise in building and managing data ingestion pipelines.Exposure to Databricks data ingestion jobs along with incident management frameworks.Expe...Show moreLast updated: 6 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    BrightEdgeThrissur, IN
    BrightEdge About Us BrightEdge is a leading SEO and content performance marketing platform that transforms online content into tangible business results. Our platform processes massive amounts of da...Show moreLast updated: 30+ days ago
    • Promoted
    GenAI Engineer

    GenAI Engineer

    XebiaThrissur, IN
    Any Xebia Location (Hybrid, 3 days office per week).Building and deploying GenAI solutions leveraging.Collaborating with global teams under US overlap hours. AWS, GenAI, Bedrock, AgenticAI.Ability t...Show moreLast updated: 25 days ago
    • Promoted
    Senior Web Scraping Engineer

    Senior Web Scraping Engineer

    S2T AI - AI-Powered InvestigationsThrissur, IN
    We are on the lookout for a highly competent, self-motivated.Gather and process raw data at scale (including writing scripts, web scraping, calling APIs). Able to work independently to complete assi...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Otomeyt AICoimbatore, IN
    We are seeking a highly skilled 5+.The ideal candidate will have strong technical expertise in Azure, Data Engineering tools, and advanced ETL design along with excellent communication and problem-...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    IntraEdgeCoimbatore, IN
    Snowflake, AWS (Lambda, Glue), DBT, and SQL.The ideal candidate will be responsible for enabling seamless data integration, transformation, and analytics to support business intelligence and advanc...Show moreLast updated: 30+ days ago
    • Promoted
    Python AWS Data Engineer

    Python AWS Data Engineer

    Digitrix Software LLPCoimbatore, IN
    Python, AWS Python (core language skill) Backend, Pandas, PySpark (DataFrame API), interacting with AWS (e.Data Processing : Spark (PySpark), Glue, EMR AWS Core Services : S3, Glue, Athena, Lambda...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer

    Data Engineer

    HISH IT SERVICESThrissur, IN
    Location : Remote(Banglore,Chennai,Pune).Pay : 14LPA - 18 LPA(Based on Experience).Timings : A couple of hours overlap with EST, as the client is Canada-based (till 12AM IST).Start Date : 20th Octob...Show moreLast updated: 5 days ago
    • Promoted
    Data Engineer

    Data Engineer

    RevXThrissur, IN
    RevX helps app businesses acquire and reengage users via programmatic to retain, monetize, and accelerate revenue.We're all about taking your app businesses to a new growth level.We rely on data sc...Show moreLast updated: 30+ days ago
    • Promoted
    GCP Data Engineer

    GCP Data Engineer

    TEKsystemsPalakkad, IN
    We are actively hiring for Skilled GCP Data Engineers for a global banking client.Years of experience : 4+ years (relevant). Looking for immediate joiners • •.Onboard new data sources - negotiate, agre...Show moreLast updated: 6 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Eucloid Data SolutionsCoimbatore, IN
    Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure on Databricks...Show moreLast updated: 6 days ago
    • Promoted
    Data Engineer - Web Scrapping & Enrichment

    Data Engineer - Web Scrapping & Enrichment

    Gala IntelligenceThrissur, IN
    We're looking for an entrepreneurial, passionate, and driven.Navneet Tech Venture situated in.As we're building our technology platform from scratch, you'll have the unique opportunity to shape our...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer - Web Scraping

    Data Engineer - Web Scraping

    Alternative PathPalakkad, IN
    Alternative Path is seeking skilled software developers to collaborate on client projects with an asset management firm.In this role, you will collaborate with individuals across various company de...Show moreLast updated: 30+ days ago