Talent.com
This job offer is not available in your country.
Data Engineer - Web Scrapping & Enrichment

Data Engineer - Web Scrapping & Enrichment

Gala IntelligenceHyderabad, IN
30+ days ago
Job description

We're looking for an entrepreneurial, passionate, and driven Data Engineer to join Startup Gala Intelligence backed by Navneet Tech Venture situated in Ahmedabad . As we're building our technology platform from scratch, you'll have the unique opportunity to shape our technology vision, architecture, and engineering culture right from the ground up. You’ll directly contribute to foundational development and establish best practices, while eventually building and contributing to our engineering team.

This role is ideal for someone eager to own the entire tech stack, who thrives on early-stage challenges, and loves building innovative, scalable solutions from day zero.

What You’ll Do

  • Web Scraping & Crawling : Build and maintain automated scrapers to extract structured and unstructured data from websites, APIs, and public datasets.
  • Scalable Scraping Systems : Develop multi-threaded, distributed crawlers capable of handling high-volume data collection without interruptions.
  • Data Parsing & Cleaning : Normalize scraped data, remove noise, and ensure consistency before passing to data pipelines.
  • Anti-bot & Evasion Tactics : Implement proxy rotation, captcha solving, and request throttling techniques to handle scraping restrictions.
  • Integration with Pipelines : Deliver clean, structured datasets into NoSQL stores and ETL pipelines for further enrichment and graph-based storage.
  • Data Quality & Validation : Ensure data accuracy, deduplicate records, and maintain a trust scoring system for data confidence.
  • Documentation & Maintenance : Keep scrapers updated when websites change, and document scraping logic for reproducibility.

Who You Are

Technical Skills :

  • 2+ years of experience in web scraping , crawling, or data collection.
  • Strong proficiency in Python (libraries like BeautifulSoup, Scrapy, Selenium, Playwright, Requests).
  • Familiarity with NoSQL databases (MongoDB, DynamoDB) and data serialization formats (JSON, CSV, Parquet).
  • Experience in handling large-scale scraping with proxy management and rate-limiting.
  • Basic knowledge of ETL processes and integration with data pipelines.
  • Exposure to graph databases (Neo4j) is a plus.
  • Soft Skills :

  • Detail-oriented, ensuring accuracy and reliability of collected data.
  • Strong problem-solving skills, particularly in adapting scrapers to evolving web structures.
  • Curious mindset with a drive to discover new data sources.
  • Comfortable working in a fast-paced, early-stage startup environment.
  • Who We Are & Our Culture

    Gala Intelligence , backed by Navneet Tech Ventures , is a tech-driven startup dedicated to solving one of the most pressing business challenges - fraud detection and prevention. We're building cutting-edge, real-time products designed to empower consumers and businesses to stay ahead of fraudsters, leveraging innovative technology and deep domain expertise.

    Our culture and values :

    We’re united by a single, critical mission - stopping fraud before it impacts businesses. Curiosity, innovation, and proactive action define our approach. We value transparency, collaboration, and individual ownership, creating an environment where talented people can do their best work.

  • Problem-Driven Innovation : We're deeply committed to solving real challenges that genuinely matter for our customers.
  • Rapid Action & Ownership : We encourage autonomy and accountability—own your projects, move quickly, and shape the future of Gala Intelligence.
  • Collaborative Excellence : Cross-team collaboration ensures alignment, sparks innovation, and drives us forward together.
  • Continuous Learning : Fraud evolves rapidly, and so do we. Continuous improvement, experimentation, and learning are core to our success.
  • If you're excited by the opportunity to leverage technology in the fight against fraud, and you're ready to build something impactful from day one, we want to hear from you!

    Create a job alert for this search

    Data Engineer • Hyderabad, IN

    Related jobs
    • Promoted
    • New!
    GCP Data Engineer

    GCP Data Engineer

    Multi Recruithyderabad, India
    Excellent Coding skills in ETLTools.Experience on running ETLpipelines from a wide variety of sources, both batch & streaming, usinglatest data frameworks and technologies with real time monitoring...Show moreLast updated: 12 hours ago
    • Promoted
    Data Engineer - Web Scraping

    Data Engineer - Web Scraping

    Alternative PathHyderabad, IN
    Alternative Path is seeking skilled software developers to collaborate on client projects with an asset management firm.In this role, you will collaborate with individuals across various company de...Show moreLast updated: 30+ days ago
    • Promoted
    Backend Developer / Data Scraping / Kubernetes

    Backend Developer / Data Scraping / Kubernetes

    Daply | Scaling Digital PublishingHyderabad, IN
    IN ORDER TO BE CONSIDERED FOR THIS ROLE, YOU MUST SUBMIT AN APPLICATION HERE : .Web Scraping / Captcha Control.Show moreLast updated: 8 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Innodata Inc.Hyderabad, IN
    CI / CD practices, Databricks (Spark), Python, Github and SQL.The ideal candidate should have hands-on expertise in building and automating data pipelines, managing multi-environment deployments, and...Show moreLast updated: 26 days ago
    • Promoted
    • New!
    Data Engineer AWS

    Data Engineer AWS

    Anicalls (Pty) Ltdhyderabad, India
    Strong computer science fundamentals such as algorithms, data structures, multithreading, object-oriented development, distributed applications, client-server architecture.Design and implement Mach...Show moreLast updated: 12 hours ago
    • Promoted
    • New!
    Data Engineer SQL / C# / AWS

    Data Engineer SQL / C# / AWS

    Anicalls (Pty) Ltdhyderabad, India
    Advanced SQL development knowledge using MS SQL Server.Experience in Data Modeling and building data warehouse / data lake solutions that involve multiple disparate sources.Experience working in an A...Show moreLast updated: 12 hours ago
    • Promoted
    • New!
    AWS Data Engineer

    AWS Data Engineer

    HGShyderabad, India
    We are looking for an AWS Data Engineer toadd to our cloud practice.Webelieve that Data Engineer enables data-driven decision making bycollecting, transforming, and publishing data.A Data Engineer ...Show moreLast updated: 12 hours ago
    • Promoted
    • New!
    AWS Data Engineer RedShift-C2H-TCS

    AWS Data Engineer RedShift-C2H-TCS

    Axiom Software Solutions Limitedhyderabad, India
    Job Title : AWS Data Engineer - RedShift.Company : Axiom Software Solutions Limited.Location : On-site (as required).Axiom Software Solutions Limited is seeking an AWS Data Engineer with a strong focu...Show moreLast updated: 12 hours ago
    • Promoted
    Data Engineer

    Data Engineer

    Canopus Infosystems - A CMMI Level 3 Companysecunderabad, telangana, in
    Python expertise and hands-on experience in handling large datasets, data cleaning, analysis, and visualization.The ideal candidate should be capable of building data pipelines, performing web scra...Show moreLast updated: 18 days ago
    • Promoted
    Data Engineer

    Data Engineer

    ACL Digitalsecunderabad, telangana, in
    Design, develop, and optimize Spark-based data pipelines on Databricks for large-scale data processing.Design, develop, and optimize AWS pipeline as applicable. Implement and manage GitHub asset bun...Show moreLast updated: 30+ days ago
    • Promoted
    AWS Data Engineer

    AWS Data Engineer

    CoforgeHyderabad, Telangana, India
    As an AWS Data Engineer, you will be responsible for designing, developing, and optimizing robust, scalable, and secure data pipelines and infrastructure on AWS. You’ll work with diverse datasets an...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Kline + CompanyHyderabad, IN
    Here at Kline our data capabilities have grown exponentially over the last four years.Having gone through a rapid digitization process and becoming a cloud-native corporation, we are looking for to...Show moreLast updated: 1 day ago
    • Promoted
    Data Engineer

    Data Engineer

    Avenue CodeHyderabad, Telangana, India
    Avenue Code is the leading software consultancy focused on delivering end-to-end development solutions for digital transformation across every vertical. We’re privately held, profitable, and have be...Show moreLast updated: 7 days ago
    • Promoted
    • New!
    Data Engineer

    Data Engineer

    Virtusahyderabad, India
    Data Engineer - CREQ Description Data Engineer 3 to 6 Years.The main goal of the Data Engineer would be to develop, optimize and. OLAP / DWH / Bigdata environments : .Good hands on building data pipeline...Show moreLast updated: 12 hours ago
    • Promoted
    • New!
    AWS Data Engineer

    AWS Data Engineer

    Astrosoft technologieshyderabad, India
    Strong experience and understanding of streaming architecture and development practices using.Strong knowledge of one or more. Very familiar with SRE concepts which includes evaluating and implement...Show moreLast updated: 12 hours ago
    • Promoted
    • New!
    Tech Lead - AWS Data Engineer Job

    Tech Lead - AWS Data Engineer Job

    YASH Technologieshyderabad, India
    Understanding the current business functionalities of the ETL processes and identifying areas for improvement.Designing and building AWS Glue workflows to replace existing ETL processes.Standardizi...Show moreLast updated: 12 hours ago
    • Promoted
    • New!
    Data Engineer- GCP

    Data Engineer- GCP

    NucleusTeqhyderabad, India
    Skills needed : Python, Airflow(Orchestration) ,GCP(Cloud),Spark SQL, PySpark, GCP, CI / CD ,Git,Git Hub.Designing and building data models to support business requirements. Developing and maintaining ...Show moreLast updated: 12 hours ago
    • Promoted
    • New!
    Data Engineer II

    Data Engineer II

    FedExhyderabad, India
    We are seeking a skilled IBM Sterling Developer to design, develop, implement, test, and support robust business-to-business (B2B) integration solutions using the IBM Sterling suite, primarily focu...Show moreLast updated: 12 hours ago