Description :
Job Opportunity : Data Engineer Acquisition (Web Crawling & Data Extraction)
We are seeking a highly skilled and passionate Data Engineer Acquisition to join our growing Data Operations Team!
If you enjoy building intelligent data pipelines, solving complex crawling challenges, and working with large-scale datasets, this role is perfect for you.
Role Overview :
As a Data Engineer focused on web crawling & platform data acquisition, you will :
- Design, develop & maintain large-scale web scraping pipelines
- Build scalable & resilient data extraction systems
- Work with anti-bot techniques, proxy management & dynamic parsing
- Ensure high-quality, high-integrity acquired data across regions
- Continuously optimize scraping workflows for performance & reliability
Key Responsibilities :
Build & maintain crawlers and parsers using cutting-edge toolsAutomate scraping workflows for accurate platform data collectionImplement proxy handling, CAPTCHA bypass & headless browser automationLeverage Python, HTML, JavaScript for data extractionOptimize pipelines for speed, resilience & efficiencyDebug website structural changes & fix blockersEnsure the data remains accurate, clean & consistentCollaborate with cross-functional teams to improve data qualityWork with JSON, XML & SQL-based data storesSkills & Experience Needed :
Minimum 3+ years of programming experienceStrong Python expertiseExperience in HTML, JavaScript, DOMHands-on experience with web scraping tools / frameworks :ScrapySeleniumPlaywrightBeautifulSoupKnowledge of distributed crawling & job schedulingFamiliarity with headless browsers & proxy rotation techniquesExperience handling CAPTCHA challengesSQL & cloud storage experienceGood to have : Spark, KafkaGood To Have :
Experience in asynchronous scrapingCloud platforms (AWS / GCP / Azure)CI / CD & automation pipeline exposurePython design patternsSoft Skills :
Strong problem-solving & analytical abilityExcellent communication & documentationTeam player & proactive mindsetAbility to adapt to changing web structuresWhy Join Us?
Opportunity to work on sophisticated data acquisition projectsExposure to global-scale platforms & distributed systemsCareer growth in fast-evolving data engineering spaceCollaborative, learning-oriented environment(ref : hirist.tech)