Web Scraping Specialist
Take ownership of the research, development, and ongoing maintenance that powers ScrapeAtlas endpoints.
You’ll transform constantly changing web sources into reliable, structured API responses while clearly communicating limitations whenever a source provides incomplete or restricted data.
What You’ll Work On
- Build and maintain TypeScript-based extractors for public profiles, posts, and other publicly accessible platform information.
- Research changes to page structures, embedded JSON, and network requests to identify extraction issues, access restrictions, rate limits, and blocking behavior.
- Create reliable regression fixtures and controlled transport logic to catch source changes before they cause larger production issues.
- Maintain accurate endpoint documentation so API behavior, available data, and known limitations are clearly represented.
- Investigate production failures alongside the team, identify their underlying causes, and validate fixes against live sources.
- Improve the reliability and consistency of extraction systems as upstream platforms continue to evolve.
Requirements
- Practical experience developing and maintaining production web scrapers that must adapt to frequent changes in upstream websites or platforms.
- Strong JavaScript or TypeScript skills, with a solid understanding of HTTP requests, browser behavior, and debugging techniques.
- Experience working with browser automation and tools used to investigate page structures and network activity.
- Strong troubleshooting skills and the ability to diagnose extraction failures, rate limiting, blocking, and unexpected source behavior.
- Careful judgment when evaluating data accuracy, access limitations, system reliability, and the operational cost of collecting information.
Experience working with publicly accessible social platforms, proxy infrastructure, API monitoring, or observability systems would be an added advantage.
Email Subject: Web Scraping Specialist application