GoTeam
Senior Web Scraping Engineer (Up to ₱250K | Remote | FREE AI Certification) Remote | Night Shift | 12:00 AM–9:00 AM | Tuesday - Saturday You own the end-to-end reliability of a large-scale web data acquisition pipeline processing tens of millions of scrapes per month. You are responsible for maintaining system uptime, reducing failures, and building resilient infrastructure that bypasses sophisticated anti-bot protections and authentication walls. We are looking for the "missing piece" in our client's engineering team—a specialist who genuinely enjoys the technical challenge of extracting large-scale data and navigating sophisticated bot protections. You will be the go-to expert who ensures our operations run smoothly, allowing the rest of the team to focus on core product features. You don’t just wait for things to break; you proactively suggest and implement improvements to stay ahead of an ever-changing web landscape. You’re Likely a Strong Fit If You Have 5+ years of software development experience with significant exposure to web scraping or data acquisition at scale. Deep Scraping Expertise: Proficiency with headless browsers, CDP (Chrome DevTools Protocol), proxy management, and CAPTCHA mitigation. Anti-Bot Evasion: Proven experience handling authenticated sessions and navigating complex anti-bot strategies. Observability Skills: Experience building and maintaining monitoring, alerting, and observability systems for production infrastructure. Technical Agility: Ability to pick up new languages quickly; while the core codebase is Ruby, we value the ability to work across diverse tools and libraries. Structural Constraints: Proven reliability working a fixed night shift (10:00 PM – 7:00 AM) in a 100% remote capacity. Strong Signals A "Scraping Enthusiast": Someone who truly wants to do this type of work on a regular basis and stays on top of best practices. Proactive Ownership: A mindset that looks at operations and suggests better ways to build, rather than just waiting for tasks. Resilience: The ability to handle "firefighting" days with urgency while using "quiet" days to experiment with new technologies and tackle the backlog. Responsibility Pillars Infrastructure Reliability & Uptime You own the heartbeat of the data acquisition pipeline—ensuring tens of millions of pages are scraped successfully. Monitor scraping and API success rates to identify and resolve issues before they cascade. Respond to system outages with urgency, diagnosing root causes and deploying fixes. Build better alerting and redundancies to reduce manual intervention over time. Platform Evolution & Research You ensure the scraping infrastructure stays ahead of an ever-changing web landscape. Evaluate and experiment with new scraping technologies and anti-bot evasion libraries. Proactively improve data retrieval resilience, including handling unique authentication wall challenges. Build dashboards and tooling to provide clear visibility into system health and performance. Architectural Collaboration & Ownership You operate as a technical lead for the data acquisition mission. Collaborate with the senior team on major architectural decisions. Exercise autonomy to move fast on day-to-day improvements without constant supervision. Communicate technical failures and status updates clearly to both engineering and non-engineering stakeholders.
GoTeam
