← Field Notes
EN/ES

The Web Scraper That Fixes Itself

February 26, 2026via github · @D4Vinci
automationself-hostingopen-sourcetools

The problem it solves

Imagine you've set up something that checks competitor prices every morning — or pulls job listings, or monitors what your industry is saying. It works great for a few weeks. Then the website updates its layout and everything breaks. Someone has to go in and fix it manually. That's the silent tax on almost every data automation that touches the live web.

Scrapling attacks that problem at the root. The first time it reads a page, it quietly memorises the fingerprint of each piece of information — not just where something is on the page, but what it looks like and what's around it. When the site changes, instead of crashing, it goes looking. It finds the new location on its own.

It also handles sites that try to block automated tools — one of its modes is specifically designed to get past those "prove you're human" checks that frustrate most scraping setups.

The whole thing runs on your own server, costs nothing to license, and has picked up 16,000 fans on GitHub in a very short time. For a small studio or agency tracking anything on the web — pricing, press, hiring signals — this is the kind of infrastructure that used to require a dedicated developer to babysit.

Words worth knowing

Web scraping — Automatically reading and collecting information from websites, the way you'd copy data from a page but done by a program, at scale.

Selector — The instruction that tells a scraping tool where on a page to find something. Like saying "the price is always in the blue box in the top right." When the design changes, the selector breaks.

Self-hosted — Running software on your own server rather than paying a third party to run it for you. More control, lower long-term cost.

MCP server — A way for AI tools like Claude to connect directly to other software and pull information from it on demand.


If you have anyone on your team who runs any kind of regular data pull from the web, worth showing them this exists: github.com/D4Vinci/Scrapling

Want us to audit your site? Takes 60 seconds →