webcrawl-mcp

Local-first web scraping, DuckDuckGo search, site mapping and crawling, using trafilatura with an optional Firecrawl fallback for JS-rendered pages.

#1434 by GitHub starsDocumentationChecked 2026-11-02

What it connects

Local-first web scraping, DuckDuckGo search, site mapping and crawling, using trafilatura with an optional Firecrawl fallback for JS-rendered pages.

Registry listing — description sourced from the publisher's entry in an awesome-list. Setup, compatibility and requirements are not verified; see the repository.

Source: https://github.com/andyliszewski/webcrawl-mcp · ★ 16 on the source repo

Setup instructions and compatibility are compiled from publisher docs and community reports, not hands-on testing, unless a step is marked Tested. Check the source before relying on a setup.