AlexPavlov 

Declarative high performance web scraper in C#. Easily crawl any web site and parse the data, save structed result to a file, DB, etc.

Declarative high performance web scraper in C#. Easily crawl any web site and parse the data, save structed result to a file, DB, etc.

SQLite embedded-store adapters for WebReaper: a local durable scheduler (and, from the next slice, a visited-link tracker) backed by SQLite via Microsoft.Data.Sqlite; resume is a query, not a hand-rolled... More information

Azure Cosmos DB sink for WebReaper. Adds ScraperEngineBuilder.WriteToCosmosDb(...). Satellite package (ADR-0009) so the WebReaper core stays dependency-light and AOT-clean.

Azure Service Bus scheduler for WebReaper: a distributed job queue backed by an Azure Service Bus queue, for sharing crawl state across workers and serverless functions. Adds... More information

MongoDB adapters for WebReaper: the MongoDbSink, plus MongoDB-backed scraper-config and cookie storage. Adds ScraperEngineBuilder.WriteToMongoDb / WithMongoDbConfigStorage / WithMongoDbCookieStorage. Satellite... More information

Redis adapters for WebReaper: the Redis scheduler, visited-link tracker, sink, and Redis-backed scraper-config and cookie storage, all sharing one connection pool (ADR-0005). Adds... More information

Raw Chrome DevTools Protocol (CDP) IPageLoadTransport for WebReaper; the AOT-clean transport that connects to any --remote-debugging-port endpoint. The bedrock for Browser-backend swaps (stealth Chromium forks... More information

Roslyn source generator emitting a static Schema and reflection-free Materialize method on classes marked with [ScrapeSchema]. Pairs with WebReaper.Extraction.Attributes. The .NET-native structural... More information

LLM-backed IContentExtractor for WebReaper, bound to Microsoft.Extensions.AI.Abstractions (IChatClient). Bring your own model: OpenAI, Anthropic, Ollama, anything implementing IChatClient. Adds... More information

CloakBrowser stealth Chromium fork backend for WebReaper. Adds .WithCloakBrowser(); one-liner that finds (or downloads from upstream) the CloakBrowser binary, launches it with the fork's recommended flags, and... More information

The [ScrapeSchema] and [ScrapeField] attributes consumed by the WebReaper.Extraction.Generators Roslyn source generator. Lightweight (no runtime deps beyond BCL). Multi-targeted to netstandard2.0 (so the... More information

Microsoft.Playwright-backed IPageLoadTransport for WebReaper. Modern .NET-native browser SDK with multi-browser support (Chromium, Firefox, WebKit), auto-wait semantics, and full Network interception via... More information

Declarative high performance web scraper in C#. Easily crawl any web site and parse the data, save structed result to a file, DB, etc.

MCP (Model Context Protocol) server satellite for WebReaper. Exposes scrape / map / extract as MCP tools over stdio for Cursor / Claude Desktop / Copilot Studio. Interop adapter; the primary agent surface... More information

An AOT-clean, OpenAI-compatible IChatClient (Microsoft.Extensions.AI) for WebReaper, built on raw HttpClient and System.Text.Json source generation with no provider SDK. Point it at OpenAI, Ollama, OpenRouter,... More information

Headless-browser (Puppeteer/Chromium) page-load transport for WebReaper, for scraping JavaScript-rendered pages (GetWithBrowser / FollowWithBrowser / PaginateWithBrowser). Adds... More information

Streamable HTTP MCP (Model Context Protocol) server host for WebReaper. Exposes scrape / map / extract / extract_with_prompt / crawl as MCP tools over HTTP, so URL-based MCP clients (n8n, hosted agents) can... More information