ZimLib 1.0.0

ZimLib

纯 C# 编写的 ZIM（openZIM）离线归档解析与读取库 —— 对标 kiwix-js 的底层解析能力。 A pure-C# reader/parser for ZIM (openZIM) offline archives — functionally comparable to kiwix-js's low-level parsing.

简介 / Introduction

ZimLib 是一个零原生依赖的 .NET 库，用于读取 ZIM 格式的离线知识库（维基百科、Wikivoyage、开源教科书等由 Kiwix / openZIM 项目生成的文件）。它把 ZIM 文件头、MIME 表、目录项、簇解压、重定向、标题索引等全部用托管代码实现，因此可以用于桌面、服务端、AOT / 单文件发布等几乎所有 .NET 场景。

ZimLib is a zero-native-dependency .NET library for reading ZIM-format offline archives (Wikipedia, Wikivoyage, open textbooks, etc., produced by the Kiwix / openZIM projects). It implements the header, MIME list, directory entries, cluster decompression, redirects and the title index entirely in managed code, so it works in desktop, server, AOT / single-file publish — virtually any .NET scenario.

本库最初抽离自「静读天下 MoonReader」项目的百科模块，并做了解耦与泛化，便于作为独立 NuGet 包共享。

This library was originally extracted from the encyclopedia module of the "静读天下 / MoonReader" project, then decoupled and generalized so it can be shipped as a standalone NuGet package.

功能特性 / Features

• 读取 ZIM 5 / 6 格式，兼容新旧命名空间方案（A / C / I / M / W / X）。 Reads ZIM 5 / 6, compatible with both legacy and new namespace schemes (A / C / I / M / W / X).
• 解析 MIME 列表、URL 指针表、簇指针表、目录项（普通条目 / 重定向条目）。 Parses the MIME list, URL-pointer table, cluster-pointer table, and directory entries (article/blob entries and redirect entries).
• 簇解压：未压缩、xz(LZMA2)、zstd（现代 ZIM 默认）。支持扩展簇（64 位偏移）。 Cluster decompression: uncompressed, xz (LZMA2), zstd (modern ZIM default). Extended clusters (64-bit offsets) supported.
• 按 (命名空间, URL) 二分查找，自动解析重定向链。 Binary search by (namespace, URL), with automatic redirect-chain resolution.
• 标题前缀搜索（基于标题顺序索引，等价于 kiwix-js 的 searchDirEntriesFromPrefix）。 Title-prefix search (over the title-order index, equivalent to kiwix-js's searchDirEntriesFromPrefix).
• 随机条目、读取 M 命名空间元数据（书名 / 语言 / 日期 / 作者…）。 Random article, and reading 'M'-namespace metadata (title / language / date / creator …).
• 线程安全：底层使用 SafeFileHandle + RandomAccess（无共享文件指针），可被 ASP.NET Core / Kestrel 并发调用。 Thread-safe: built on SafeFileHandle + RandomAccess (no shared file position), safe to call concurrently from e.g. Kestrel.
• LRU 簇缓存（默认 96 MB），避免反复解压同一簇。 LRU cluster cache (96 MB by default) avoids re-decompressing the same cluster.

安装 / Installation

dotnet add package ZimLib

或从源码引用：把 ZimLib 项目加入你的解决方案即可。 Or reference from source: add the ZimLib project to your solution.

快速上手 / Quick Start

using ZimLib;
using System.Text;

// 打开 ZIM 文件（失败会抛 InvalidDataException）/ Open a ZIM (throws InvalidDataException on failure)
using var zim = ZimFile.Open(@"D:\wikipedia_zh_all_nopic_2026-07.zim");

// 读取元数据 / Read metadata
Console.WriteLine($"书名 Title     : {zim.GetMetadata("Title")}");
Console.WriteLine($"语言 Language  : {zim.GetMetadata("Language")}");
Console.WriteLine($"条目数 Entries : {zim.EntryCount:N0}");
Console.WriteLine($"版本 Version   : {zim.MajorVersion}.{zim.MinorVersion}");

// 读取主页内容 / Read the main page
var main = zim.GetMainPage();
byte[]? html = zim.ReadContent(main);
Console.WriteLine($"主页 Main page : {main?.Url}  ({html?.Length:N0} 字节 bytes)");

// 按标题前缀搜索 / Search by title prefix
Console.WriteLine("搜索 '北京' 的前 10 条 / First 10 hits for '北京':");
foreach (var e in zim.SearchByTitlePrefix("北京", 10))
    Console.WriteLine($"  {e.Title}  ->  /zim/{Uri.EscapeDataString(e.Url)}");

// 读取指定条目（兼容 "命名空间/路径" 与裸路径）/ Resolve a specific entry
var beijing = zim.FindByPath("北京市");
byte[]? content = zim.ReadContent(beijing);
Console.WriteLine($"北京市 MIME : {zim.GetMime(beijing!.MimeIndex)}");

// 随机条目 / Random article
Console.WriteLine("随机 Random : " + zim.GetRandomArticle()?.Url);

API 参考 / API Reference

ZimFile

ZimEntry

ZimService

全局会话服务，持有“当前打开的 ZIM”并记录最近文件。 A global session holder for the currently open ZIM, plus a recent-files list.

支持的 ZIM 格式 / Supported formats

在 ASP.NET Core 中提供网页浏览 / Serving a web viewer with ASP.NET Core

下面的中间件把 ZIM 内容以普通 HTTP 资源形式暴露，浏览器 / WebView 即可像访问普通网站一样渲染（对标 kiwix-js 用 Service Worker 拦截条目的做法）。复制 ZimWebRoutes.cs 到你的项目并在 Program.cs 中调用 app.MapZim(...) 即可。

The middleware below exposes ZIM content as ordinary HTTP resources so a browser / WebView can render it like a normal website (equivalent to kiwix-js's Service Worker interception). Copy ZimWebRoutes.cs into your project and call app.MapZim(...) in Program.cs.

ZimWebRoutes.cs

using System;
using System.IO;
using System.Linq;
using System.Text;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Routing;
using ZimLib;

/// <summary>
/// 将 ZIM 内容以 HTTP 资源形式暴露，便于浏览器 / WebView 渲染（对标 kiwix-js 的 Service Worker 拦截）。
/// Exposes ZIM content as HTTP resources so a browser / WebView can render it (equivalent to kiwix-js's Service Worker interception).
///
/// ⚠️ /zimapi/open 允许客户端指定任意本地路径，仅适用于本机 / 受信任环境（如桌面应用内嵌的 localhost 服务）。
/// ⚠️ /zimapi/open lets the client name any local file — only use on localhost / trusted environments (e.g. an embedded server inside a desktop app).
/// </summary>
public static class ZimWebRoutes
{
    /// <summary>
    /// 注册全部百科路由。frontendRoot 为静态前端目录（可选）；为 null 时不提供 /wiki 页面。
    /// Maps all encyclopedia routes. frontendRoot is the static frontend folder (optional); /wiki is not served when null.
    /// </summary>
    public static void MapZim(this IEndpointRouteBuilder app, string? frontendRoot = null)
    {
        // ---------- 前端静态页（可选）/ optional frontend ----------
        if (!string.IsNullOrEmpty(frontendRoot))
        {
            app.MapGet("/wiki/{**path}", (HttpContext ctx, string? path) =>
            {
                path = string.IsNullOrEmpty(path) ? "index.html" : path.Replace("..", "");
                var full = Path.Combine(frontendRoot!, path);
                if (!File.Exists(full)) return Results.NotFound();
                ctx.Response.Headers.CacheControl = "no-cache";
                var mime = full.EndsWith(".js") ? "text/javascript"
                         : full.EndsWith(".css") ? "text/css"
                         : full.EndsWith(".svg") ? "image/svg+xml"
                         : "text/html; charset=utf-8";
                return Results.File(full, mime);
            });
        }

        // ---------- API ----------
        app.MapGet("/zimapi/info", () =>
        {
            var zim = ZimService.Instance.Current;
            if (zim == null)
                return Results.Json(new { open = false, recent = ZimService.Instance.GetRecent() });

            var main = zim.GetMainPage();
            return Results.Json(new
            {
                open = true,
                file = zim.FilePath,
                fileName = Path.GetFileName(zim.FilePath),
                name = FirstNonEmpty(zim.GetMetadata("Title"), Path.GetFileNameWithoutExtension(zim.FilePath)),
                description = zim.GetMetadata("Description"),
                language = zim.GetMetadata("Language"),
                date = zim.GetMetadata("Date"),
                creator = zim.GetMetadata("Creator"),
                entryCount = zim.EntryCount,
                clusterCount = zim.ClusterCount,
                version = $"{zim.MajorVersion}.{zim.MinorVersion}",
                mainPath = main?.Url ?? "",
                recent = ZimService.Instance.GetRecent()
            });
        });

        app.MapGet("/zimapi/open", (string? path) =>
        {
            if (string.IsNullOrWhiteSpace(path) || !File.Exists(path))
                return Results.Json(new { ok = false, error = "文件不存在 / File not found" });
            try
            {
                var zim = ZimService.Instance.Open(path);
                var main = zim.GetMainPage();
                return Results.Json(new { ok = true, mainPath = main?.Url ?? "" });
            }
            catch (Exception ex) { return Results.Json(new { ok = false, error = ex.Message }); }
        });

        app.MapGet("/zimapi/close", () => { ZimService.Instance.Close(); return Results.Json(new { ok = true }); });

        app.MapGet("/zimapi/search", (string? q, int? limit) =>
        {
            var zim = ZimService.Instance.Current;
            if (zim == null || string.IsNullOrWhiteSpace(q)) return Results.Json(Array.Empty<object>());
            var n = Math.Clamp(limit ?? 25, 1, 100);
            var hits = zim.SearchByTitlePrefix(q.Trim(), n);
            return Results.Json(hits.Select(e => new
            {
                title = string.IsNullOrEmpty(e.Title) ? e.Url : e.Title,
                path = e.Url,
                redirect = e.IsRedirect
            }).ToList());
        });

        app.MapGet("/zimapi/random", () =>
        {
            var zim = ZimService.Instance.Current;
            var e = zim?.GetRandomArticle();
            return Results.Json(new { path = e?.Url ?? "", title = e == null ? "" : (string.IsNullOrEmpty(e.Title) ? e.Url : e.Title) });
        });

        // ---------- ZIM 条目内容 / ZIM entry content ----------
        app.MapMethods("/zim/{**path}", new[] { "GET", "HEAD" }, (HttpContext ctx, string? path) =>
        {
            var zim = ZimService.Instance.Current;
            if (zim == null)
                return Results.Text("尚未打开 ZIM 文件 / No ZIM opened", "text/plain; charset=utf-8", statusCode: 503);
            if (string.IsNullOrEmpty(path))
            {
                var mp = zim.GetMainPage();
                if (mp == null) return Results.NotFound();
                return Results.Redirect("/zim/" + Uri.EscapeDataString(mp.Url), false);
            }

            var key = path.Split('#')[0];
            // 兼容链接中空格与下划线互换的写法 / tolerate space/underscore swap in links
            var entry = zim.FindByPath(key)
                     ?? zim.FindByPath(key.Replace(' ', '_'))
                     ?? zim.FindByPath(key.Replace('_', ' '));
            if (entry == null) return NotFoundHtml(key);

            // 重定向条目返回 302，使浏览器地址与真实条目一致（同 kiwix-js）
            // Redirect entries return 302 so the browser URL matches the real entry (kiwix-js behavior)
            if (entry.IsRedirect)
            {
                var target = zim.GetEntry(entry.RedirectIndex);
                if (target == null) return NotFoundHtml(key);
                return Results.Redirect("/zim/" + Uri.EscapeDataString(target.Url), false);
            }

            var data = zim.ReadContent(entry);
            if (data == null) return NotFoundHtml(key);

            var mime = zim.GetMime(entry.MimeIndex);
            if (mime.StartsWith("text/", StringComparison.OrdinalIgnoreCase) && !mime.Contains("charset"))
                mime += "; charset=utf-8";

            ctx.Response.Headers.CacheControl = "public, max-age=86400";
            ctx.Response.Headers.ContentLength = data.Length;
            if (HttpMethods.IsHead(ctx.Request.Method)) { ctx.Response.ContentType = mime; return Results.Empty; }
            return Results.Bytes(data, mime);
        });
    }

    private static string FirstNonEmpty(params string[] values)
        => values.FirstOrDefault(v => !string.IsNullOrWhiteSpace(v)) ?? "";

    private static IResult NotFoundHtml(string key)
    {
        var html = "<!DOCTYPE html><meta charset=\"utf-8\">"
                 + "<style>body{font:15px/1.7 system-ui,'Microsoft YaHei';padding:48px;color:#444}"
                 + "code{background:#f2f2f5;padding:2px 6px;border-radius:4px}</style>"
                 + "<h2>条目不存在 / Not found</h2><p>ZIM 中找不到 / Not in ZIM: <code>"
                 + System.Net.WebUtility.HtmlEncode(key) + "</code></p>";
        return Results.Text(html, "text/html; charset=utf-8", Encoding.UTF8, 404);
    }
}

Program.cs

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

// 传入前端静态目录（存放下面的 index.html 等），不传则只暴露 API 与条目内容。
// Pass the frontend folder (holding the index.html below); omit it to expose only the API + entry content.
app.MapZim(Path.Combine(app.Environment.ContentRootPath, "wwwroot", "wiki"));

app.Run();

路由契约 / Route contract:

• GET /wiki/ —— 前端页面（若提供 frontendRoot）/ frontend page
• GET /zim/{**path} —— 条目内容（重定向返回 302）/ entry content (302 on redirect)
• GET /zimapi/info | open?path= | close | search?q=&limit= | random —— JSON API

最小前端页面 / Minimal frontend page

把以下内容保存为 wwwroot/wiki/index.html，它即可通过上面的中间件工作（搜索建议、iframe 渲染、前进/后退/首页/随机）。

Save the following as wwwroot/wiki/index.html; it works with the middleware above (search suggestions, iframe rendering, back/forward/home/random).

<!DOCTYPE html>
<html lang="zh">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>ZimLib Viewer</title>
<style>
  * { box-sizing: border-box; }
  html, body { height: 100%; margin: 0; font: 14px/1.5 system-ui, "Microsoft YaHei", sans-serif; }
  body { display: flex; flex-direction: column; }
  #bar { display: flex; gap: 6px; align-items: center; padding: 8px 10px; background: #20232a; color: #eee; }
  #bar input { flex: 1; padding: 6px 10px; border: 0; border-radius: 6px; background: #2c3038; color: #fff; }
  #bar button { padding: 6px 10px; border: 0; border-radius: 6px; background: #3a3f4b; color: #fff; cursor: pointer; }
  #bar button:hover { background: #4a5160; }
  #info { margin-left: 8px; color: #9aa; font-size: 12px; }
  #suggest { position: absolute; top: 46px; left: 10px; right: 10px; max-height: 50%; overflow: auto;
             background: #fff; color: #222; border-radius: 8px; box-shadow: 0 8px 24px rgba(0,0,0,.3); z-index: 9; }
  #suggest .item { padding: 8px 12px; cursor: pointer; border-bottom: 1px solid #f0f0f0; }
  #suggest .item:hover { background: #f4f7ff; }
  iframe { flex: 1; border: 0; width: 100%; }
  #empty { position: absolute; inset: 46px 0 0 0; display: flex; flex-direction: column; gap: 12px;
           align-items: center; justify-content: center; background: #f7f8fa; color: #555; text-align: center; padding: 24px; }
  #empty input[type=file] { font-size: 14px; }
  .hidden { display: none !important; }
</style>
</head>
<body>
  <div id="bar">
    <input id="search" placeholder="搜索条目 / Search… (Ctrl+K)">
    <button id="home">🏠 首页</button>
    <button id="rand">🎲 随机</button>
    <button id="back">← 后退</button>
    <button id="fwd">前进 →</button>
    <button id="open">📂 打开</button>
    <span id="info"></span>
  </div>
  <div id="suggest" class="hidden"></div>
  <iframe id="frame" src="about:blank"></iframe>
  <div id="empty" class="hidden">
    <h2>尚未打开百科文件 / No ZIM opened</h2>
    <p>点击右上角「📂 打开」，在桌面端由宿主应用选择本地 .zim 文件。/ Click “📂 打开” (desktop host picks a local .zim).</p>
    <div id="recent"></div>
  </div>
  <script>
    const $ = s => document.querySelector(s);
    const frame = $('#frame'), empty = $('#empty'), suggest = $('#suggest');

    async function refresh() {
      const d = await (await fetch('/zimapi/info')).json();
      if (!d.open) { empty.classList.remove('hidden'); frame.classList.add('hidden'); renderRecent(d.recent || []); $('#info').textContent = ''; return; }
      empty.classList.add('hidden'); frame.classList.remove('hidden');
      $('#info').textContent = d.name + ' · ' + d.entryCount.toLocaleString() + ' 条 / entries';
      if (!frame.dataset.loaded) { frame.src = '/zim/' + encodeURIComponent(d.mainPath); frame.dataset.loaded = '1'; }
    }
    function renderRecent(list) {
      const el = $('#recent'); el.innerHTML = '';
      list.forEach(p => { const b = document.createElement('button'); b.textContent = p; b.onclick = () => openZim(p); el.appendChild(b); });
    }
    async function openZim(path) {
      const d = await (await fetch('/zimapi/open?path=' + encodeURIComponent(path))).json();
      if (d.ok) { empty.classList.add('hidden'); frame.classList.remove('hidden'); frame.dataset.loaded = ''; frame.src = '/zim/' + encodeURIComponent(d.mainPath); }
      else alert(d.error);
    }
    let timer;
    $('#search').oninput = e => {
      clearTimeout(timer); const q = e.target.value.trim();
      if (!q) { suggest.classList.add('hidden'); return; }
      timer = setTimeout(async () => {
        const list = await (await fetch('/zimapi/search?q=' + encodeURIComponent(q) + '&limit=15')).json();
        suggest.innerHTML = '';
        if (!list.length) { suggest.classList.add('hidden'); return; }
        list.forEach(it => {
          const div = document.createElement('div'); div.className = 'item';
          div.textContent = it.title || it.path;
          div.onclick = () => { frame.src = '/zim/' + encodeURIComponent(it.path); suggest.classList.add('hidden'); $('#search').value = it.title || it.path; };
          suggest.appendChild(div);
        });
        suggest.classList.remove('hidden');
      }, 120);
    };
    $('#home').onclick = () => refresh();
    $('#rand').onclick = async () => { const d = await (await fetch('/zimapi/random')).json(); if (d.path) { frame.dataset.loaded = ''; frame.src = '/zim/' + encodeURIComponent(d.path); } };
    $('#back').onclick = () => frame.contentWindow.history.back();
    $('#fwd').onclick = () => frame.contentWindow.history.forward();
    // 桌面端：由宿主（如 Avalonia WebView）注入 window.zimPickAndOpen() 来真正打开文件。
    // Desktop: the host (e.g. Avalonia WebView) injects window.zimPickAndOpen() to actually open a file.
    $('#open').onclick = () => { if (window.zimPickAndOpen) window.zimPickAndOpen(); else empty.classList.remove('hidden'); };
    document.addEventListener('keydown', e => { if ((e.ctrlKey || e.metaKey) && e.key === 'k') { e.preventDefault(); $('#search').focus(); } });
    // 外链交给系统浏览器 / external links -> system browser
    frame.addEventListener('load', () => { try { frame.contentWindow.addEventListener('click', ev => { const a = ev.target.closest('a'); if (a && a.href && /^https?:/i.test(a.href)) { ev.preventDefault(); window.open(a.href, '_blank'); } }); } catch (_) {} });
    refresh();
  </script>
</body>
</html>

注意：浏览器中的 <input type=file> 出于安全限制拿不到本地绝对路径。因此「打开文件」在纯网页场景应由**宿主应用（C# 侧）**完成——宿主用系统的文件选择器拿到真实路径后调用 ZimService.Instance.Open(path)，再让前端 refresh()。前端「📂 打开」按钮即调用宿主注入的 window.zimPickAndOpen()。 Note: a browser <input type=file> cannot read the local absolute path (security). So "open file" in a pure-web scenario should be done by the host app (C# side) — the host uses the OS file picker, calls ZimService.Instance.Open(path), then asks the frontend to refresh(). The frontend "📂 打开" button calls the host-injected window.zimPickAndOpen().

在 Avalonia WebView 中渲染 / Rendering inside an Avalonia WebView

典型的桌面集成方式（与「静读天下 MoonReader」一致）：

The typical desktop integration (same as 静读天下 / MoonReader):

1. 内嵌一个 Kestrel 本地服务，调用 app.MapZim(frontendRoot) 暴露 /wiki、/zim、/zimapi/*。 Embed a local Kestrel server and call app.MapZim(frontendRoot) to expose /wiki, /zim, /zimapi/*.
2. 顶栏放一个「🌏 百科」按钮，打开一个窗口承载 NativeWebView，导航到 http://localhost:<port>/wiki/。 Put a "🌏 百科" button on the toolbar that opens a window hosting NativeWebView, navigated to http://localhost:<port>/wiki/.
3. 文件选择器由 C# 侧处理：拿到路径后 ZimService.Instance.Open(path)，并通过 WebView 注入的桥调用 window.zimRefresh()（或直接 web.Refresh()）让前端刷新。 The C# side handles the file picker: after getting the path, call ZimService.Instance.Open(path), then invoke the injected window.zimRefresh() (or simply web.Refresh()) to refresh the frontend.

// 宿主侧：打开 ZIM 并通知前端 / Host side: open ZIM and notify the frontend
private void OpenZim(string path)
{
    ZimService.Instance.Open(path);   // 解析并设为当前 / parse and set as current
    // 让 WebView 里的页面重新拉取 /zimapi/info / ask the page to re-fetch /zimapi/info
    WebView.InvokeScript("window.zimRefresh && window.zimRefresh()");
    // 若脚本桥尚未就绪，可回退为整页刷新 / fall back to a full reload if the bridge isn't ready
    // WebView.Refresh();
}

前端 window.zimRefresh 即上面 index.html 中的 refresh() 函数（重命名注入即可）。 The frontend's window.zimRefresh is simply the refresh() function from the index.html above (rename it when injecting).

性能与线程安全 / Performance & thread safety

• 簇 LRU 缓存：默认上限 96 MB，解压后的簇会被缓存并在并发访问间共享；超容量时按 LRU 淘汰。 LRU cluster cache: 96 MB by default; decompressed clusters are cached and shared across concurrent readers; LRU-evicted when over capacity.
• 无共享文件指针：使用 SafeFileHandle + RandomAccess，多个线程/请求可同时读取同一文件而不互相干扰。 No shared file position: SafeFileHandle + RandomAccess lets multiple threads/requests read the same file concurrently without interference.
• 标题索引懒加载：首次调用 SearchByTitlePrefix 时才读取标题顺序索引（大型 ZIM 可能占用十几 MB 内存，加载约 1 秒），之后搜索为 0 毫秒级。 Lazy title index: the title-order index is read on the first SearchByTitlePrefix call (a large ZIM may use ~10+ MB and take ~1s to load); subsequent searches are sub-millisecond.
• 打开一个 40 GB 级 ZIM 通常在数毫秒内完成（仅解析头部），读取单篇文章通常 < 10 ms。 Opening a 40 GB-class ZIM typically takes a few milliseconds (header only); reading a single article is usually < 10 ms.

已知限制 / Known limitations

• 无 Xapian 全文检索：本库仅实现标题前缀搜索（与 kiwix-js 在无 Xapian 索引时的降级行为一致）。如需全文检索，需集成原生 libxapian 或自行建立外部索引。 No Xapian full-text search: only title-prefix search is implemented (matching kiwix-js's fallback when no Xapian index is present). Full-text search would require native libxapian or a custom external index.
• 不支持已废弃的 zlib(2) / bzip2(3) 压缩：现代 ZIM 默认 zstd，老文件多为 xz，二者均支持；极老文件若存在这两种压缩会抛 NotSupportedException。 Deprecated zlib(2) / bzip2(3) compression is unsupported: modern ZIM defaults to zstd and older files use xz — both supported; extremely old files using zlib/bzip2 will throw NotSupportedException.
• 仅在读模式下工作（ZIM 本身是离线分发格式，不提供写入）。 Read-only (ZIM is a distribution format; no writing).

许可 / License

MIT —— 可自由用于商业与非商业项目，请保留版权声明。 MIT — free for commercial and non-commercial use; please retain the copyright notice.