LMSupply.Reranker 0.115.1

LMSupply.Reranker

Local semantic reranking for .NET with cross-encoder models.

Features

• Zero-config: Models download automatically from HuggingFace
• GPU Acceleration: CUDA, DirectML (Windows), CoreML (macOS)
• Cross-platform: Windows, Linux, macOS
• RAG Integration: Perfect for improving retrieval quality
• Multi-Tokenizer Support: Automatic detection of WordPiece, Unigram, BPE tokenizers

Quick Start

using LMSupply.Reranker;

// Load the default model
await using var reranker = await LocalReranker.LoadAsync("default");

// Rerank documents
var results = await reranker.RerankAsync(
    query: "What is machine learning?",
    documents: ["ML is a branch of AI...", "The weather is nice..."],
    topK: 5);

foreach (var result in results)
    Console.WriteLine($"{result.OriginalIndex}: {result.Score:F3}");

Available Models

Sizes are what a load downloads (the ONNX exports are fp32). multilingual-fast additionally fetches the llama-server binary on first use. Scores are on the same 0..1 scale on both routes, and LocalReranker.IsModelDownloaded / DownloadModelAsync work for every alias above.

For languages other than English and Chinese use multilingual or multilingual-fast: quality and large tokenize any language but were not trained on it, and on a Korean corpus quality can rank worse than no reranking at all.

Tokenizer Auto-Detection

The reranker automatically detects the correct tokenizer type:

This ensures compatibility with virtually any cross-encoder model from HuggingFace.

GPU Acceleration

Do not add ONNX Runtime packages (Microsoft.ML.OnnxRuntime*): LMSupply provisions the runtime itself, and a second copy conflicts with it. ExecutionProvider.Auto uses CUDA when the CUDA 12 runtime and cuDNN 9 are installed on the machine, CoreML on macOS, and the CPU otherwise. On Windows with an AMD or Intel GPU, ONNX sessions run on the CPU (DirectML was removed in 0.67.0). See GPU acceleration.

Configuration

using LMSupply;   // ExecutionProvider

var options = new RerankerOptions
{
    Provider = ExecutionProvider.Auto,  // GPU auto-detection
    MaxSequenceLength = 512,
    BatchSize = 32
};

var reranker = await LocalReranker.LoadAsync("default", options);

Version History

v0.8.9

• Fixed JSON parsing error for Unigram vocab Array format (tuple-like arrays ["token", score])

v0.8.7

• Added automatic tokenizer type detection (WordPiece, Unigram, BPE)
• Fixed compatibility with bge-reranker-base and other Unigram-based models

v0.8.6

• Fixed vocab parsing for Array vs Object format