PolyPrompt 2.0.0

<img src="https://github.com/jchristn/PolyPrompt/blob/main/assets/logo.png?raw=true" width="192" height="192">

PolyPrompt

PolyPrompt is a lightweight, unified .NET library for chat completions, tool calling, text generation, embeddings, and model management across Ollama, OpenAI, and Google Gemini APIs. Write your LLM integration code once and swap providers without changing your application logic.

What It Does

PolyPrompt provides a single, consistent API surface for interacting with multiple LLM providers. Instead of learning three different SDKs with different conventions, response formats, and streaming patterns, you use one set of methods that work identically across all supported providers.

• Chat Completions - Streaming and non-streaming conversational AI with system prompts
• Tool Calling - Provider-normalized function declarations, model tool calls, streaming tool-call deltas, and tool-result follow-up messages
• Text Generation - Streaming and non-streaming text generation (completion-style)
• Embeddings - Single and batch embedding vector generation for semantic search and RAG
• Model Management - List models, check existence, get model details, pull, and delete
• Connectivity Validation - Verify provider reachability before running workloads
• Timing Metrics - Built-in performance tracking including time-to-first-token, tokens/sec, and overall throughput
• Call Recording - Every HTTP call is recorded with full request/response details for debugging and auditing
• Provider-Specific Options - Fine-tune each provider's unique parameters without losing portability

Use Cases

PolyPrompt is a good fit when you need to:

• Build provider-agnostic applications - Let users choose their preferred LLM provider (local Ollama, cloud OpenAI, or Google Gemini) without rewriting integration code
• Add tool-backed workflows - Let models request application functions while your code stays in charge of tool execution
• Compare providers side-by-side - Benchmark the same prompts across Ollama, OpenAI, and Gemini to evaluate quality, latency, and cost
• Prototype rapidly - Get a chat completion, embedding, or text generation working in a few lines of code without studying provider-specific SDKs
• Build RAG pipelines - Generate embeddings for document chunks using any provider's embedding models, then query with semantic search
• Create AI-powered CLI tools - The simple API makes it easy to add LLM capabilities to command-line applications
• Manage local model infrastructure - Pull, list, inspect, and delete Ollama models programmatically
• Monitor LLM performance - Use built-in timing metrics and call recording to track latency, throughput, and errors in production
• Build multi-model workflows - Use different providers for different tasks (e.g., Ollama for embeddings, OpenAI for chat) through the same interface

When Not to Use It

PolyPrompt may not be the right choice if you need:

• Advanced multimodal or lifecycle APIs - Vision/image inputs, structured outputs, fine-tuning APIs, batch APIs, and provider-specific agent runtimes are not currently supported
• Automatic agent execution - PolyPrompt returns requested tool calls, but your application executes tools and appends tool results
• Conversation storage - PolyPrompt sends the messages you provide; it does not persist conversation history or manage context windows
• Token counting or cost estimation - While some providers return token usage in responses, PolyPrompt does not provide pre-request token counting
• Official SDK parity - If you need every feature of a specific provider's API, use their official SDK instead

Installation

dotnet add package PolyPrompt

Current documented package version: 2.0.0.

PolyPrompt targets both .NET 8.0 and .NET 10.0.

Quick Start

Ollama

using PolyPrompt.Clients;
using PolyPrompt.Models;

using OllamaClient client = new OllamaClient("http://localhost:11434");
client.Model = "gemma3:4b";

ChatResponse response = await client.ChatAsync("What is the capital of France?");
Console.WriteLine(response.Text);

OpenAI

using PolyPrompt.Clients;
using PolyPrompt.Models;

using OpenAiClient client = new OpenAiClient("https://api.openai.com", "sk-your-api-key");
client.Model = "gpt-4o";

ChatResponse response = await client.ChatAsync("What is the capital of France?");
Console.WriteLine(response.Text);

OpenAI-compatible endpoints may be supplied either as the API root or as a versioned /v1 base URL. For example, an Ollama instance exposing the OpenAI API can be used as:

using PolyPrompt.Clients;

using OpenAiClient client = new OpenAiClient("http://localhost:11434/v1");
client.Model = "gpt-oss:20b";

Gemini

using PolyPrompt.Clients;
using PolyPrompt.Models;

using GeminiClient client = new GeminiClient(
    "https://generativelanguage.googleapis.com",
    "your-api-key");
client.Model = "gemini-2.5-flash";

ChatResponse response = await client.ChatAsync("What is the capital of France?");
Console.WriteLine(response.Text);

Detailed Examples

Chat with System Prompt

using PolyPrompt.Clients;
using PolyPrompt.Models;

using OllamaClient client = new OllamaClient("http://localhost:11434");
client.Model = "gemma3:4b";
client.SystemPrompt = "You are a helpful assistant that responds in haiku format.";
client.Temperature = 0.7;
client.MaxTokens = 256;

ChatResponse response = await client.ChatAsync("Tell me about the ocean.");
if (response.Success)
{
    Console.WriteLine(response.Text);
    Console.WriteLine("Runtime: " + response.OverallRuntimeMs + " ms");
}
else
{
    Console.WriteLine("Error: " + response.Error);
}

Chat with Provider-Specific Options

using PolyPrompt.Clients;
using PolyPrompt.Models;
using PolyPrompt.Options;

using OllamaClient client = new OllamaClient("http://localhost:11434");
client.Model = "gemma3:4b";

OllamaChatCompletionOptions options = new OllamaChatCompletionOptions();
options.Temperature = 0.5;
options.TopP = 0.9;
options.MaxTokens = 512;
options.TopK = 40;
options.RepeatPenalty = 1.1;
options.Seed = 42;
options.SystemPrompt = "You are a concise technical writer.";

ChatResponse response = await client.ChatAsync("Explain dependency injection.", options);
Console.WriteLine(response.Text);

Tool Calling

Tool calling is explicit. Use ToolChatAsync or ToolChatStreamingAsync when a model may request application functions, then execute those functions in your code and send the result back as another message. PolyPrompt normalizes the provider protocol; it does not run your tools for you.

using PolyPrompt.Clients;
using PolyPrompt.Models;

using OpenAiClient client = new OpenAiClient("https://api.openai.com", "sk-your-api-key");
client.Model = "gpt-4o-mini";

ToolChatRequest request = new ToolChatRequest();
request.Messages.Add(ChatMessage.System("Answer with practical weather guidance."));
request.Messages.Add(ChatMessage.User("What is the weather in Seattle, and should I bring a jacket?"));
request.Tools.Add(ToolDefinition.Function(
    "get_weather",
    "Get current weather for a city.",
    new Dictionary<string, object>
    {
        { "type", "object" },
        { "properties", new Dictionary<string, object>
            {
                { "city", new Dictionary<string, object>
                    {
                        { "type", "string" },
                        { "description", "City name." }
                    }
                },
                { "unit", new Dictionary<string, object>
                    {
                        { "type", "string" },
                        { "enum", new List<string> { "fahrenheit", "celsius" } }
                    }
                }
            }
        },
        { "required", new List<string> { "city" } }
    }));

ToolChatResponse first = await client.ToolChatAsync(request);

if (first.ToolCalls.Count > 0)
{
    request.Messages.Add(first.ToAssistantMessage());
}

foreach (ToolCall call in first.ToolCalls)
{
    if (call.Name == "get_weather")
    {
        string weatherJson = "{\"temperature\":72,\"conditions\":\"clear\"}";
        request.Messages.Add(ChatMessage.ToolResult(call.Id, call.Name, weatherJson));
    }
}

request.Tools.Clear();
request.ToolChoice = "none";

ToolChatResponse final = await client.ToolChatAsync(request);
Console.WriteLine(final.Text);

Streaming Tool Calling

ToolChatStreamingAsync streams assistant text and tool-call deltas while accumulating final Text and ToolCalls on the response as you enumerate Chunks. OpenAI-compatible, Ollama, and Gemini clients support it.

ToolChatStreamingResponse stream = await client.ToolChatStreamingAsync(request);

await foreach (ToolChatStreamingChunk chunk in stream.Chunks)
{
    if (!string.IsNullOrEmpty(chunk.Text))
    {
        Console.Write(chunk.Text);
    }
}

if (stream.ToolCalls.Count > 0)
{
    request.Messages.Add(stream.ToAssistantMessage());

    foreach (ToolCall call in stream.ToolCalls)
    {
        string resultJson = "{\"temperature\":72,\"conditions\":\"clear\"}";
        request.Messages.Add(ChatMessage.ToolResult(call.Id, call.Name, resultJson));
    }
}

Provider protocol shapes differ:

• OpenAI-compatible uses /v1/chat/completions SSE chunks and parses delta.tool_calls argument fragments.
• Ollama uses /api/chat newline-delimited JSON chunks and parses streamed message.tool_calls.
• Gemini uses models/{model}:streamGenerateContent?alt=sse with the same GenerateContentRequest body shape as ToolChatAsync: contents, optional systemInstruction, tools.functionDeclarations, and toolConfig. It parses streamed GenerateContentResponse chunks from candidates[].content.parts[], including text, complete functionCall objects, finishReason, responseId, modelVersion, and usageMetadata.

Streaming Chat

using PolyPrompt.Clients;
using PolyPrompt.Models;

using OpenAiClient client = new OpenAiClient("https://api.openai.com", "sk-your-api-key");
client.Model = "gpt-4o";

ChatStreamingResponse stream = await client.ChatStreamingAsync("Write a short story about a robot.");

await foreach (ChatStreamingChunk chunk in stream.Chunks)
{
    if (!string.IsNullOrEmpty(chunk.Text))
    {
        Console.Write(chunk.Text);
    }
}

Console.WriteLine();
Console.WriteLine("Time to first token: " + stream.TimeToFirstTokenMs + " ms");
Console.WriteLine("Tokens/sec: " + stream.OverallTokensPerSecond.ToString("F1"));
Console.WriteLine("Total chunks: " + stream.ChunkCount);

Single Embedding

using PolyPrompt.Clients;
using PolyPrompt.Models;

using OllamaClient client = new OllamaClient("http://localhost:11434");

OllamaEmbeddingOptions options = new OllamaEmbeddingOptions();
options.Model = "all-minilm";

EmbeddingResponse response = await client.EmbedAsync("The quick brown fox jumps over the lazy dog.", options);
if (response.Success && response.Embeddings.Count > 0)
{
    float[] vector = response.Embeddings[0].Embedding;
    Console.WriteLine("Dimensions: " + vector.Length);
    Console.WriteLine("First 5 values: " + string.Join(", ", vector.Take(5)));
}

Batch Embeddings

using PolyPrompt.Clients;
using PolyPrompt.Models;

using OpenAiClient client = new OpenAiClient("https://api.openai.com", "sk-your-api-key");

OpenAiEmbeddingOptions options = new OpenAiEmbeddingOptions();
options.Model = "text-embedding-3-small";
options.Dimensions = 256;

List<string> documents = new List<string>
{
    "Machine learning is a subset of artificial intelligence.",
    "Neural networks are inspired by biological neurons.",
    "Deep learning uses multiple layers of neural networks."
};

EmbeddingResponse response = await client.EmbedAsync(documents, options);
if (response.Success)
{
    for (int i = 0; i < response.Embeddings.Count; i++)
    {
        Console.WriteLine("Document " + i + ": " + response.Embeddings[i].Embedding.Length + " dimensions");
    }
}

Text Generation (Non-Streaming)

using PolyPrompt.Clients;
using PolyPrompt.Models;

using OllamaClient client = new OllamaClient("http://localhost:11434");
client.Model = "gemma3:4b";

GenerationResponse response = await client.GenerateAsync("Once upon a time, in a land far away,");
Console.WriteLine(response.Text);
Console.WriteLine("Runtime: " + response.OverallRuntimeMs + " ms");

Text Generation (Streaming)

using PolyPrompt.Clients;
using PolyPrompt.Models;

using GeminiClient client = new GeminiClient(
    "https://generativelanguage.googleapis.com",
    "your-api-key");
client.Model = "gemini-2.5-flash";

GenerationStreamingResponse stream = await client.GenerateStreamingAsync("Write a limerick about coding.");

await foreach (GenerationStreamingChunk chunk in stream.Chunks)
{
    if (!string.IsNullOrEmpty(chunk.Text))
    {
        Console.Write(chunk.Text);
    }
}

Console.WriteLine();
Console.WriteLine("Time to first token: " + stream.TimeToFirstTokenMs + " ms");
Console.WriteLine("Tokens/sec: " + stream.OverallTokensPerSecond.ToString("F1"));

List Available Models

using PolyPrompt.Clients;
using PolyPrompt.Models;

using OllamaClient client = new OllamaClient("http://localhost:11434");

await foreach (ModelInformation model in client.ListModelsAsync())
{
    Console.WriteLine(model.Name
        + (model.DisplayName != null ? " (" + model.DisplayName + ")" : "")
        + (model.SizeBytes.HasValue ? " [" + (model.SizeBytes.Value / 1_000_000_000.0).ToString("F1") + " GB]" : ""));
}

Check If a Model Exists

using PolyPrompt.Clients;

using OllamaClient client = new OllamaClient("http://localhost:11434");

bool exists = await client.ModelExistsAsync("gemma3:4b");
Console.WriteLine("gemma3:4b exists: " + exists);

// Also matches without tags: "gemma3" matches "gemma3:latest"
bool existsNoTag = await client.ModelExistsAsync("gemma3");
Console.WriteLine("gemma3 exists: " + existsNoTag);

Get Model Details

using PolyPrompt.Clients;
using PolyPrompt.Models;

using OllamaClient client = new OllamaClient("http://localhost:11434");

ModelInformation? info = await client.GetModelInformationAsync("gemma3:4b");
if (info != null)
{
    Console.WriteLine("Name: " + info.Name);
    Console.WriteLine("Modified: " + info.ModifiedUtc);

    foreach (KeyValuePair<string, string?> kv in info.Metadata)
    {
        Console.WriteLine("  " + kv.Key + ": " + kv.Value);
    }
}

Pull a Model (Ollama)

using PolyPrompt.Clients;
using PolyPrompt.Models;

using OllamaClient client = new OllamaClient("http://localhost:11434");

bool success = await client.PullModelAsync("gemma3:4b", async (ModelPullProgress progress) =>
{
    if (progress.PercentComplete.HasValue)
    {
        Console.Write("\r" + progress.Status + " " + progress.PercentComplete.Value.ToString("F1") + "%");
    }
    else
    {
        Console.WriteLine(progress.Status);
    }
});

Console.WriteLine();
Console.WriteLine(success ? "Pull succeeded." : "Pull failed.");

Delete a Model (Ollama)

using PolyPrompt.Clients;

using OllamaClient client = new OllamaClient("http://localhost:11434");

bool deleted = await client.DeleteModelAsync("gemma3:4b");
Console.WriteLine(deleted ? "Model deleted." : "Delete failed.");

Validate Connectivity

using PolyPrompt.Clients;

using GeminiClient client = new GeminiClient(
    "https://generativelanguage.googleapis.com",
    "your-api-key");

bool reachable = await client.ValidateConnectivityAsync();
Console.WriteLine(reachable ? "Connected." : "Cannot reach provider.");

Inspect Call Details

using PolyPrompt.Clients;
using PolyPrompt.Models;

using OllamaClient client = new OllamaClient("http://localhost:11434");
client.Model = "gemma3:4b";

ChatResponse response = await client.ChatAsync("Hello!");

foreach (CompletionCallDetail detail in client.CallDetails)
{
    Console.WriteLine(detail.Method + " " + detail.Url);
    Console.WriteLine("  Status: " + detail.StatusCode);
    Console.WriteLine("  Time: " + detail.ResponseTimeMs + " ms");
    Console.WriteLine("  Success: " + detail.Success);
}

// CallDetails returns a detached snapshot. Use MaxCallDetails to bound retention
// and ClearCallDetails to release retained diagnostics on long-lived clients.
client.MaxCallDetails = 100;
client.ClearCallDetails();

Using CancellationToken

using PolyPrompt.Clients;
using PolyPrompt.Models;

using OllamaClient client = new OllamaClient("http://localhost:11434");
client.Model = "gemma3:4b";
client.TimeoutMs = 10000;

using CancellationTokenSource cts = new CancellationTokenSource(TimeSpan.FromSeconds(10));

try
{
    ChatResponse response = await client.ChatAsync("Write a very long essay.", token: cts.Token);
    Console.WriteLine(response.Text);
}
catch (OperationCanceledException)
{
    Console.WriteLine("Request was cancelled.");
}

TimeoutMs is enforced with per-call cancellation tokens and is honored for both non-streaming requests and streaming response bodies. Values must be greater than zero and are not silently clamped.

Provider-Agnostic Code

using PolyPrompt.Clients;
using PolyPrompt.Models;

CompletionClientBase CreateClient(string provider, string endpoint, string? apiKey)
{
    switch (provider)
    {
        case "ollama":
            return new OllamaClient(endpoint, apiKey);
        case "openai":
            return new OpenAiClient(endpoint, apiKey);
        case "gemini":
            return new GeminiClient(endpoint, apiKey);
        default:
            throw new ArgumentException("Unknown provider: " + provider);
    }
}

// Same code works regardless of provider
using CompletionClientBase client = CreateClient("ollama", "http://localhost:11434", null);
client.Model = "gemma3:4b";

ChatResponse chat = await client.ChatAsync("Hello!");
Console.WriteLine(chat.Text);

await foreach (ModelInformation model in client.ListModelsAsync())
{
    Console.WriteLine("  " + model.Name);
}

API Reference

Client Properties

Client Methods

Tool Calling Models

ToolChatAsync and ToolChatStreamingAsync use a message-based request because tool calling is inherently multi-step. A model can return tool calls instead of final text, and the caller decides how to execute those tools.

Provider-Specific Options

Each provider exposes option classes that extend the base options with provider-specific parameters:

Ollama-specific parameters: ContextLength, TopK, RepeatPenalty, Seed, MinP, RepeatLastN

OpenAI-specific parameters: FrequencyPenalty, PresencePenalty, Seed, Dimensions, EncodingFormat, Echo, Suffix, Logprobs

Gemini-specific parameters: TopK, CandidateCount, PresencePenalty, FrequencyPenalty, TaskType, Title

Default Models

Provider Feature Support

Unsupported entries are intentionally explicit. PolyPrompt prefers a clear provider-level NotSupportedException over silently falling back to a different protocol shape.

Ollama tool calling is model-dependent. For example, gemma3:4b is a valid Ollama chat, streaming chat, and generation model, but Ollama reports that it does not support tools. Use a tool-capable model such as gpt-oss:20b when you want the live suite to exercise actual Ollama tool-call and streaming tool-call paths.

Project Structure

PolyPrompt/
|-- src/
|   |-- PolyPrompt/              # Core library (NuGet package)
|   |   |-- Clients/             # CompletionClientBase, OllamaClient, OpenAiClient, GeminiClient
|   |   |-- Models/              # Request/response data models
|   |   `-- Options/             # Provider-specific option classes
|   |-- OllamaConsole/           # Interactive Ollama test harness, including tc/toolchat
|   |-- OpenAIConsole/           # Interactive OpenAI test harness, including tc/toolchat
|   |-- GeminiConsole/           # Interactive Gemini test harness, including tc/toolchat
|   |-- Test.Shared/             # Shared Touchstone test descriptors
|   |-- Test.Automated/          # Touchstone console runner
|   |-- Test.Xunit/              # xUnit adapter over Test.Shared
|   `-- Test.Nunit/              # NUnit adapter over Test.Shared
`-- assets/
    `-- logo.png

Building from Source

dotnet restore src/PolyPrompt.sln
dotnet build src/PolyPrompt.sln

Running the Automated Tests

# Local self-tests for request translation, timeout, cancellation, response disposal, CallDetails, chat, streaming chat, tool chat, streaming tool chat, generation, embeddings, and model management
dotnet run --project src/Test.Automated --framework net8.0 -- selftest

# Local self-tests through xUnit and NUnit
dotnet test src/Test.Xunit/Test.Xunit.csproj
dotnet test src/Test.Nunit/Test.Nunit.csproj

# Live provider tests through the Touchstone console runner. OpenAI and Gemini default to their public API endpoints.
dotnet run --project src/Test.Automated -- --openai-key sk-your-key --openai-model gpt-4o-mini
dotnet run --project src/Test.Automated -- --ollama-endpoint http://localhost:11434 --ollama-model gpt-oss:20b --ollama-embedding-model all-minilm
dotnet run --project src/Test.Automated -- --gemini-key your-key --gemini-model gemini-2.5-flash

# Ollama can also be validated through its OpenAI-compatible /v1 API.
dotnet run --project src/Test.Automated -- --openai-endpoint http://localhost:11434/v1 --openai-model gpt-oss:20b --openai-embedding-model all-minilm

# Live tool-chat cases verify successful tool use when the configured model supports tools,
# and verify the provider's unsupported-model error when it does not.

# Generic named form and positional form are also supported
dotnet run --project src/Test.Automated -- --provider ollama --endpoint http://localhost:11434 --model gpt-oss:20b --embedding-model all-minilm
dotnet run --project src/Test.Automated -- ollama http://localhost:11434 "" gpt-oss:20b all-minilm

# Live provider tests can also be enabled for xUnit and NUnit with environment variables
set POLYPROMPT_TEST_PROVIDER=ollama
set POLYPROMPT_TEST_ENDPOINT=http://localhost:11434
set POLYPROMPT_TEST_MODEL=gpt-oss:20b
set POLYPROMPT_TEST_EMBEDDING_MODEL=all-minilm
dotnet test src/Test.Xunit/Test.Xunit.csproj
dotnet test src/Test.Nunit/Test.Nunit.csproj

# Provider-specific environment variables can be used instead of POLYPROMPT_TEST_PROVIDER
set POLYPROMPT_TEST_OPENAI_API_KEY=sk-your-key
set POLYPROMPT_TEST_OPENAI_MODEL=gpt-4o-mini
dotnet test src/Test.Xunit/Test.Xunit.csproj

Issues and Discussions

Have a bug to report or a feature to request? Please open an issue on GitHub:

https://github.com/jchristn/PolyPrompt/issues

Want to ask a question or start a conversation? Use GitHub Discussions:

https://github.com/jchristn/PolyPrompt/discussions

License

PolyPrompt is available under the MIT License. See the LICENSE.md file for full details.