ActDim.Observability 1.0.6

ActDim.Observability

A lightweight, OpenTelemetry-centric observability library for .NET applications built on top of Microsoft.Extensions.Logging and System.Diagnostics.Activity.

Features

• Zero-Ceremony Developer API: Developers write standard ILogger calls and logger.BeginScope() without needing custom logger interfaces.
• DI Decorator (EventObservabilityLoggerFactory): Transparently decorates ILoggerFactory via DI container to inject EventObservabilityBridge for enriching logs and traces.
• Activity & OpenTelemetry Enrichment: Automatically transforms scope objects, DTOs, and structured log parameters into flattened, dotted OpenTelemetry attributes (user.id, order.price).
• Auto Activity Creation on Scope: Automatically starts an Activity span on logger.BeginScope() when no ambient span exists (Activity.Current == null), resolved via observability.PushActivitySourceName(...) or EventObservabilityOptions.DefaultActivitySourceName.
• Ambient Context Separation: IAmbientContext serves as a neutral ambient variable store. Only properties explicitly pushed via IObservabilityContext are exported to Activity tags.
• Status & Progress Tracking: First-class support for setting operation status text, icons, and progress percentage (observability.SetStatus("Downloading", icon: "🚀"), observability.SetProgress(45.5)).
• Selective Provider & Scope Suppression: Dynamically suppress console loggers, specific logger providers, or external scope export per async flow (observability.SuppressConsole(), observability.SuppressProviders("File"), observability.SuppressExternalScopes()).
• Provider Alias Resolution: Automatically resolves provider aliases via official .NET [ProviderAlias] attributes or custom provider mappings.

Architectural Rationale: Dedicated Observability Engines vs Relational Databases

Storing high-throughput logs and distributed traces in traditional relational databases (like PostgreSQL or MySQL) creates significant operational and performance bottlenecks. Dedicated observability engines solve these problems through specialized architectures.

Key Bottlenecks of Relational Databases for Telemetry

• I/O & WAL Overhead: Transactional databases prioritize strict ACID guarantees. Every log entry generates Write-Ahead Log (WAL) traffic and buffer churn, causing massive disk I/O overhead that degrades core application performance.
• Storage Inefficiency & Bloat: Row-oriented architectures compress telemetry poorly. Rotating old data via deletes or TTL triggers heavy background cleanup processes (like VACUUM), causing table bloat and CPU spikes.
• JSON Indexing Trade-Off: Querying dynamic JSON attributes requires heavy indexing (such as GIN indexes), which cripples write speeds and increases index size beyond the data itself. Without indexes, queries result in slow full-table scans.
• Lack of Observability Tooling: Relational databases lack native primitives for live tailing, distributed trace waterfalls (spans/DAGs), and log-centric aggregate pipelines.

Core Advantages of Dedicated Observability Engines

• Columnar & Append-Only Storage: Uses efficient storage engines (e.g., Apache Parquet, LSM-trees) tailored for time-series and log data, achieving 10–15x higher compression ratios.
• Telemetry-First Query Languages: Purpose-built query languages (like LogsQL or telemetry-aware SQL dialects) parse, extract, and filter arbitrary JSON fields on the fly without heavy index maintenance.
• Built-in APM Visualizations: Native support for end-to-end trace waterfalls, span trees, and real-time log streaming right out of the box.

Recommended Lightweight Open-Source Solutions

• VictoriaLogs: A high-performance, resource-efficient log engine requiring minimal CPU and RAM (~50–100 MB). It eliminates high-cardinality bottlenecks, indexes all fields automatically, and features the expressive LogsQL language for structured JSON analysis.
• OpenObserve: A single Rust binary that covers logs, traces, and metrics out of the box. It uses Apache Parquet for storage, natively accepts OpenTelemetry (OTLP) data, and provides a full-featured web UI with trace waterfalls, log exploration, and dashboards without requiring Docker, Java, or external databases.

Installation

Install via the .NET CLI:

dotnet add package ActDim.Observability

Or via Package Manager Console:

Install-Package ActDim.Observability

Registration

Register observability in your IServiceCollection:

services.AddEventObservability(logging =>
{
    logging.AddConsole();
}, options =>
{
    options.IncludeExternalScopes = false; // Default: false
});

Usage

1. Status & Progress Reporting

var observability = serviceProvider.GetRequiredService<IObservabilityContext>();

using (observability.SetStatus("Downloading Dataset", icon: "🚀"))
using (observability.SetProgress(45.5))
using (observability.Push("priority", "high"))
{
    logger.LogInformation("Importing rows into database");
}

2. Method Scopes with OpenTelemetry Semantic Conventions

Use logger.BeginMethodScope() to automatically capture the executing method name, source file, and line number without manual string formatting. Scope properties strictly adhere to the OpenTelemetry Source Code Semantic Conventions:

public class OrderService
{
    private readonly ILogger<OrderService> _logger;

    public OrderService(ILogger<OrderService> logger)
    {
        _logger = logger;
    }

    public async Task ProcessOrderAsync(string orderId)
    {
        // Automatically captures code.function="ProcessOrderAsync", code.filename="OrderService.cs", code.lineno=...
        using (_logger.BeginMethodScope())
        {
            _logger.LogInformation("Processing order {OrderId}", orderId);
        }

        // Merge custom state with caller code context
        using (_logger.BeginMethodScope(new Dictionary<string, object?> { ["OrderId"] = orderId }))
        {
            _logger.LogInformation("Order completed");
        }
    }
}

Why OpenTelemetry Semantic Conventions? Standard attribute names (code.function, code.filepath, code.lineno) enable APM tools, log aggregators, and distributed trace visualizers (Jaeger, Grafana Tempo, Datadog, Dynatrace, and .NET Aspire) to natively index, filter, and navigate directly to source code locations.

3. Selective Provider Suppression

// Suppress Console logger output while preserving Activity traces and other logger sinks
using (observability.SuppressConsole())
{
    logger.LogInformation("Log without console output");
}

// Suppress specific providers by alias or name (e.g., "File", "Console")
using (observability.SuppressProviders("File", "Console"))
{
    logger.LogInformation("Log without File and Console outputs");
}

4. Integration Testing & Tooling (VictoriaLogs & OpenObserve)

ActDim.Observability.Tests includes integration test suites and developer scripts for validating telemetry ingestion and log search:

• VictoriaLogs Integration (VictoriaLogsIntegrationTests):

  • Validates NDJSON ingestion (/insert/jsonline), _msg field format, AmbientContext properties, BeginMethodScope() OTel caller metadata (code.function, code.filename, code.filepath, code.lineno), and LogsQL queries.
  • Launcher & Download Scripts: Tools/victoria-logs/run-victoria-logs.cmd (auto-opens VMUI Web GUI at http://localhost:9428/select/vmui) and download-victoria-logs.cmd.

• OpenObserve Integration (OpenObserveIntegrationTests):

  • Validates JSON log ingestion (/api/{org}/{stream}/_json), AmbientContext enrichment, and SQL Search API (POST /api/{org}/_search).
  • Launcher & Download Scripts: Tools/openobserve/run-openobserve.cmd (auto-opens Web GUI at http://localhost:5080 with default admin credentials root@example.com / Complexpass#123) and download-openobserve.cmd.

• Process Auto-Launch: Both integration tests automatically detect running local instances or auto-launch local binaries from Tools/ into isolated temporary storage paths.

Testing & Quality

• Test Suite: ActDim.Observability.Tests
• Total Tests: 30 passed (100% success rate, 0 failed, 0 skipped)
• Target Framework: .NET 10.0

dotnet test Tests/Observability.Tests/ActDim.Observability.Tests.csproj

License

This project is licensed under the MIT License.