Cerbi.NLog.GovernanceAnalyzer 1.1.15

Cerbi.NLog.GovernanceAnalyzer

Cerbi Governance Analyzer for NLog — compile-time log validation using Roslyn analyzers and JSON-based governance profiles + runtime governance validation with optional score shipping.

Cerbi governs and scores log events at the source. It does not route or replace your existing sinks; it only annotates events with governance metadata and, optionally, ships minimal scoring payloads.

This package lets you enforce logging governance rules for NLog at build time and runtime, so bad log patterns never make it to production, and governance violations are automatically detected and scored.

Part of the Cerbi Suite for governed logging, observability, and compliance.

---

Runtime Governance Target (CerbiGovernanceTarget)

For runtime governance that matches this repo's semantics, use the CerbiGovernance NLog target. It:

• Inspects each LogEventInfo and builds a structured property bag
• Resolves the configured governance profile via GovernanceConfigLoader
• Invokes RuntimeGovernanceEnricher to validate and annotate events
• Attaches canonical metadata keys used throughout this solution:
  • GovernanceScoreImpact (numeric impact score)
  • GovernanceViolations (list of violation strings)
  • GovernanceRelaxed (bool, derived from current governance mode)
• Never drops events; forbidden fields become violations, not filters
• Does not ship or route logs; it only annotates the event

NLog.config Example

<nlog>
  <extensions>
    <add assembly="CerbiStream.GovernanceAnalyzer" />
  </extensions>

  <targets>
    
    <target xsi:type="File" name="file" fileName="logs/app.log" />

    
    <target xsi:type="CerbiGovernance"
            name="cerbi_governance"
            governanceProfile="PIILog"
            enableGovernance="true" />
  </targets>

  <rules>
    
    <logger name="*" minlevel="Info" writeTo="cerbi_governance,file" />
  </rules>
</nlog>

Code-Based Configuration Example

using CerbiStream.GovernanceAnalyzer.Targets;
using NLog;
using NLog.Config;
using NLog.Targets;

var config = new LoggingConfiguration();

var file = new FileTarget("file") { FileName = "logs/app.log" };
config.AddTarget(file);
config.AddRuleForAllLevels(file);

var cerbi = new CerbiGovernanceTarget
{
    Name = "cerbi_governance",
    GovernanceProfile = "PIILog",
    EnableGovernance = true
};
config.AddTarget(cerbi);
config.AddRuleForAllLevels(cerbi);

LogManager.Configuration = config;

With this setup, each log event is validated against the PIILog profile. If violations exist, the event is kept, but the following properties are attached:

• GovernanceScoreImpact – accumulated impact score
• GovernanceViolations – violations like MissingField: userId, ForbiddenField: password, etc.
• GovernanceRelaxed – true when the current mode is non-strict and violations exist

You can then consume these properties downstream (e.g., in JSON layout or external sinks).

---

Runtime Score Shipping (GovernanceScoreTarget)

For optional centralized scoring payloads, you can continue to use the GovernanceScore target. It looks for:

• GovernanceScoreImpact (numeric, required to emit)
• GovernanceViolations (optional list/string)
• GovernanceRelaxed (optional bool)

If present and shipping is enabled and licensed, a GovernanceScoreEvent is enqueued and batch-shipped asynchronously via ScoreShipper.

NLog Config Example (Score Target)

<nlog>
  <extensions>
    <add assembly="CerbiStream.GovernanceAnalyzer" />
  </extensions>
  <targets>
    <target xsi:type="GovernanceScore"
            name="governanceScore"
            appName="MyApp"
            environment="Production"
            enableAutoEnrichment="true"
            governanceProfile="PIILog"
            scoreShippingEnabled="true"
            licenseAllowsScoring="true"
            scoreEndpoint="https://score.cerbi.systems/ingest"
            scoreMaxBatchSize="50"
            scoreMaxQueueSize="1000"
            scoreFlushIntervalSeconds="5" />
  </targets>
  <rules>
    <logger name="*" minlevel="Info" writeTo="governanceScore" />
  </rules>
</nlog>

How it fits together:

• CerbiGovernanceTarget performs local validation and tagging only.
• GovernanceScoreTarget consumes the governance metadata and ships compact score events.
• Your existing NLog targets (file, console, JSON, etc.) continue to handle routing and storage.

---

📆 What It Does

Cerbi.NLog.GovernanceAnalyzer provides dual-layer governance:

Compile-Time (Roslyn Analyzer)

• ✅ Enforce required fields (e.g., userId, operationId)
• ❌ Block or warn on forbidden fields (e.g., ssn, creditCardNumber)
• 🧠 Validate field types, enums, and encryption settings
• 🔍 Violations surfaced as compiler diagnostics in Visual Studio, Rider, CI/CD

Runtime (CerbiGovernanceTarget + GovernanceScoreTarget)

• 🔄 Realtime validation: automatically validates logs against governance profiles
• 📊 Impact scoring: computes violation severity (MissingField=5pts, Forbidden=10pts, etc.)
• 📡 Async score shipping: via GovernanceScoreTarget and ScoreShipper
• ⚡ High throughput: non-blocking enqueue, channel-based queue for shipping
• 🎯 Auto-enrichment: opt-in via GovernanceScoreTarget.EnableAutoEnrichment

---

🔧 Example – NLog Usage & Diagnostics

_logger.Info("userId={0}, ssn={1}", "abc-123", "123-45-6789");

If your cerbi_governance.json declares:

• userId as Required
• ssn as Forbidden

Compile-Time

[CERBI001] Missing Required Field: 'userId'
[CERBI002] Forbidden Field: 'ssn'

Runtime

{
  "App": "MyApp",
  "Environment": "Production",
  "Impact": 15.0,
  "Violations": ["MissingField:userId", "ForbiddenField:ssn"],
  "Relaxed": false
}

---

📄 Governance Configuration

Governance rules are defined in a JSON profile shared with the rest of the Cerbi ecosystem (via Cerbi.Governance.Core).

Example cerbi_governance.json:

{
  "EnforcementMode": "Strict",
  "LoggingProfiles": {
    "PIILog": {
      "AllowedLevels": [ "Info", "Warn", "Error" ],
      "FieldSeverities": {
        "userId": "Required",
        "ssn": "Forbidden",
        "sessionId": "Warning"
      },
      "FieldTypes": {
        "userId": "Guid"
      },
      "FieldEnums": {
        "Region": ["US", "EU", "APAC"]
      }
    }
  }
}

In your .csproj, make sure the config is copied alongside your build output:

<ItemGroup>
  <None Include="cerbi_governance.json">
    <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  </None>
</ItemGroup>

Tip: You can keep multiple profiles (e.g. default, PIILog, AuditLog) and select them via Cerbi configuration. The NLog analyzer consumes whatever profile you point it at.

---

🚀 Runtime Governance Score Shipping (Optional)

Enable automatic validation and scoring by adding the custom NLog target GovernanceScore. It inspects LogEventInfo.Properties for these keys:

• GovernanceScoreImpact (numeric) – REQUIRED to emit an event
• GovernanceViolations (list/string) – optional list of violation messages
• GovernanceRelaxed (bool) – optional flag indicating relaxed enforcement mode

If present, and shipping is enabled & licensed, a GovernanceScoreEvent is queued and batch-shipped asynchronously.

NLog Config Example

<nlog>
  <extensions>
    <add assembly="CerbiStream.GovernanceAnalyzer" />
  </extensions>
  <targets>
    <target xsi:type="GovernanceScore"
            name="governanceScore"
            appName="MyApp"
            environment="Production"
            enableAutoEnrichment="true"
            governanceProfile="PIILog"
            scoreShippingEnabled="true"
            licenseAllowsScoring="true"
            scoreEndpoint="https://score.cerbi.systems/ingest"
            scoreMaxBatchSize="50"
            scoreMaxQueueSize="1000"
            scoreFlushIntervalSeconds="5" />
  </targets>
  <rules>
    <logger name="*" minlevel="Info" writeTo="governanceScore" />
  </rules>
</nlog>

How it works:

1. Every log passes through RuntimeGovernanceEnricher
2. Validates against PIILog profile
3. If violations exist, computes impact score
4. Attaches GovernanceScoreImpact, GovernanceViolations, GovernanceRelaxed
5. Ships to endpoint via async batch

Manual Mode

Attach properties manually:

var logEvent = new LogEventInfo(LogLevel.Info, "", "Example governance event");
logEvent.Properties["GovernanceScoreImpact"] = 12.5;
logEvent.Properties["GovernanceViolations"] = new[]{"MissingField:userId"};
logEvent.Properties["GovernanceRelaxed"] = false;
Logger.Log(logEvent);

Gating Rules

Event shipped only if:

1. scoreShippingEnabled = true
2. licenseAllowsScoring = true
3. GovernanceScoreImpact exists & parses as numeric

Non-blocking: dropped if queue full; no hot-path locks.

---

⚙️ Configuration Reference (Target Attributes)

Attribute	Default	Description
appName	(required)	Application identifier in shipped events.
environment	(required)	Deployment environment (Prod/Stage/Dev).
enableAutoEnrichment	false	Auto-validate and score every log.
governanceProfile	"default"	Profile name for auto-enrichment validation.
scoreShippingEnabled	false	Master switch for score emission.
licenseAllowsScoring	false	License gate (entitlement check).
scoreMaxBatchSize	50	Max events per HTTP POST batch.
scoreMaxQueueSize	1000	Queue capacity; drops when full.
scoreFlushIntervalSeconds	5	Background flush interval.
scoreMaxRetries	3	HTTP retry attempts per batch.
scoreRetryBackoffMilliseconds	250	Base backoff; scales linearly (250ms, 500ms, 750ms).
scoreEndpoint	https://cerbi.systems/score	POST destination URL.
scoreTransport	Http	Transport used for score ingestion (Http or AzureServiceBusQueue).
serviceBusConnectionString	(blank)	Required when scoreTransport=AzureServiceBusQueue; connection string for the namespace.
serviceBusQueueName	(blank)	Required when scoreTransport=AzureServiceBusQueue; destination queue name.

You can also programmatically configure the target after LogManager.Configuration loads if needed.

---

📑 GovernanceScoreEvent Schema (JSON Batch Payload)

A flush sends a JSON array of events:

{
  "App": "MyApp",
  "Environment": "Production",
  "UtcTimestamp": "2025-01-15T12:34:56Z",
  "Impact": 15.0,
  "Relaxed": false,
  "Violations": [
    "MissingField: userId (User identifier)",
    "ForbiddenField: ssn (Social Security Number)"
  ],
  "Properties": {
    "CorrelationId": "abc-123",
    "RequestPath": "/api/users"
  }
}

Reserved property keys are not duplicated inside Properties: GovernanceScoreImpact, GovernanceViolations, GovernanceRelaxed.

---

⚡ Performance & Reliability

High-Throughput Design

• Channel-based queue: System.Threading.Channels with BoundedChannelFullMode.DropWrite
• Non-blocking enqueue: O(1) lock-free write; drops when full (no hot-path contention)
• Async HTTP: HttpClient.PostAsync with fire-and-forget batching
• Background flushing: continuous Task.Run loop eliminates timer overhead
• Profile caching: loaded once on InitializeTarget(), reused per event
• Inline validation: avoids external helper call overhead
• Fast impact scoring: char prefix matching (err[0] == 'M') instead of Contains()
• Zero-allocation enqueue: no intermediate buffers, direct channel write

Benchmark

Metric	Performance
Enqueue throughput	100,000+ events/sec
Hot-path overhead	<1 µs (cached profile + fast ContainsKey)
Queue capacity	1,000 events (configurable)
Batch size	50 events/POST (configurable)
Concurrent flushing	Supported (fire-and-forget async)
Backpressure	Drops events when queue full (no blocking)

Failure Modes

• HTTP failure: retries with exponential backoff; drops batch after MaxRetries
• Malformed impact: suppresses event emission (no exception)
• Queue overflow: silently drops new events (non-blocking)
• Shutdown: drains remaining events on Dispose() (5s timeout)

---

🔐 License Gating

Two independent gates:

1. Feature flag: scoreShippingEnabled (on/off switch)
2. Entitlement check: licenseAllowsScoring (license validation)

Use licenseAllowsScoring=false to disable dynamically (e.g., expired contract, regional restriction, missing entitlement).

---

🧪 Testing Guidance

Unit Tests

// Test auto-enrichment
var target = CreateTarget(fakeShipper, t => {
    t.EnableAutoEnrichment = true;
    t.GovernanceProfile = "PIILog";
});

var logEvent = new LogEventInfo(LogLevel.Info, "", "test");
logEvent.Properties["password"] = "secret"; // forbidden field
target.Emit(logEvent);

Assert.Equal(1, fakeShipper.EnqueueCount);
Assert.True(fakeShipper.LastEvent.Impact > 0);
Assert.Contains("ForbiddenField: password", fakeShipper.LastEvent.Violations);

High-Throughput Test

[Fact]
public async Task HighThroughput_ThousandsOfEvents_NoBlocking()
{
    var shipper = new ScoreShipper(httpClient, options);
    var start = DateTime.UtcNow;
    
    for (int i = 0; i < 1000; i++)
        shipper.Enqueue(new GovernanceScoreEvent { Impact = i });
    
    var elapsed = DateTime.UtcNow - start;
    Assert.True(elapsed.TotalMilliseconds < 100); // <100ms for 1000 events
}

---

Testing & Validation

All runtime behaviors discussed above are covered by the automated test suite:

• CerbiGovernanceTargetTests exercise the target in isolation (valid, missing required, forbidden, relax scenarios).
• CerbiGovernancePipelineTests wire CerbiGovernanceTarget together with GovernanceScoreTarget + a fake shipper to prove end-to-end score payload generation in both strict and warn-only (relaxed) modes.

Run dotnet test inside the solution root to execute the full suite.

---

🔌 Extensibility

Custom Shipper

Replace internal ScoreShipper with custom implementation:

public class MyCustomTarget : GovernanceScoreTarget
{
    protected override void InitializeTarget()
    {
        // Inject custom shipper with metrics, circuit breaker, etc.
        _scoreShipper = new MyCustomShipper(_httpClient, ScoreShipping);
    }
}

Custom Validation

Wrap additional metadata:

logEvent.Properties["TeamId"] = "team-42";
logEvent.Properties["ServiceVersion"] = "1.2.3";
// Automatically copied to GovernanceScoreEvent.Properties

---

📦 Installation

dotnet add package Cerbi.NLog.GovernanceAnalyzer

• Analyzer: runs automatically at build
• Target: activates via <extensions> in NLog config

---

🔐 Encryption & NLog

This package does not implement log payload encryption. NLog lacks first-class encryption support for structured data.

For end-to-end encrypted logging with governance-aware encryption modes (AES/Base64/None):

• CerbiStream – governed logging library
• Cerbi.Governance.Runtime – runtime validation engine

---

🧠 IDE & CI Integration

Local Development

• Underlines / squiggles in Visual Studio and Rider
• Diagnostics in Error List
• Treat as warnings for dev productivity

CI/CD

dotnet build /warnaserror:CERBI001,CERBI002

• Escalate to errors for critical profiles (PII/PHI)
• Block deployment on governance violations

---

🧹 Plugin Architecture

Layer custom rules via ICustomGovernancePlugin:

public class TeamIdPlugin : ICustomGovernancePlugin
{
    public bool Validate(
        string profile,
        Dictionary<string, object> logData,
        out List<string> failures,
        out int impactScore)
    {
        failures = new();
        impactScore = 0;

        var isProd = logData.TryGetValue("Environment", out var env) &&
                     env?.ToString() == "Production";

        if (isProd && !logData.ContainsKey("TeamId"))
        {
            failures.Add("MissingField: TeamId is required in Production.");
            impactScore = 5;
            return false;
        }
        return true;
    }
}

Use plugins for:

• Environment-specific rules (stricter in Production)
• Team/domain-specific checks
• Custom impact scoring for dashboards (CerbiShield / CerbIQ)

---

🌍 How It Fits with Other Loggers & Stacks

This package targets NLog specifically, but it’s part of a bigger story:

• Serilog / MEL / NLog / Log4Net
  • Other Cerbi analyzers and runtime components cover these stacks.
• OpenTelemetry Logging / OTLP / Collector
  • Cerbi ensures logs are governed before they’re exported and shipped downstream.
• Seq, Grafana Loki / Promtail / Alloy, Fluentd / FluentBit, ELK / OpenSearch, Graylog, VictoriaLogs / VictoriaMetrics, Journald / syslog
  • Keep using your existing sinks; this analyzer + target just keeps the input sane and compliant.

Cerbi.NLog.GovernanceAnalyzer doesn’t replace NLog or your observability platform. It makes sure your NLog usage is safe, structured, and policy-compliant.

---

📜 License

MIT © Cerbi LLC — https://cerbi.io

See LICENSE.txt for details.

---

Queue-Based Scoring Ingest

By default ScoreShipper batches to the HTTPS endpoint specified in ScoreShippingOptions.Endpoint. Set scoreTransport="AzureServiceBusQueue" on GovernanceScore to use the built-in Service Bus driver, or plug in your own IScoreShipper implementation (see ClassLibrary1/Services/IScoreShipper.cs) for other queues.

• Azure Service Bus (v1) – first-class driver for marketplace deployments; the default queue option when you deploy the managed offering. Configure by setting scoreTransport="AzureServiceBusQueue" plus serviceBusConnectionString and serviceBusQueueName on the GovernanceScore target:

<target xsi:type="GovernanceScore"
        name="governanceScore"
        scoreTransport="AzureServiceBusQueue"
        serviceBusConnectionString="Endpoint=sb://..."
        serviceBusQueueName="cerbi-governance-scores"
        ... />

• Azure Storage Queues (planned) – placeholder driver for lighter-weight queue workloads.
• AWS SQS (planned)
• GCP Pub/Sub (planned)

Scoring ingestion therefore supports Azure Service Bus today, with Azure Storage Queues, AWS SQS, and GCP Pub/Sub arriving in upcoming milestones. Both HTTP and queue payloads are accepted by the backend service; on the queue side, we are Service Bus-first until the other drivers land.

---

CI/CD & Publishing

A GitHub Actions workflow (.github/workflows/nuget-publish.yml) builds, tests, packs, and pushes the package to NuGet. Trigger it by pushing a tag like v1.2.3 or running the workflow manually.

1. Store your NuGet API key in the repo settings as NUGET_API_KEY.
2. Push a tag (e.g., git tag v1.2.3 && git push origin v1.2.3) or use Run workflow to publish.
3. Optionally provide versionOverride when running manually to pack a custom version.

The workflow executes:

• dotnet restore
• dotnet build -c Release
• dotnet test on Tests-CerbiStream.GovernanceAnalyzer
• dotnet pack (outputs under artifacts/)
• dotnet nuget push to https://api.nuget.org/v3/index.json