SiLA2.Audit 10.2.2

SiLA2.Audit

NuGet Package	SiLA2.Audit on NuGet.org
Repository	https://gitlab.com/SiLA2/sila_csharp
SiLA Standard	https://sila-standard.com
License	MIT

Automatic Auditing in 3 Lines of Code

Add comprehensive audit logging to your entire SiLA2 server with just 3 lines:

builder.Services.AddAuditServiceWithInterceptor();
builder.Services.AddGrpc(options => options.Interceptors.Add<AuditInterceptor>());
app.MapGrpcService<YourFeatureService>();  // All calls automatically audited!

That's it! Every command, property access, and error is now automatically logged. Zero code changes needed in your service implementations.

---

About the SiLA2 C# Implementation

SiLA2.Audit is an optional module within the broader .NET 10 implementation of the SiLA2 Standard for laboratory automation. Understanding how this audit module fits into the overall architecture will help you integrate it effectively into your SiLA2 servers.

How This Module Fits In

The SiLA2 C# implementation follows a Feature Definition Language (FDL) → XSLT → Proto → C# pipeline architecture:

• Laboratory features are defined in .sila.xml files
• XSLT transforms generate Protocol Buffer definitions
• gRPC services are auto-generated from .proto files
• Service implementations provide the business logic

SiLA2.Audit sits at the gRPC layer, using interceptors to automatically capture all RPC calls without requiring changes to your feature implementations. This design means:

• ✅ Works with any SiLA2 feature (core or custom)
• ✅ No code changes needed in service implementations
• ✅ Consistent auditing across all features
• ✅ Captures the complete lifecycle of commands, properties, and errors

Module Structure

SiLA2.Audit is categorized as an Optional Module in the SiLA2 ecosystem, alongside:

• SiLA2.Database.SQL - EntityFramework Core integration
• SiLA2.Database.NoSQL - NoSQL database abstractions
• SiLA2.IPC.NetMQ - Inter-process communication
• SiLA2.AnIML - Scientific data format support

Core dependencies:

• SiLA2.Core - Server implementation and domain models
• SiLA2.AspNetCore - ASP.NET Core integration and DI extensions

When to Use This Module

Add SiLA2.Audit to your project when you need:

• Compliance: FDA 21 CFR Part 11, EU Annex 11, GDPR requirements
• Traceability: Complete audit trails for laboratory workflows
• Debugging: Detailed logs of all server operations
• Security: Track who accessed what and when
• Performance Monitoring: Identify slow commands and bottlenecks

Where to Learn More

For comprehensive information about the SiLA2 C# implementation:

• Main Repository README - Architecture overview, build commands, getting started guide
• Project Wiki - Detailed documentation and tutorials
• Example Implementations - TemperatureController, ShakerController, and more

This document focuses on SiLA2.Audit-specific features. Refer to the main README for:

• Overall project architecture and FDL → Proto pipeline
• Building and running SiLA2 servers
• Implementing custom features
• Client-server communication patterns

---

What Gets Automatically Audited

With the interceptor enabled, every gRPC call is automatically logged with:

Unary Commands (Standard RPC)

✅ Command Started
   - Timestamp (UTC, millisecond precision)
   - Command identifier (e.g., "SetTemperature")
   - Feature identifier (extracted from gRPC path)
   - Request parameters (JSON serialized)
   - Client metadata (IP, headers, user identity)
   - Correlation ID (for tracking related events)

✅ Command Completed
   - Response data (JSON serialized)
   - Execution duration (milliseconds)
   - Success status

✅ Command Failed (if error occurs)
   - Error message
   - Full stack trace
   - Error type (RpcException, ValidationError, etc.)
   - Execution duration up to failure

Observable Commands (Long-Running)

✅ Command Initiated
   - Command execution UUID
   - Initial parameters

✅ Progress Updates (automatically tracked)
   - Execution status (Running, Completed, Failed)
   - Progress percentage
   - Estimated remaining time

✅ Result Retrieved
   - Final response data
   - Total execution time

Observable Properties (Subscriptions)

✅ Subscription Started
   - Property identifier
   - Subscriber information

✅ Subscription Ended
   - Duration
   - Number of values emitted

Errors (Any Exception)

✅ Error Event
   - Exception type and message
   - Stack trace
   - Context (which command/property)
   - Associated correlation ID

All of this happens automatically with zero code in your service methods!

---

Why This Is So Easy

Traditional Approach (Without Interceptor)

You'd need to manually add audit code to every single RPC method:

public override async Task<Response> MyCommand(Parameters request, ServerCallContext context)
{
    // ❌ Manual audit code in EVERY method
    var correlationId = Guid.NewGuid().ToString();
    var stopwatch = Stopwatch.StartNew();

    await _audit.RecordEventAsync(new CommandAuditEvent
    {
        EventType = AuditEventType.CommandStarted,
        // ... 20 lines of boilerplate ...
    });

    try
    {
        var response = DoWork(request);

        stopwatch.Stop();
        await _audit.RecordEventAsync(new CommandAuditEvent
        {
            EventType = AuditEventType.CommandCompleted,
            // ... another 20 lines ...
        });

        return response;
    }
    catch (Exception ex)
    {
        stopwatch.Stop();
        await _audit.RecordEventAsync(new CommandAuditEvent
        {
            EventType = AuditEventType.CommandFailed,
            // ... more boilerplate ...
        });
        throw;
    }
}

Problems:

• ❌ ~50 lines of boilerplate per method
• ❌ Easy to forget or make mistakes
• ❌ Inconsistent across developers
• ❌ Hard to maintain and update
• ❌ Performance impact if not done right

With Interceptor (Automatic)

public override Task<Response> MyCommand(Parameters request, ServerCallContext context)
{
    // ✅ Just your business logic - auditing happens automatically!
    return Task.FromResult(DoWork(request));
}

Benefits:

• ✅ Zero boilerplate in service methods
• ✅ 100% consistent auditing across all methods
• ✅ Can't be forgotten or skipped
• ✅ Centralized configuration
• ✅ Optimized performance (batching, async)

Code Savings: For a typical SiLA2 server with 20 commands:

• Without interceptor: ~1000 lines of repetitive audit code
• With interceptor: 3 lines in Program.cs

---

Overview

SiLA2.Audit provides automatic, zero-code auditing for SiLA2 laboratory automation servers:

• ✅ Automatic Command Tracking: Every RPC call is logged (start, completion, duration, errors)
• ✅ Observable Command Lifecycle: Progress updates and intermediate responses
• ✅ Property Subscriptions: Track who's monitoring what
• ✅ Error Capture: Full stack traces and context
• ✅ Client Metadata: IP addresses, user identity, client headers
• ✅ Production Storage: SQL Server, PostgreSQL, or file-based

Core Concepts

Audit Events

All audit events inherit from AuditEvent base class and include:

• Unique event ID
• UTC timestamp
• Event type and severity
• Correlation ID for workflow tracking
• Feature identifier
• Metadata (client info, user, session, etc.)

Event Types

• CommandAuditEvent: Unobservable command executions
• ObservableCommandAuditEvent: Observable command lifecycle
• PropertyAuditEvent: Property operations
• ErrorAuditEvent: Error occurrences
• ServerLifecycleEvent: Server state changes

Service Interfaces

• IAuditService: Record audit events
• IAuditEventStore: Storage abstraction
• IAuditQueryService: Query and analyze events

Usage Examples

⚡ Quick Start (30 Seconds)

1. Install Package (when published)

dotnet add package SiLA2.Audit

2. Add to Program.cs

using SiLA2.Audit.Extensions;

var builder = WebApplication.CreateBuilder(args);

// Add auditing with automatic interception (in-memory storage for dev/testing)
builder.Services.AddAuditServiceWithInterceptor();

// Register gRPC with interceptor
builder.Services.AddGrpc(options =>
{
    options.Interceptors.Add<AuditInterceptor>();
});

var app = builder.Build();
app.MapGrpcService<MyFeatureService>();  // ✅ Automatically audited!
app.Run();

Done! Every RPC call is now automatically logged with:

• ✅ Command start/completion/failure
• ✅ Observable property subscriptions
• ✅ Client metadata (IP, user, headers)
• ✅ Request/response payloads
• ✅ Execution duration
• ✅ Error stack traces

No code changes needed in your service implementations!

---

🏢 Production Setup with SQL Server (Recommended)

For production environments, use SQL Server or PostgreSQL for robust, queryable audit logs:

using Microsoft.EntityFrameworkCore;
using SiLA2.Audit.Extensions;
using SiLA2.Audit.Storage.Database;

var builder = WebApplication.CreateBuilder(args);

// Production: SQL Server with automatic auditing
builder.Services.AddSqlAuditService(
    auditOptions =>
    {
        auditOptions.MinimumSeverity = AuditSeverity.Information;
        auditOptions.ExcludedEventTypes.Add(AuditEventType.PropertyRead);  // Too verbose
        auditOptions.EnablePiiMasking = true;  // GDPR compliance
        auditOptions.BufferSize = 500;
    },
    dbOptions =>
    {
        // SQL Server (recommended for enterprise)
        dbOptions.UseSqlServer(
            builder.Configuration.GetConnectionString("AuditDatabase"),
            sqlOptions => sqlOptions.EnableRetryOnFailure()
        );
    });

// Register interceptor for automatic auditing
builder.Services.AddAuditInterceptor();

// Add gRPC with interceptor
builder.Services.AddGrpc(options =>
{
    options.Interceptors.Add<AuditInterceptor>();
});

var app = builder.Build();

// Ensure database exists
using (var scope = app.Services.CreateScope())
{
    var dbContextFactory = scope.ServiceProvider
        .GetRequiredService<IDbContextFactory<AuditDbContext>>();
    using var dbContext = dbContextFactory.CreateDbContext();
    dbContext.Database.Migrate();  // Apply migrations
}

app.MapGrpcService<MyFeatureService>();  // ✅ All calls audited to SQL Server!
app.Run();

Connection Strings (appsettings.json):

{
  "ConnectionStrings": {
    "AuditDatabase": "Server=localhost;Database=SiLA2Audit;Trusted_Connection=True;TrustServerCertificate=True;"
  }
}

---

🐘 PostgreSQL Setup

builder.Services.AddSqlAuditService(
    auditOptions => { /* same as above */ },
    dbOptions =>
    {
        // PostgreSQL
        dbOptions.UseNpgsql(
            builder.Configuration.GetConnectionString("AuditDatabase"),
            pgOptions => pgOptions.EnableRetryOnFailure()
        );
    });

Connection String:

{
  "ConnectionStrings": {
    "AuditDatabase": "Host=localhost;Database=sila2audit;Username=postgres;Password=yourpassword"
  }
}

Add NuGet Package:

dotnet add package Npgsql.EntityFrameworkCore.PostgreSQL

---

📁 File-Based Storage (No Database Required)

Perfect for edge devices or when database isn't available:

builder.Services.AddFileAuditServiceWithInterceptor(
    auditOptions =>
    {
        auditOptions.MinimumSeverity = AuditSeverity.Information;
        auditOptions.BufferSize = 100;
    },
    fileOptions =>
    {
        fileOptions.StoragePath = "/var/log/sila2/audit";
        fileOptions.RotationPolicy = FileRotationPolicy.Daily;
        fileOptions.RetentionDays = 90;
        fileOptions.MaxFileSize = 100 * 1024 * 1024;  // 100 MB
    });

builder.Services.AddGrpc(options => options.Interceptors.Add<AuditInterceptor>());

Features:

• Daily/hourly/size-based file rotation
• Automatic old file cleanup
• JSON format (easily parseable)
• No database dependencies

---

🧪 SQLite (Development/Testing Fallback)

For local development or when SQL Server/PostgreSQL aren't available:

builder.Services.AddSqlAuditService(
    auditOptions => { /* config */ },
    dbOptions =>
    {
        // SQLite - simple file-based database
        dbOptions.UseSqlite("Data Source=audit.db");
    });

builder.Services.AddAuditInterceptor();
builder.Services.AddGrpc(options => options.Interceptors.Add<AuditInterceptor>());

Note: SQLite is great for development but consider SQL Server or PostgreSQL for production due to:

• Better concurrent write performance
• Enterprise backup/recovery
• Advanced querying capabilities
• Built-in replication/clustering

🎛️ Advanced Configuration

Selective Auditing (Reduce Noise)

builder.Services.AddAuditServiceWithInterceptor(options =>
{
    // Only log warnings and errors
    options.MinimumSeverity = AuditSeverity.Warning;

    // Exclude verbose events
    options.ExcludedEventTypes.Add(AuditEventType.PropertyRead);
    options.ExcludedEventTypes.Add(AuditEventType.ObservablePropertyValueChanged);

    // Only audit critical features
    options.IncludedFeatures.Add("org.silastandard/core/SiLAService/v1");
    options.IncludedFeatures.Add("com.mycompany/devices/CriticalDevice/v1");
});

PII Masking for GDPR Compliance

builder.Services.AddSqlAuditService(
    auditOptions =>
    {
        // Enable PII masking
        auditOptions.EnablePiiMasking = true;

        // Customize masked fields
        auditOptions.PiiMaskingFields.Clear();
        auditOptions.PiiMaskingFields.Add("UserId");
        auditOptions.PiiMaskingFields.Add("RemoteEndpoint");
        auditOptions.PiiMaskingFields.Add("SessionId");
    },
    dbOptions => dbOptions.UseSqlServer(connectionString));

Result: user@example.com → us****om, 192.168.1.100 → 19****00

---

Basic Setup (In-Memory Storage)

Perfect for development and testing:

// Program.cs
using SiLA2.Audit.Extensions;

var builder = WebApplication.CreateBuilder(args);

// Register audit service with in-memory storage
builder.Services.AddAuditService(options =>
{
    options.MinimumSeverity = AuditSeverity.Information;
    options.BufferSize = 100;
    options.FlushInterval = TimeSpan.FromSeconds(5);
});

var app = builder.Build();
app.Run();

File-Based Storage

For persistent audit logs:

// Program.cs
using SiLA2.Audit.Extensions;

var builder = WebApplication.CreateBuilder(args);

// Register audit service with file-based storage
builder.Services.AddFileAuditService(
    auditOptions =>
    {
        auditOptions.MinimumSeverity = AuditSeverity.Information;
        auditOptions.BufferSize = 100;
    },
    fileOptions =>
    {
        fileOptions.StoragePath = "/var/log/sila2/audit";
        fileOptions.RotationPolicy = FileRotationPolicy.Daily;
        fileOptions.RetentionDays = 90;
    });

var app = builder.Build();
app.Run();

Recording Audit Events in Services

using SiLA2.Audit.Core;
using SiLA2.Audit.Models;
using System.Text.Json;

public class MyFeatureService : MyFeature.MyFeatureBase
{
    private readonly Feature _siLA2Feature;
    private readonly IAuditService _auditService;

    public MyFeatureService(ISiLA2Server siLA2Server, IAuditService auditService)
    {
        _siLA2Feature = siLA2Server.ReadFeature(
            Path.Combine("Features", "MyFeature-v1_0.sila.xml"));
        _auditService = auditService;
    }

    public override async Task<Response> MyCommand(
        Parameters request,
        ServerCallContext context)
    {
        var correlationId = Guid.NewGuid().ToString();
        var stopwatch = System.Diagnostics.Stopwatch.StartNew();

        // Audit command start
        await _auditService.RecordEventAsync(new CommandAuditEvent
        {
            EventType = AuditEventType.CommandStarted,
            CommandIdentifier = "MyCommand",
            ParametersJson = JsonSerializer.Serialize(request),
            CorrelationId = correlationId,
            FeatureIdentifier = _siLA2Feature.FeatureIdentifier,
            Metadata = new AuditMetadata
            {
                ClientId = context.GetHttpContext()?.Connection.RemoteIpAddress?.ToString(),
                UserId = context.GetHttpContext()?.User?.Identity?.Name
            }
        });

        try
        {
            // Execute command logic
            var response = DoWork(request);

            stopwatch.Stop();

            // Audit command completion
            await _auditService.RecordEventAsync(new CommandAuditEvent
            {
                EventType = AuditEventType.CommandCompleted,
                CommandIdentifier = "MyCommand",
                ResponseJson = JsonSerializer.Serialize(response),
                CorrelationId = correlationId,
                FeatureIdentifier = _siLA2Feature.FeatureIdentifier,
                DurationMs = stopwatch.ElapsedMilliseconds
            });

            return response;
        }
        catch (Exception ex)
        {
            stopwatch.Stop();

            // Audit command failure
            await _auditService.RecordEventAsync(new CommandAuditEvent
            {
                EventType = AuditEventType.CommandFailed,
                CommandIdentifier = "MyCommand",
                ErrorMessage = ex.Message,
                StackTrace = ex.StackTrace,
                CorrelationId = correlationId,
                FeatureIdentifier = _siLA2Feature.FeatureIdentifier,
                DurationMs = stopwatch.ElapsedMilliseconds,
                Severity = AuditSeverity.Error
            });

            throw;
        }
    }
}

Advanced Configuration

Event Filtering

builder.Services.AddAuditService(options =>
{
    // Only audit warnings and errors
    options.MinimumSeverity = AuditSeverity.Warning;

    // Exclude verbose property reads
    options.ExcludedEventTypes.Add(AuditEventType.PropertyRead);
    options.ExcludedEventTypes.Add(AuditEventType.ObservablePropertyValueChanged);

    // Only audit specific features
    options.IncludedFeatures.Add("org.silastandard/core/SiLAService/v1");
    options.IncludedFeatures.Add("de.cetoni/controllers/TemperatureController/v1");
});

PII Masking

builder.Services.AddAuditService(options =>
{
    // Enable PII masking for GDPR compliance
    options.EnablePiiMasking = true;

    // Customize which fields to mask
    options.PiiMaskingFields.Add("UserId");
    options.PiiMaskingFields.Add("ClientId");
    options.PiiMaskingFields.Add("RemoteEndpoint");
    options.PiiMaskingFields.Add("SessionId");
});

Querying Audit Events

// Inject IAuditEventStore to query events
public class AuditAnalysisService
{
    private readonly IAuditEventStore _auditStore;

    public AuditAnalysisService(IAuditEventStore auditStore)
    {
        _auditStore = auditStore;
    }

    public async Task<IEnumerable<AuditEvent>> GetWorkflowEvents(string correlationId)
    {
        return await _auditStore.GetEventsByCorrelationIdAsync(correlationId);
    }

    public async Task<IEnumerable<AuditEvent>> GetTodaysEvents()
    {
        var today = DateTime.UtcNow.Date;
        return await _auditStore.GetEventsByTimeRangeAsync(
            today,
            today.AddDays(1));
    }

    public async Task<IEnumerable<AuditEvent>> GetFailedCommands()
    {
        return await _auditStore.GetEventsByTypeAsync(
            AuditEventType.CommandFailed);
    }
}

Storage Backend Comparison

Choose the right storage backend for your needs:

Feature	SQL Server	PostgreSQL	File-Based	SQLite	In-Memory
Production Ready	✅ Best	✅ Best	✅ Good	⚠️ Dev Only	❌ Testing Only
Concurrent Writes	✅ Excellent	✅ Excellent	⚠️ Limited	⚠️ Limited	✅ Excellent
Query Performance	✅ Fast	✅ Fast	⚠️ Slow	⚠️ Slow	✅ Very Fast
Data Persistence	✅ Yes	✅ Yes	✅ Yes	✅ Yes	❌ Lost on restart
Backup/Recovery	✅ Enterprise	✅ Enterprise	⚠️ Manual	⚠️ Manual	❌ N/A
Scalability	✅ High	✅ High	⚠️ Limited	❌ Low	⚠️ Memory Limited
Setup Complexity	Medium	Medium	✅ Simple	✅ Simple	✅ None
External Dependencies	Database	Database	None	None	None
Best For	Enterprise	Enterprise	Edge Devices	Development	Unit Tests

Recommendations:

• 🏢 Enterprise/Production: SQL Server or PostgreSQL
• 📦 Edge Devices/No DB: File-based storage
• 🧪 Development: SQLite (quick setup) or In-Memory (fastest)
• ✅ Testing: In-Memory (fastest, no cleanup needed)

---

Database Setup

SQL Server

1. Install EF Core Tools:

dotnet tool install --global dotnet-ef

2. Create Migration:

cd src/SiLA2.Audit
dotnet ef migrations add InitialCreate --context AuditDbContext --output-dir Migrations

3. Apply Migration:

// In Program.cs startup
using var scope = app.Services.CreateScope();
var dbContextFactory = scope.ServiceProvider
    .GetRequiredService<IDbContextFactory<AuditDbContext>>();
using var dbContext = dbContextFactory.CreateDbContext();
dbContext.Database.Migrate();

Or use EF CLI:

dotnet ef database update --context AuditDbContext

PostgreSQL

1. Add NuGet Package:

dotnet add package Npgsql.EntityFrameworkCore.PostgreSQL

2. Configure in Program.cs:

builder.Services.AddSqlAuditService(
    auditOptions => { /* config */ },
    dbOptions => dbOptions.UseNpgsql(connectionString)
);

3. Create Database (psql):

CREATE DATABASE sila2audit;
CREATE USER sila2user WITH PASSWORD 'yourpassword';
GRANT ALL PRIVILEGES ON DATABASE sila2audit TO sila2user;

4. Apply Migrations:

dotnet ef migrations add InitialCreate --context AuditDbContext
dotnet ef database update --context AuditDbContext

---

Configuration Options

All configuration is done through AuditOptions:

builder.Services.AddAuditService(options =>
{
    // Enable/disable auditing globally
    options.Enabled = true;

    // Minimum severity to record
    options.MinimumSeverity = AuditSeverity.Information;

    // Include only specific event types
    options.IncludedEventTypes.Add(AuditEventType.CommandStarted);
    options.IncludedEventTypes.Add(AuditEventType.CommandFailed);

    // Exclude verbose event types
    options.ExcludedEventTypes.Add(AuditEventType.PropertyRead);

    // Include only specific features
    options.IncludedFeatures.Add("org.silastandard/core/SiLAService/v1");

    // Exclude features
    options.ExcludedFeatures.Add("com.test/debug/DebugFeature/v1");

    // Buffering settings
    options.BufferSize = 100;                          // Events before auto-flush
    options.FlushInterval = TimeSpan.FromSeconds(5);   // Max time before flush

    // Retention
    options.RetentionPeriod = TimeSpan.FromDays(90);   // Auto-delete after 90 days

    // PII Masking
    options.EnablePiiMasking = true;
    options.PiiMaskingFields.Add("UserId");
    options.PiiMaskingFields.Add("ClientId");
    options.PiiMaskingFields.Add("RemoteEndpoint");
});

File Storage Options:

builder.Services.AddFileAuditStore(fileOptions =>
{
    fileOptions.StoragePath = "/var/log/sila2/audit";
    fileOptions.RotationPolicy = FileRotationPolicy.Daily;  // or Hourly, Size
    fileOptions.MaxFileSize = 100 * 1024 * 1024;  // 100 MB (for Size rotation)
    fileOptions.RetentionDays = 90;
    fileOptions.EnableIndexing = true;  // Fast queries
    fileOptions.WriteBatchSize = 100;
});

Current Capabilities

✅ Production Ready - The audit system is fully functional with:

• Zero-Code Auditing: Automatic capture of all gRPC calls
• Multiple Storage Backends: In-memory, file-based, or SQL database
• Intelligent Filtering: By severity, event type, and feature
• PII Protection: Configurable field masking for compliance
• Performance Optimized: < 1ms overhead with buffering
• Thread-Safe: Concurrent operations fully supported
• Resilient: Failed writes are re-queued automatically