Serilog.Sinks.InfluxDB.Syslog 2.3.5

Serilog.Sinks.InfluxDB.Syslog

A serilog sink that writes events to InfluxDB in syslog message format (by default) as described on the Influx blog. The exact fields, tags and even measurement names can be customised via the sink configuration, as outlined below. Supports platforms compatible with the .NET Platform Standard netstandard2.0.

Compatible only with InfluxDB v2.0 and upwards

Warning: use for InfluxDB v1.X following nuget package :

for V1 see Get Started for V1.X

Getting Started

InfluxDB v2.X

To get started install the Serilog.Sinks.InfluxDB.Syslog package:

PM> Install-Package Serilog.Sinks.InfluxDB.Syslog

OR

$ dotnet add package Serilog.Sinks.InfluxDB.Syslog

If running locally for development purpose, you can use docker-compose-v2.yml at root of this repository and adapt volumes if needed

$ docker-compose -f docker-compose-v2.yml up -d

Point the logger to InfluxDb (quickest way using default _internal database):

Log.Logger = new LoggerConfiguration()
    .WriteTo.InfluxDB(applicationName: "Quick test",
                    uri: new Uri("http://127.0.0.1:8086"),
                    organizationId: "88e1f5a5ad074d9e",  // Organization Id - unique id can be found under Profile > About > Common Ids)
                    bucketName: "logs",
                    token: "bGfBKhSycNiUOia4k7peib2jHFewkz3o6Hv2uz1xAoUcdnEFRW7cHn03KICySLemA4VPZKvc0CwzSQT8GNl2DA==")
    .CreateLogger();

Another sample using InfluxDBSinkOptions for more control over periodic batching options and connection information:

Log.Logger = new LoggerConfiguration()
    .WriteTo.InfluxDB(new InfluxDBSinkOptions()
    {
        ApplicationName = "fluentSample",
        InstanceName = "fluentSampleInstance",
        ConnectionInfo = new InfluxDBConnectionInfo()
        {
            Uri = new Uri("http://127.0.0.1:8086"),
            BucketName = "logs",
            OrganizationId = "88e1f5a5ad074d9e",  // Organization Id - unique id can be found under Profile > About > Common Ids
            // To be set if bucket already created and give write permission and set CreateBucketIfNotExists to false
            Token = null,
            CreateBucketIfNotExists = true,
            //To specify if Bucket needs to be created and if token not known or without all access permissions
            AllAccessToken = "bGfBKhSycNiUOia4k7peib2jHFewkz3o6Hv2uz1xAoUcdnEFRW7cHn03KICySLemA4VPZKvc0CwzSQT8GNl2DA==",
            BucketRetentionPeriod = TimeSpan.FromDays(1)
        },
        BatchOptions = new PeriodicBatching.PeriodicBatchingSinkOptions()
        {
            BatchSizeLimit = 50,
            Period = TimeSpan.FromSeconds(10),
            EagerlyEmitFirstEvent = true,
            QueueLimit = null
        }
    })
    .CreateLogger();

If using appsettings.json for configuration the following example illustrates using InfluxDb and Console sinks.

{
    "Serilog": {
        "Using": ["Serilog.Sinks.Console", "Serilog.Sinks.InfluxDB.Syslog"],
        "MinimumLevel": {
          "Default": "Information",
          "Override": {
            "Microsoft": "Warning",
            "System": "Warning"
          }
        },
        "WriteTo:Influx": {
          "Name": "InfluxDB",
          "Args": {
            "sinkOptions": {
              "ApplicationName": "testApp",
              "InstanceName": "testInstance",
              "ConnectionInfo": {
                "Uri": "http://localhost:8086",
                "BucketName": "logs",
                "OrganizationId": "88e1f5a5ad074d9e",
                "Token": "edBlcWgLkoPOituD_6V1ftCznpDR8niFcF46MJCSYuSxc1FM_srm9cuoc84yX5kOjOH_11Zvxk_juqr44S-57A==",
                "CreateBucketIfNotExists": false 
                //"Username": "influxdbroot",
                //"Password": "TBD"
                "BucketRetentionPeriod": "7.00:00:00",
              },
              "BatchOptions": {
                "EagerlyEmitFirstEvent": true,
                "BatchSizeLimit": 100,
                "Period": "0.00:00:30",
                "QueueLimit": 1000000
              }
            }
          }
        },
        "Properties": {
            "Application": "Serilog Sink InfluxDb Console Sample"
        }
    }
}

All those samples can be found under project subdirectory samples of this repository.

Optional parameters

"sinkOptions": {
  "MeasurementName": "syslog",
  "ApplicationName": "testApp",
  "InstanceName": "testInstance",
  "IncludeFullException": true,
  "IncludeHostname": false,
  "IncludeLevel": true,
  "IncludeSeverity": true,
  "IncludeDefaultFields": true,
  "ExtendedFields": ["ReleaseNumber"],
  "ExtendedTags": [],
  ...
}

Non-syslog format support

Since initial creation of this plugin, InfluxDB now supports a more flexible logging approach that eliminates the strict requirement of the syslog format. This means it's possible to create leaner logging payloads without the additional syslog-specific entries with a configuration such as the following:

"sinkOptions": {
  "MeasurementName": "mymeasurement",
  "IncludeHostname": false,
  "IncludeLevel": false,
  "IncludeSeverity": false,
  "IncludeDefaultFields": false,
  ...
}

This configuration will produce a log entry with only a message and time:

[
  {
    Time: DateTime_1,
    Field: message,
    Value: Some warning "Some parameter",
    Tags: {}
  }
]

InfluxDB v1.X

**For InfluxDB v1.X use following nuget package 😗*

PM> Install-Package Serilog.Sinks.InfluxDBV1.Syslog -Version 1.3.1

OR

$ dotnet add package Serilog.Sinks.InfluxDBV1.Syslog --version 1.3.1

If running locally for development purpose, you can use docker-compose.yml at root of this repository and adapt volumes if needed

$ docker-compose -f docker-compose.yml up -d

Point the logger to InfluxDb (quickest way using default _internal database):

Log.Logger = new LoggerConfiguration()    
    .WriteTo.InfluxDB(
        applicationName: "Quick Test", 
        uri : new Uri("http://127.0.0.1:8086"));

Another sample using InfluxDBSinkOptions for more control over periodic batching options and connection information:

Log.Logger = new LoggerConfiguration()
    .WriteTo.InfluxDB(new InfluxDBSinkOptions()
    {
        ApplicationName = "fluentSample",               // Application Name
        InstanceName = "fluentSampleInstance",          // Instance or Environment Name
        ConnectionInfo = new InfluxDBConnectionInfo()   // Connection Details
        {
            Uri = new Uri("http://127.0.0.1:8086"),
            DbName = "_internal",
        },
        BatchOptions = new PeriodicBatching.PeriodicBatchingSinkOptions()
        {
            BatchSizeLimit = 50,
            Period = TimeSpan.FromSeconds(10),
            EagerlyEmitFirstEvent = true,
            QueueLimit = null
        }
    })
    .CreateLogger();

If using appsettings.json for configuration the following example illustrates using InfluxDb and Console sinks.

{
    "Serilog": {
        "Using": ["Serilog.Sinks.Console", "Serilog.Sinks.InfluxDBV1.Syslog"],
        "MinimumLevel": {
          "Default": "Information",
          "Override": {
            "Microsoft": "Warning",
            "System": "Warning"
          }
        },
        "WriteTo": [
          { "Name": "Console" },
          {
            "Name": "InfluxDB",
            "Args": {
              "sinkOptions": {
                "applicationName": "testApp",
                "instanceName": "testInstance",
                "ConnectionInfo": {
                  "Uri": "http://localhost:8086",
                  "DbName": "_internal",
                  "Username": "",
                  "Password": ""
                },
                "BatchOptions": {
                  "EagerlyEmitFirstEvent": true,
                  "BatchSizeLimit": 200,
                  "Period": "0.00:00:30",
                  "QueueLimit":  null
                }
              }
            }
          }
        ],
        "Properties": {
            "Application": "Serilog Sink InfluxDb Console Sample"
        }
    }
}

Build Status

Branch	Status
Main Branch

Benchmarks

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19042

Intel Core i7-2640M CPU 2.80GHz (Sandy Bridge), 1 CPU, 4 logical and 2 physical cores

.NET Core SDK=5.0.101 [Host] : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT [AttachedDebugger]
DefaultJob : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT

Troubleshooting

Nothing showed up, what can I do?

If events don't appear in InfluxDb after looking in corresponding database via Chronograf, Grafana or else. Either your application was unable to contact the InfluxDb server, or else the InfluxDb server rejected the log events for some reason.

Server-side issues

The InfluxDb server may reject incoming events if they're missing required credentials (check troubleshoot articles on influxdb, if the payload is corrupted somehow, or if the log events are too large to accept.

Client-side issues

If there's no information in the ingestion log, the application was probably unable to reach the server because of network configuration or connectivity issues. These are reported to the application through Serilog's SelfLog.

Add the following line after the logger is configured to print any error information to the console:

Serilog.Debugging.SelfLog.Enable(Console.Error);

If the console is not available, you can pass a delegate into SelfLog.Enable() that will be called with each error message:

Serilog.Debugging.SelfLog.Enable(message => {
    // Do something with `message`
});

Troubleshooting checklist

• Check InfluxDb connectivity and if Server-side issues see section above
• Turn on the Serilog SelfLog as described above to check for connectivity problems and other issues on the client side.
• Make sure your application calls Log.CloseAndFlush(), or disposes the root Logger, before it exits - otherwise, buffered events may be lost.
• If your app is a Windows console application, it is also important to close the console window by exiting the app; Windows console apps are terminated "hard" if the close button in the title bar is used, so events buffered for sending to InfluxDb may be lost if you use it.