Nefarius.Utilities.ETW
3.1.0
Prefix Reserved
dotnet add package Nefarius.Utilities.ETW --version 3.1.0
NuGet\Install-Package Nefarius.Utilities.ETW -Version 3.1.0
<PackageReference Include="Nefarius.Utilities.ETW" Version="3.1.0" />
<PackageVersion Include="Nefarius.Utilities.ETW" Version="3.1.0" />
<PackageReference Include="Nefarius.Utilities.ETW" />
paket add Nefarius.Utilities.ETW --version 3.1.0
#r "nuget: Nefarius.Utilities.ETW, 3.1.0"
#:package Nefarius.Utilities.ETW@3.1.0
#addin nuget:?package=Nefarius.Utilities.ETW&version=3.1.0
#tool nuget:?package=Nefarius.Utilities.ETW&version=3.1.0
Nefarius.Utilities.ETW
ETW Log Files (.ETL) to JSON parser/converter library.
This is a fork of the fantastic ETW2JSON project by Microsoft and
contributors.
While this library is designed to support universal ETW trace sources, the primary personal goal and highest development priority has been making it work reliably with WPP Software Tracing in particular. Other trace source types may work but receive less focused attention and testing.
Changes of this fork
- Converted console tool into a reusable class library
- Replaced P/Invoke code with source generators
- Changed namespace to
Nefarius.Utilities.ETWto avoid conflicts with the origin library - Added support for WPP Software Tracing decoding
- Supports
.PDBfiles as a decoding source — WPP provider GUIDs (ETW trace control GUIDs) are automatically extracted fromTMC:annotations in the PDB symbol stream so no manual GUID lookup is required - Supports
.TMFfiles as a decoding source - Support for the WPP extended format specification strings
(
%!FUNC!,%!LEVEL!,%!FLAGS!,%!IPADDR!,%!TIMESTAMP!,%!delta!,%!due!,%!GUID!,%!CLSID!/%!LIBID!/%!IID!,%!PORT!,%!STATUS!,%!WINERROR!,%!HRESULT!,%!NDIS_STATUS!,%!NDIS_OID!,%!sid!, bitset/list enumerations, and more); known exceptions are%!CLSID!(no registry name lookup) and%!LINE!(rendered empty) (see Known limitations) USEPREFIX/USESUFFIXtrace message prefixes are automatically expanded — the%0standard-prefix sentinel and%!FUNC!/%!LEVEL!context markers are resolved from the TMF metadata at decode time
- Supports
- Added
EtwUtil.EnumeratePdbReferencesfor lightweight pre-scanning of ETL files to collect all PDB metadata (symbol GUIDs, ages and file names) referenced in the trace before performing a full decode — enabling proper multi-PDB symbol resolution via symbol servers or local paths - Added
EtwUtil.EnumerateEventsAsync— a streamingIAsyncEnumerable<ReadOnlyMemory<byte>>API that yields each decoded ETW event as a self-contained UTF-8 JSON buffer as it is produced, rather than waiting for the full trace to finish; a dedicated background thread runs the blockingProcessTracecall and feeds a bounded channel so the consumer is naturally backpressured and can process events concurrently with parsing; works well as a data source for real-time delivery scenarios such as FastEndpoints Server Sent Events:
public override async Task HandleAsync(CancellationToken ct)
{
await Send.EventStreamAsync("etw-event", GetEtwStream(ct), ct);
}
private async IAsyncEnumerable<object> GetEtwStream([EnumeratorCancellation] CancellationToken ct)
{
await foreach (ReadOnlyMemory<byte> eventJson in EtwUtil.EnumerateEventsAsync(
[@"C:\traces\capture.etl"],
opts => opts.WppDecodingContext = myDecodingContext,
ct))
{
yield return JsonSerializer.Deserialize<object>(eventJson.Span)!;
}
}
- Added a managed, cross-platform
.etlreader (EtwBackend):EtwUtil.ConvertToJson,EnumerateEventsAsyncandEnumeratePdbReferencesnow also run on Linux/macOS, including WPP decoding and TraceLogging events (custom-serialized fields use their base in-type unless a TraceLoggingCustomDecoders decoder is registered; both backends decode TraceLogging from the embedded schema) - Added realtime ETW session support via
EtwRealtimeSessionandEtwUtil.EnumerateRealtimeEventsAsync:EtwRealtimeSession.Create(name)starts a user-mode ETW session (StartTraceW)EnableProvider/DisableProvidertoggle providers at runtime viaEnableTraceEx2EtwUtil.EnumerateRealtimeEventsAsyncattaches to any externally-managed session and streams decoded events as UTF-8 JSON using the same bounded-channel / background-thread architecture as the file-basedEnumerateEventsAsyncEtwUtil.ConvertRealtimeToJsonblocks and writes events to aUtf8JsonWriteruntil cancelledEtwUtil.StopOrphanSessioncleanly stops a session left behind by a previous crash
Realtime decoding
The library ships two independent layers for realtime ETW consumption.
Layer 1 — attach to any existing session (consumer-only)
If the session is already running (started by logman, xperf, another process, or your own code), attach directly with a session name:
using CancellationTokenSource cts = new();
// Cancel after Ctrl+C or when your app shuts down.
Console.CancelKeyPress += (_, e) => { e.Cancel = true; cts.Cancel(); };
await foreach (ReadOnlyMemory<byte> eventJson in EtwUtil.EnumerateRealtimeEventsAsync(
"MySession",
cancellationToken: cts.Token))
{
Console.WriteLine(System.Text.Encoding.UTF8.GetString(eventJson.Span));
}
Layer 2 — full session management (start + enable + stop)
Use EtwRealtimeSession when your application owns the session lifetime.
Requires administrator / SeSystemProfilePrivilege.
// Clean up any orphan from a previous crash.
EtwUtil.StopOrphanSession("MyApp-Live");
using EtwRealtimeSession session = EtwRealtimeSession.Create("MyApp-Live",
opts =>
{
opts.BufferSizeKb = 64;
opts.FlushTimerSeconds = 1;
opts.ClockResolution = EtwClockResolution.QueryPerformanceCounter;
});
// Enable one or more providers at the desired verbosity level.
Guid microsoftWindowsKernelProcess = new("22FB2CD6-0E7B-422B-A0C7-2FAD1FD0E716");
session.EnableProvider(microsoftWindowsKernelProcess, TraceEventLevel.Information);
using CancellationTokenSource cts = new();
Console.CancelKeyPress += (_, e) => { e.Cancel = true; cts.Cancel(); };
await foreach (ReadOnlyMemory<byte> eventJson in EtwUtil.EnumerateRealtimeEventsAsync(
session.SessionName,
cancellationToken: cts.Token))
{
Console.WriteLine(System.Text.Encoding.UTF8.GetString(eventJson.Span));
}
// Disposing session calls ControlTraceW(STOP) — no orphan left behind.
Realtime caveats
- Admin required to start a session.
EtwRealtimeSession.CreatethrowsEtwStartTraceExceptionwithERROR_ACCESS_DENIEDwhen the process is not elevated. - Sessions persist after process exit. Always dispose
EtwRealtimeSession, and callEtwUtil.StopOrphanSession("name")at startup as a safety net. - WPP decoding in realtime mode requires a pre-built
DecodingContext. The file-basedEnumeratePdbReferencespre-scan only works on.etlfiles, not on live sessions. Build theDecodingContextfrom known.pdbor.tmfpaths before starting the session. - Kernel providers:
EnumerateRealtimeEventsAsyncattaches by name to an existingNT Kernel Loggersession (verified against an externally created one).EtwRealtimeSession.Createcan also create kernel sessions: setEtwRealtimeSessionOptions.SystemLoggerand enableEtwSystemProvidersGUIDs (system logger mode is available from Windows 8, enabling the system provider GUIDs requires Windows 11+), or create a session namedEtwRealtimeSession.NtKernelLoggerNamewithEnableFlags(the single system-wide classic logger).
Symbol resolution
Nefarius.Utilities.ETW.Symbols loads .pdb and .tmf inputs and resolves PDB identities without shelling out to the CLI:
SymbolStoreConfiguration store = SymbolStoreConfiguration.Resolve(
configuredServer: null,
configuredCache: null,
ntSymbolPath: Environment.GetEnvironmentVariable("_NT_SYMBOL_PATH"),
defaultCacheDirectory: Path.Combine(Path.GetTempPath(), "symbols"));
DecodingSourceSet explicitSources = DecodingSources.Load([@"D:\symbols\driver.pdb", @"D:\tmf"]);
TraceSymbolResolution resolved = await DecodingSources.ResolveTraceSymbolsAsync(
[@"C:\traces\capture.etl"],
store);
PortableExecutableSymbols.TryReadCodeViewPdb(@"C:\Windows\System32\drivers\example.sys");
Search order is the local symbol cache, then configured directories or filename globs whose PDB GUID and age match, then the HTTP symbol server. srv*cache*url, srv*url, and cache*dir segments of _NT_SYMBOL_PATH are recognized.
CLI (etwutils)
The tools/Nefarius.Utilities.ETW.CLI project builds a .NET global tool (etwutils) that wraps the realtime API and writes decoded events as NDJSON or plain tab-separated text to stdout.
dotnet tool install -g Nefarius.Utilities.ETW.CLI
See tools/Nefarius.Utilities.ETW.CLI/README.md for full installation instructions, command reference, and examples.
Raw ETW relay and OpenTelemetry gateway
Capture clients that must not hold .pdb/.tmf files can forward raw EVENT_RECORDs to a trusted server that decodes them (WPP included) with this library. The wire format is language-neutral and easy to produce from C/C++.
await foreach (ReadOnlyMemory<byte> json in EtwUtil.EnumerateRelayEventsAsync(authenticatedBody,
o => o.WppDecodingContext = new DecodingContext(PdbFileDecodingContextType.CreateFrom("server-only.pdb"))))
{
}
An optional OpenTelemetry adapter and a standalone decoding gateway (OTLP in, decoded standard OTLP Logs out) are included; OpenTelemetry is not required by the core package.
To try the gateway without writing code, etwutils realtime --format relay captures without symbols and the optional etwrelay global tool (dotnet tool install -g Nefarius.Utilities.ETW.Relay.CLI) delivers the frames as OTLP/HTTP:
etwutils realtime --format relay <provider-guid> | etwrelay --endpoint http://localhost:4318
This binary pipe needs PowerShell 7.4+, cmd.exe or a Unix shell. On older PowerShell versions use --format relay-ndjson with etwrelay --input-format base64-ndjson. See tools/Nefarius.Utilities.ETW.Relay.CLI.
- Raw ETW relay: wire format, C/C++ producer, limits and hosting contract
- OpenTelemetry integration and decoding gateway
Known limitations
- Reading
.etlfiles works on Windows, Linux and macOS through a managed reader (EtwBackend.Managed, the default off Windows; setEtwJsonConverterOptions.Backend/EtwMetadataScanOptions.Backendto choose). Real-time sessions and session management remain Windows-only. Manifest-described events decode on every platform, including structs, fixed and variable array counts, fixed and variable lengths, declared out-types and value/bit maps (maps are kept in the property metadata; JSON output keeps the raw values) and throwPlatformNotSupportedExceptionelsewhere - The managed reader reads compressed (
EVENT_TRACE_COMPRESSED_MODE, Xpress LZ77).etlfiles by decompressing every buffer into memory first, and covers the record types found in WPP, kernel and TraceLogging/manifest traces; unsupported record types are skipped with a warning viaReportError %!ItemEnum!/%!ItemFlagsEnum!types are shown with enumerator names when the enum type (named in the TMF asItemEnum(Name)) is found in a PDB type stream; from TMF files, or when the type is missing, the raw value is shown. Without aWppDecodingContext, WPP events are still emitted with the "No format information found." message andOnWppFormatMissingfires%!CLSID!prints the GUID without a registry name lookup and%!LINE!is rendered as an empty string- Relayed records are never decoded with TDH, so manifest-described events need a custom manifest on the server; raw (QPC) timestamps are not supported by the relay, and one relay decoder or session must be driven by a single thread at a time
- The CLI does not create kernel sessions; use the library API (
SystemLogger/NtKernelLoggerName) or attach to an existing session by name
Documentation
Sources & 3rd party credits
- Microsoft/ETW2JSON
- Microsoft/ETW
- WPP Software Tracing
- microsoftarchive/bcl/Tools/ETW/traceEvent/SymbolEventParser.cs
enum _TDH_CONTEXT_TYPE- Learn / Windows / Windows Drivers / How do I add a prefix and suffix to a trace message?
- Learn / Windows / Windows Drivers / What are the WPP extended format specification strings
- kaitai-pdb
- MinVer
- Fast access to .net fields/properties
- Nefarius.Shared.PdbUtils
- Information from Microsoft about the PDB format
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net8.0 is compatible. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. net8.0-windows8.0 is compatible. net9.0 is compatible. net9.0-android was computed. net9.0-browser was computed. net9.0-ios was computed. net9.0-maccatalyst was computed. net9.0-macos was computed. net9.0-tvos was computed. net9.0-windows was computed. net9.0-windows8.0 is compatible. net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. net10.0-windows8.0 is compatible. |
-
net10.0
- FastMember (>= 1.5.0)
- KaitaiStruct.Runtime.CSharp (>= 0.11.0)
- Nefarius.Shared.PdbUtils (>= 1.0.0)
-
net10.0-windows8.0
- FastMember (>= 1.5.0)
- KaitaiStruct.Runtime.CSharp (>= 0.11.0)
- Nefarius.Shared.PdbUtils (>= 1.0.0)
-
net8.0
- FastMember (>= 1.5.0)
- KaitaiStruct.Runtime.CSharp (>= 0.11.0)
- Nefarius.Shared.PdbUtils (>= 1.0.0)
-
net8.0-windows8.0
- FastMember (>= 1.5.0)
- KaitaiStruct.Runtime.CSharp (>= 0.11.0)
- Nefarius.Shared.PdbUtils (>= 1.0.0)
-
net9.0
- FastMember (>= 1.5.0)
- KaitaiStruct.Runtime.CSharp (>= 0.11.0)
- Nefarius.Shared.PdbUtils (>= 1.0.0)
-
net9.0-windows8.0
- FastMember (>= 1.5.0)
- KaitaiStruct.Runtime.CSharp (>= 0.11.0)
- Nefarius.Shared.PdbUtils (>= 1.0.0)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.
| Version | Downloads | Last Updated |
|---|---|---|
| 3.1.0 | 83 | 11/17/2026 |
| 3.0.0 | 86 | 11/15/2026 |
| 2.20.1 | 260 | 11/11/2026 |
| 2.20.0 | 576 | 11/2/2026 |
| 2.19.0 | 561 | 10/11/2026 |
| 2.18.0 | 248 | 8/5/2026 |
| 2.17.1 | 140 | 7/21/2026 |
| 2.17.0 | 163 | 6/16/2026 |
| 2.16.0 | 145 | 6/16/2026 |
| 2.15.1 | 140 | 6/1/2026 |
| 2.15.0 | 140 | 6/1/2026 |
| 2.14.2 | 136 | 5/31/2026 |
| 2.14.1 | 147 | 5/31/2026 |
| 2.14.0 | 140 | 5/30/2026 |
| 2.13.0 | 142 | 5/30/2026 |
| 2.12.0 | 155 | 5/29/2026 |
| 2.11.0 | 147 | 5/28/2026 |
| 2.10.2 | 168 | 5/28/2026 |
| 2.10.1 | 142 | 5/28/2026 |
| 2.10.0 | 157 | 5/28/2026 |