MidiSharp.OwnAudio 1.0.0

MidiSharp

A cross-platform, pure-managed C# software synthesizer that renders MIDI files through SoundFont presets — SF2, SF3, SFZ, and DLS, all loaded into one common in-memory representation (the SoundBank IR). Built around a fully spec-compliant MIDI 1.0 / GM / GM2 / GS / XG protocol implementation and an SF2-style generator/modulator engine validated against fluidsynth.

It also ships a patch-level instrument-substitution layer: list the instruments a song calls for and swap any of them for an instrument cherry-picked from another font, without altering the song's sequencing.

On top of that sits a per-part mixer + effects layer: each MIDI track gets its own fader, pan, sends, mute/solo and a drag-and-drop insert rack, plus a master bus — all driven by MidiSharp.Dsp, a small decoupled DSP library (EQ, brickwall limiter, gain, chainable processors) the host wires in caller-side. The browser player is a live mixing console on top of this.

And it can host external audio plugins — CLAP, VST2, VST3, Audio Units (macOS), and LADSPA — as both effects (in any insert rack, beside the built-in DSP) and instruments (an alternative sound source to the synth). Plugins are discovered and run out-of-process, so a crashing plugin can't take the host down, and their native editor windows open on Linux (X11), Windows (Win32), and macOS (Cocoa).

Quick start

# Live playback (use single quotes around files with ! or spaces)
dotnet run --project samples/MidiSharp.Demo -- '<song>.mid' '<base>.sf2'
# Controls: Space pause/resume, S reset, Q quit

# Render to WAV instead of live playback
dotnet run --project samples/MidiSharp.Demo -- '<song>.mid' '<base>.sf2' /tmp/out.wav

# List the patches (instruments) a song uses, named against a base font
dotnet run --project samples/MidiSharp.Demo -- --patches '<song>.mid' '<base>.sf2'

# Override patches with instruments from other fonts (repeatable), then play or render
dotnet run --project samples/MidiSharp.Demo -- '<song>.mid' '<base>.sf2' \
  --map 30=OtherGM.sf2            # program 30 ← OtherGM's program 30 (GM-aligned)
  # --map 30=Guitars.sf2:5       program 30 ← Guitars' program 5
  # --map 128:0=OtherDrums.sf2   swap the whole drum kit (bank 128)

A browser-based player is also included:

dotnet run --project samples/MidiSharp.Server -- --sf-root ~/soundfonts --midi-root ~/midi
# then open http://localhost:5005 — a live mixing console: one channel strip per MIDI track
# (source-font substitution, fader/pan/sends/mute/solo, and a drag-and-drop EQ/limiter insert
# rack), plus a master strip. Saveable per-song setups. Audio plays on the server machine.

Tested daily against GeneralUser-GS.sf2 and TyrolandSFX.sf2. End-to-end A/B vs fluidsynth on real GS-heavy content stays within ±3.5 dB worst case across the entire spectrum, with bass essentially identical (±1 dB).

Project layout

MidiSharp.slnx                       Root solution file
├── src/
│   ├── MidiSharp.Core/              MIDI parsing, sequencing, tempo map, lyrics, AND the SoundBank IR (netstandard2.1)
│   ├── MidiSharp.Audio/             Sample-file decoders — WAV/AIFF/FLAC/Vorbis/PCM (netstandard2.1)
│   ├── MidiSharp.Loader/            Format readers (Sf2/Sf3/Sfz/Dls) + format-to-IR translators + sample sources
│   ├── MidiSharp.Synth/             Synth + RealtimePlayer + per-instrument mixer & insert hook; consumes the IR (netstandard2.1)
│   ├── MidiSharp.Synth.OwnAudio/    Cross-platform audio output via OwnAudioSharp (net10.0)
│   ├── MidiSharp.PatchMap/          Instrument substitution — composes SoundBanks (netstandard2.1, Core-only)
│   ├── MidiSharp.Dsp/               Buffer-processing DSP "plugins" — EQ, limiter, gain, chains (netstandard2.1, no synth deps)
│   ├── MidiSharp.Hosting/           Format-agnostic plugin-host core — IHostedPlugin/IPluginFormat, PlanarBridge, registry (no P/Invoke; net10.0)
│   ├── MidiSharp.Hosting.{Clap,Vst2,Vst3,AudioUnit,Ladspa}/   Per-format native adapters — all the plugin interop lives here (AudioUnit is macOS-only)
│   ├── MidiSharp.Hosting.EditorHost/  Native plugin-editor windows behind a per-OS seam — Linux/ (X11), Windows/ (Win32), MacArm/ (Cocoa)
│   ├── MidiSharp.Hosting.Sandbox/, .Worker/   Out-of-process discovery + load (a crashing plugin can't take the host down)
│   └── MidiSharp/                   Umbrella package host → ships the above as MidiSharp.Player on nuget.org
├── samples/
│   ├── MidiSharp.Demo/              CLI: live playback / WAV render / --patches / --map / --limiter
│   └── MidiSharp.Server/            Browser player: a live per-track mixing console (substitution + mix + EQ/limiter & hosted-plugin racks)
├── tests/                           xUnit — 326 tests (Core, Synth, Loader, Audio, SF2.Net, PatchMap, Dsp, Hosting)
├── vendor/NVorbis/                  Vendored pure-managed Ogg Vorbis decoder (MIT, v0.10.5; assembly MidiSharp.Audio.Vorbis)
├── docs/                            SoundBank IR design doc
└── MIDI/                            Reference PDFs (MIDI 1.0 / 2.0 / SMF / RPs / Universal SysEx)

The Loader project is MidiSharp-free apart from MidiSharp.Core (which owns the SoundBank IR): each format's reader parses the source bytes and a translator flattens them into the IR, so the synth never sees a source format. Audio output goes through MidiSharp.Synth.OwnAudio, a single cross-platform backend (OwnAudioSharp) used by both samples.

Build and test everything from the repo root:

dotnet build MidiSharp.slnx
dotnet test  MidiSharp.slnx

What's implemented

MIDI protocol surface

Synthesis engine

• All 10 SF2 default modulators (velocity → attenuation/filter, CC91/93 sends, CC1/aftertouch/poly-aftertouch → vibrato depth)
• EMU8k attenuation factor (0.4 scaling) applied to InitialAttenuation
• 7-point windowed-sinc interpolation
• Per-voice envelopes (volume + modulation) and LFOs (modulation + vibrato); 2-pole resonant RBJ biquad in every response type (low/high-pass, band-pass, notch, low/high-shelf, peaking), an optional cascaded second filter, and per-zone peaking-EQ bands
• For SFZ v2/ARIA fonts: a generic N-LFO subsystem (incl. sample-and-hold and stepped waveforms) and multi-segment flex envelopes, both routable to pitch/volume/cutoff/EQ, run per-voice alongside the SF2 slots
• FDN reverb (Jot 1991, 8 lines, Hadamard feedback) and stereo chorus
• Voice retrigger semantics, exclusive class handling, per-region polyphony caps, sample looping with loop-until-release
• The engine consumes the format-neutral SoundBank IR and never branches on source format; RealtimePlayer drives it from the audio callback for sample-accurate event timing on every platform

Multi-format loading (SoundBank IR)

Every supported format loads through one in-memory representation. SoundBankLoader.Load(path) sniffs the format and dispatches to a translator that flattens the source hierarchy into the IR (SoundBank → Patch → PatchZone → SampleRef); the synth resolves every note through a single seam, SoundBank.FindPatch(bank, program), and reads samples through one ISampleSource.

SF2 is the most thoroughly validated; SF3/SFZ/DLS render through the same IR and synth path. SFZ coverage is data-driven: across a 2027-font test collection, every opcode the collection actually uses is handled. The IR contract is documented in docs/sound-bank-ir.md.

Recommended Practices

Implemented: RP-001 (SMF 1.0), RP-013 (GM Level 1), RP-014 (Bank Select Response), RP-015 (Reset All Controllers), RP-018 (RPN Sensitivity), RP-019 (Device/Program Name Meta), RP-021 (Sound Controllers), RP-022 (General MIDI 2), RP-026 (Lyric/Display Extensions — via MidiSharp.Lyrics.LyricStream), RP-038 (GM2 Default Modulators), CA-031 (High-Resolution Velocity Prefix). RP-017 (Lyric Definition) is satisfied by dispatching events to callers verbatim.

Verification

326 tests across the suite cover MIDI parsing, sequencer timing, tempo map, RP-026 lyric parsing, the SF2 reader, synthesis (including the shelf/peaking filters and sample-and-hold/stepped LFOs), the sample decoders and all four loaders, patch substitution, the per-instrument mixer + insert engine (with bit-identity guards when untouched), the DSP effects (EQ/limiter/chain), and the plugin-hosting stack (loading, the planar bridge, the AU render shim, the per-OS editor backends); platform-specific editor/plugin tests self-skip off their OS.

Mixing & effects (MidiSharp.Dsp + the synth's mix layer)

A complete signal path sits on top of the spec-faithful renderer, without contaminating it:

• Per-instrument mixer (in the synth). Every voice carries a mix identity; Synthesizer.GetInstrumentMix(bank, program) exposes a live InstrumentMix (gain trim / pan offset / mute / solo / reverb + chorus sends). Gains augment the file's own CC7/CC11 automation rather than replacing it, and an untouched mixer is bit-identical to the pre-mixer engine.
• Per-instrument inserts (Tier-2). An instrument with a registered IInstrumentInsert is summed into a private stereo bus, run through the insert, then mixed to master; instruments with no insert sum straight to master (bit-identical). The synth stays decoupled from MidiSharp.Dsp via this tiny interface — the host adapts a ProcessorChain to it.
• MidiSharp.Dsp — the effects library. IAudioProcessor (interleaved-stereo, in place), ProcessorChain (lock-free reorderable rack), a clean-room RBJ BiquadFilter, ParametricEq (stereo cascade), a stereo-linked brickwall LimiterProcessor, and GainProcessor. No reference to the synth or MIDI — the host wires it at the audio-callback seam (master bus) and per-instrument inserts.
• Track-keyed mixing in the player. The browser console keys each strip on the source MIDI track (the part), so a performer keeps one fader even as their program changes, and two tracks sharing a program get independent faders.

Plugin hosting (MidiSharp.Hosting)

The mixer and insert racks aren't limited to the built-in DSP — they can host external audio plugins. One format-agnostic core (MidiSharp.Hosting, no P/Invoke) defines IHostedPlugin/IPluginFormat; each format's native interop lives in its own adapter, and a no-GC/no-lock PlanarBridge shuttles the engine's interleaved-stereo buffers to and from each plugin's planar float**.

• Five formats — CLAP, VST2, VST3, AU, LADSPA. CLAP is the anchor (pure C ABI, both effects and instruments, sample-accurate events); VST2/VST3 are adapters over the same plumbing; AU (Audio Units, macOS) hosts effects and instruments over the AudioToolbox C API, bridging AU's pull-based render to the engine's push processing; LADSPA is the Linux effects spike. All transcribed clean-room — no vendor SDK headers.
• Effects and instruments. A hosted effect is an IAudioProcessor and drops into any ProcessorChain/insert rack beside the built-in EQ/limiter/gain. A hosted instrument feeds a part as an alternative sound source to the synth, carrying the full channel strip (gain/pan/mute/solo + its own inserts) and fed note-on/off with sample-accurate timing.
• Out-of-process by default (MidiSharp.Hosting.Sandbox + .Worker). Discovery and load run in worker processes, so a plugin that crashes on scan or load takes down only its worker — not the server; per-file scan resume, a hung-plugin watchdog, and audio/parameter/state proxying over a shared-memory ring. Plugin state persists into saved per-song setups.
• Native editor windows (MidiSharp.Hosting.EditorHost). A plugin's own editor opens as a real OS window (in the worker that holds the instance), behind one per-OS backend: X11 (Linux), Win32 (Windows), and Cocoa (macOS/arm64) — each pure platform P/Invoke (libX11 / user32+gdi32 / libobjc+AppKit) with its own run loop, embedding the plugin's child window and driving its timers/fds. AU editors embed the unit's own Cocoa view (kAudioUnitProperty_CocoaUI, with an AUGenericView fallback) through the same Cocoa backend. Verified by embedding real plugins (u-he Podolski on Windows; Surge XT VST3 + CLAP and Apple's built-in Audio Units on macOS).

Discovery uses each format's standard per-OS directories (e.g. ~/.clap, %COMMONPROGRAMFILES%\CLAP, /Library/Audio/Plug-Ins/CLAP); AU uses the macOS component registry (AudioComponentFindNext), scanned in-process so Apple's built-in units surface even under the sandbox. AU v3 (AUAudioUnit) and AAX are not adapted.

Instrument substitution (MidiSharp.PatchMap)

A standalone module that lets you re-author what a song plays without touching the song. The mental model: a soundfont is a (bank, program) → instrument map, and the synth resolves every note through exactly one seam (SoundBank.FindPatch). So substitution is a SoundBank-composition problem, not a synth change.

• PatchUsageAnalyzer — walks the sequenced events and reports the patches a song actually uses (the resolved (bank, program) active at each note), named against a base font. Bank resolution is shared with the synth (MidiSharp.SoundBank.BankResolution) so the list matches playback exactly.
• PatchMapSession — holds a base font, any number of preloaded source fonts, and a map of overrides: (logical bank, program) → a patch in a source font. BuildComposite() returns a SoundBank.
• SoundBankComposer + ConcatenatedSampleSource — the composite is base patches ∪ overrides over a concatenated sample space (overridden zones get their SampleId re-based; stereo links rebased). The synth consumes it through the ordinary LoadSoundFont/FindPatch contract and can't tell it from a native font — zero synth changes.

Sequencing is untouched: the file's program changes still pick logical patches; only what they resolve to changes. Overrides are sparse by default (a few on top of a base font) or can cover the whole set; drums are swapped as a whole kit (bank 128). The session owns the lifetime of the base + source fonts; composites are lightweight borrowed views, rebuilt per playback.

Both front-ends use it: the Demo CLI (--patches, --map) and the web player (file browser + per-patch override pickers, building the composite server-side before play).

Out-of-scope by design

• Mastering / EQ / limiting inside the synth — the synth still only renders the spec. Master and per-instrument EQ/limiting now live in MidiSharp.Dsp, a decoupled DSP library the host wires in caller-side (see Mixing & effects above) — kept deliberately out of the synth so the renderer stays pure.
• Hardware / OS MIDI-port output — this renders audio from MIDI files; it is not a MIDI router. RealtimePlayer drives the in-process synth directly. (The legacy WinMM/CoreMIDI IMidiOutput + MidiPlayer path has been removed.)
• SoundFont authoring / writing — load-and-render only; the standalone SF2 read/write library has been removed. Recreate it if editing is ever needed.
• MIDI 2.0 / UMP — different wire format; the per-note expression model doesn't map cleanly to SF2's channel-sourced modulators, and there's no real-world content yet. Could revisit when real Clip Files appear (MPE is the lower-cost on-ramp if per-note expression is ever wanted).
• XMF container format (RP-032 through RP-037), Mobile DLS (RP-031), Scalable Polyphony (RP-027), Mobile Phone Control (RP-028).
• GM1/GM2 sound-set name registries (RP-024/029/039) — descriptive only; the soundfont supplies the actual sounds.

Future work

• Memory-mapped/streaming sample sources for SFZ and DLS (SF2 is mmap'd, SF3 lazily decodes Vorbis).
• Global "search all fonts by name" in the web player's file browser (it currently filters per folder).
• Saved override presets ("remaps") and instrument upload in the web player (overrides are currently in-session, sourced from the configured font folders).
• UMP-to-MIDI-1.0 translation layer if MIDI 2.0 hardware ever shows up (most "MIDI 2.0" devices ship in 1.0-compatibility mode anyway).
• Further SF3/SFZ/DLS hardening toward the SF2 path's fluidsynth-validated fidelity.
• AU v3 (AUAudioUnit) hosting on macOS — AU v2 (effects, instruments, Cocoa editor) is supported; v3 and AAX remain unadapted.

License

MIT — see LICENSE for the full text.