Singulink.FulcrumFS.Core 2.3.0

Prefix Reserved
dotnet add package Singulink.FulcrumFS.Core --version 2.3.0
                    
NuGet\Install-Package Singulink.FulcrumFS.Core -Version 2.3.0
                    
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Singulink.FulcrumFS.Core" Version="2.3.0" />
                    
For projects that support PackageReference, copy this XML node into the project file to reference the package.
<PackageVersion Include="Singulink.FulcrumFS.Core" Version="2.3.0" />
                    
Directory.Packages.props
<PackageReference Include="Singulink.FulcrumFS.Core" />
                    
Project file
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.
paket add Singulink.FulcrumFS.Core --version 2.3.0
                    
#r "nuget: Singulink.FulcrumFS.Core, 2.3.0"
                    
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
#:package Singulink.FulcrumFS.Core@2.3.0
                    
#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package.
#addin nuget:?package=Singulink.FulcrumFS.Core&version=2.3.0
                    
Install as a Cake Addin
#tool nuget:?package=Singulink.FulcrumFS.Core&version=2.3.0
                    
Install as a Cake Tool

Singulink Fulcrum Logo

FulcrumFS

Chat on Discord Build and Test

FulcrumFS is a high-performance file processing pipeline and storage engine that layers on top of any file system, transforming standard directories into transactional file repositories with database-aligned commit semantics and file variant management.

While it serves as a key component of our upcoming FulcrumDB database engine, it is also designed to function independently, bringing robust file handling to any application or database. FulcrumFS is especially well-suited for managing user-uploaded content such as documents, images and videos, offering strong guarantees around consistency and integrity.

When your application commits its database transaction before the repository transaction, FulcrumFS guarantees that every file the database references stays accessible in the repository (even after a crash or power loss), and every file the database no longer references is eventually deleted so storage never leaks.

Details of each component are provided below:

Library Status Package
FulcrumFS Icon Singulink.FulcrumFS Public View nuget package
FulcrumFS Icon Singulink.FulcrumFS.Core Public View nuget package
FulcrumFS Icon Singulink.FulcrumFS.Documents Public View nuget package
FulcrumFS Icon Singulink.FulcrumFS.Images Public View nuget package
FulcrumFS Icon Singulink.FulcrumFS.Local Public View nuget package
FulcrumFS Icon Singulink.FulcrumFS.Pdf Public View nuget package
FulcrumFS Icon Singulink.FulcrumFS.Videos Public View nuget package

Supported Runtimes: .NET 10.0+

Libraries may be in the following states:

  • Internal: Source code (and possibly a nuget package) is available but the library is intended for internal use at this time.
  • Preview: Library is available for public preview but the APIs may not be fully documented and the API surface is subject to change without notice.
  • Public: Library is intended for public use with a fully documented and stable API surface.

You are welcome to use any libraries or code in this repository that you find useful and feedback/contributions are appreciated regardless of library state.

We are a small team of engineers and designers dedicated to building beautiful, functional and well-engineered software solutions. We offer very competitive rates as well as fixed-price contracts and welcome inquiries to discuss any custom development / project support needs you may have.

This package is part of our Singulink Libraries collection. Visit https://github.com/Singulink to see our full list of publicly available libraries and other open-source projects.

Components

FulcrumFS

Main library that enables transactional file storage and processing, providing a foundation for building reliable file repositories on top of any file system. The FulcrumFS package contains the interfaces, processing pipeline and shared types; the local file system repository implementation is in FulcrumFS.Local.

Features:

✔️ Commit-then-cleanup coordination with transactional databases
✔️ No distributed transactions required
✔️ Validate, pre-process, and post-process files during storage and retrieval
✔️ Generate and manage file variants (e.g. alternate formats, resolutions, thumbnails)
✔️ Durable, with graceful recovery from crashes, power failures and disconnected storage volumes
✔️ Runs on local file systems; shared and highly available storage coming soon
✔️ Scales to millions of files per repository while maintaining good file system performance characteristics
✔️ Provides direct FileStream access for efficient, low-overhead file I/O
✔️ Stored files remain browsable in standard file managers (e.g. File Explorer, Finder)
✔️ Fully compatible with file system features like encryption and compression
✔️ Designed to work seamlessly with existing backup, redundancy, replication, and storage tools

Application code works against the IFileRepo interface rather than the local FileRepo directly. The local implementation in FulcrumFS.Local stores files in a directory tree; a repository server, with a primary and a synchronously replicated secondary reached over HTTP, is planned as a second implementation, so an application can move from a single machine to shared, highly available storage without changing how it uses the repository. See the Repository Abstraction article.

FulcrumFS.Core

A lightweight library that exposes the FileId type, repository paths, and the standalone FileFormat content validation API. It can be used independently of the main FulcrumFS library; for example, in a client application that needs to convert file IDs received from a service to a resulting path, or to pre-validate user-selected files in a front-end before uploading them to a service that hosts a FulcrumFS repository.

The FileFormat API ships with built-in singletons for common formats (Jpeg, Png, Pdf, Mp4, Mkv, Docx, Zip, and many more), factory methods for text formats (TextAscii, TextUnicode, TextEncoding) and content-agnostic types (AnyContent), and is extensible so you can derive your own FileFormat for custom formats. Validation returns a FileFormatValidationResult (valid or invalid with an error message) rather than throwing, so callers can handle outcomes however they prefer.

The main FulcrumFS library uses FileFormat via the FileFormatValidationProcessor, which integrates validation into a processing pipeline and converts invalid results to FileProcessingExceptions.

FulcrumFS.Documents

Extension package that adds document to PDF conversion capabilities to file processing pipelines, allowing PDF variants (e.g. for previewing or archiving) to be generated from word processing documents, spreadsheets and presentations.

Document conversion is provided by LibreOffice, which must be installed on the host: see the Document PDF Conversion guide.

FulcrumFS.Images

Extension package that adds customizable image processing capabilities to file processing pipelines, including validation, thumbnail generation, resizing, format conversion and metadata stripping.

Image processing is provided by the fantastic ImageSharp library.

FulcrumFS.Local

The local file system repository implementation: FileRepo stores a repository in a directory tree and implements IFileRepo. Applications that store their files on a local file system install this package, which brings in the main FulcrumFS library.

FulcrumFS.Pdf

Extension package that adds PDF image extraction capabilities to file processing pipelines, rendering PDF pages to images (e.g. for thumbnail generation).

PDF rendering is provided by PDFium via the PDFtoImage library.

FulcrumFS.Videos

Extension package that adds customizable video processing capabilities to file processing pipelines, including validation, thumbnail generation, resizing, format conversion, metadata stripping, bitrate limiting and audio stripping.

Video processing is provided by FFmpeg and FFprobe, from a build you supply that includes the libraries the package uses. The Setting Up FFmpeg guide covers what the build needs, with scripts that check an existing build or build a complete one.

Usage

The canonical pattern: open a database transaction and a file-repository transaction, process and add the file, save the resulting FileId on the database row that owns it, then commit the database first and the file repository second.

The pipeline below uses FileProcessingPipelineSelector to route by extension - PDFs are validated, images are re-encoded with a 256x256 JPEG thumbnail variant, and videos are normalized to H.264/AAC MP4 with a JPEG thumbnail variant. Any other extension is rejected up front by the selector.

using FulcrumFS;
using Singulink.IO;

var baseDir = DirectoryPath.ParseAbsolute(@"C:\AppData\MyApp\Files");
using var repo = new FileRepo(baseDir, RepoFormatVersion.Version2);

// PDFs: validate the content matches the extension.
var pdfPipeline = new FileFormatValidationProcessor(new FileFormatValidationOptions(FileFormat.Pdf));

// Shared 256x256 JPEG thumbnail processor (accepts JPEG/PNG sources, always outputs JPEG).
var thumbnailProcessor = new ImageProcessor(new ImageProcessingOptions {
    Formats = [
        new(ImageFormat.Jpeg),
        new(ImageFormat.Png, resultFormat: ImageFormat.Jpeg),
    ],
    Resize = new(ImageResizeMode.FitDown, 256, 256),
});

// Images: accept JPEG/PNG and produce the shared thumbnail variant.
var imagePipeline = new ImageProcessor(new ImageProcessingOptions {
        Formats = [new(ImageFormat.Jpeg), new(ImageFormat.Png)],
    })
    .WithVariant("thumbnail", thumbnailProcessor);

// Videos: normalize to H.264/AAC MP4 and produce a thumbnail variant by chaining
// the video frame extractor into the shared thumbnail processor.
var videoPipeline = new VideoProcessor(VideoProcessingOptions.StandardizedH264AACMP4)
    .WithVariant("thumbnail", new FileProcessingPipeline(
        new VideoFrameExtractionProcessor(VideoFrameExtractionProcessingOptions.Standard),
        thumbnailProcessor,
    ));

// Route by file type. Unsupported extensions are rejected with a FileProcessingException.
var pipeline = new FileProcessingPipelineSelector(pdfPipeline, imagePipeline, videoPipeline);

// Open transactions for both the database and the file repository.
await using var dbTxn = await myDb.BeginTransactionAsync();
await using var fileTxn = await repo.BeginTransactionAsync();

// Process and add the file to the repo.
using var stream = File.OpenRead(@"C:\incoming\upload.png");
var added = await fileTxn.AddAsync(stream, leaveOpen: false, pipeline);

// Save the FileId on the database row that owns the file.
await myDb.Documents.AddAsync(new Document {
    Name = "upload.png",
    FileId = added.FileId, // FileId type implicitly converts to Guid
});

// Commit the database FIRST, then the file repository.
await dbTxn.CommitAsync();
await fileTxn.CommitAsync();

// Read the main file (and enumerate any auto-generated variants) at any time using the stored FileId.
var info = await repo.GetGroupAsync(added.FileId);
using var readStream = info.MainFile.Open(FileAccessPattern.Sequential);
// info.VariantFiles contains the "thumbnail" entry for images and videos.

The commit ordering is what guarantees consistency. If a crash occurs after dbTxn.CommitAsync() but before fileTxn.CommitAsync(), the added file is still on disk and reachable through the FileId stored in the database. If a crash occurs before dbTxn.CommitAsync(), the file is left in an indeterminate state on disk with no database reference to it, and the next clean operation will reclaim it.

The same holds when an exception skips fileTxn.CommitAsync(). Disposing a file transaction that was neither committed nor rolled back does not discard its files, because the database commit may have taken effect even though the call threw (a connection lost before the server's answer arrived, for example). The files are left indeterminate, and the next clean operation with a resolution callback keeps the ones a database row references and reclaims the rest. When you know the database did not commit, call fileTxn.RollbackAsync() to discard them at once.

Repository Layout

Files are stored in standard directories under the repository base directory and remain fully browsable in any file manager:

{base}/
    fulcrumfs.info               Repository marker / version info
    fulcrumfs.lock               Held by the active FileRepo instance
    files/
        {yy}/{zz}/{file-id}/     Sharded by FileId bytes
            $main.pdf            Main file
            thumbnail.jpg        Optional variants
            scaled.jpg
    temp/                        In-progress work files
    cleanup/                     Pending delete / indeterminate markers

Each FileId maps deterministically to a sharded subdirectory under files/, keeping any one directory at a manageable file count even when the repository contains millions of entries. The main file is always stored as $main plus its original extension and variants live alongside it as named siblings (thumbnail.jpg, scaled.mp4, etc.).

Periodic FileRepoCleaner.CleanAsync calls (e.g. on a nightly schedule) reclaim files left indeterminate by failed or interrupted commits and by transactions that were disposed without a commit, along with files marked for deferred deletion past their grace period.

Why Not Just Store Files Directly?

A few common alternatives and how FulcrumFS compares:

Storing blobs in the database. Simple to reason about, but blob storage inflates database size, complicates backups, and limits I/O patterns to whatever the database driver supports. FulcrumFS keeps file bytes out of the database and gives callers a direct FileStream for reads and processing.

Writing files to the file system directly and tracking them in the database. Works until something fails midway. Files orphan on disk when an insert rolls back, or database rows reference files that were never flushed when a crash occurs between the two writes. FulcrumFS handles both cases: orphans from failed commits are cleaned up automatically, and a committed database row is guaranteed to find its file on disk.

Distributed transactions (e.g. MSDTC, XA). Solves the consistency problem but at significant cost: coordinator overhead on every commit, extra crash-recovery state on the database that must be resolved before the database can be reopened, and a hard dependency on every participating resource manager supporting the protocol. FulcrumFS reaches the same end-to-end guarantee using only a defined commit order and a periodic cleanup pass, with no coordinator and no crash-recovery state to resolve outside of normal database recovery.

Further Reading

Please head over to the project documentation site to view articles, examples and the fully documented API.

Product Compatible and additional computed target framework versions.
.NET net10.0 is compatible.  net10.0-android was computed.  net10.0-browser was computed.  net10.0-ios was computed.  net10.0-maccatalyst was computed.  net10.0-macos was computed.  net10.0-tvos was computed.  net10.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (1)

Showing the top 1 NuGet packages that depend on Singulink.FulcrumFS.Core:

Package Downloads
Singulink.FulcrumFS

Transactional file processing pipeline and storage engine with two-phase commit and file variant management. This package contains the repository interfaces, the processing pipeline and the shared types; the local file system implementation is in Singulink.FulcrumFS.Local.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last Updated
2.3.0 130 11/12/2026
2.2.0 154 11/9/2026
2.1.0 193 10/15/2026
2.0.2 161 10/10/2026
2.0.1 152 10/7/2026
1.0.6 211 8/28/2026
1.0.5 159 8/28/2026
1.0.4 161 8/28/2026
1.0.3 170 8/28/2026
1.0.2 168 8/28/2026
1.0.1 167 8/27/2026
1.0.0 181 8/20/2026
1.0.0-preview9 191 8/20/2026
1.0.0-preview8 173 8/19/2026
1.0.0-preview7 218 7/22/2026
1.0.0-preview6 159 7/11/2026
1.0.0-preview5 167 7/11/2026
1.0.0-preview4 173 7/10/2026
Loading failed