Singulink.FulcrumFS
2.3.0
Prefix Reserved
dotnet add package Singulink.FulcrumFS --version 2.3.0
NuGet\Install-Package Singulink.FulcrumFS -Version 2.3.0
<PackageReference Include="Singulink.FulcrumFS" Version="2.3.0" />
<PackageVersion Include="Singulink.FulcrumFS" Version="2.3.0" />
<PackageReference Include="Singulink.FulcrumFS" />
paket add Singulink.FulcrumFS --version 2.3.0
#r "nuget: Singulink.FulcrumFS, 2.3.0"
#:package Singulink.FulcrumFS@2.3.0
#addin nuget:?package=Singulink.FulcrumFS&version=2.3.0
#tool nuget:?package=Singulink.FulcrumFS&version=2.3.0

FulcrumFS
FulcrumFS is a high-performance file processing pipeline and storage engine that layers on top of any file system, transforming standard directories into transactional file repositories with database-aligned commit semantics and file variant management.
While it serves as a key component of our upcoming FulcrumDB database engine, it is also designed to function independently, bringing robust file handling to any application or database. FulcrumFS is especially well-suited for managing user-uploaded content such as documents, images and videos, offering strong guarantees around consistency and integrity.
When your application commits its database transaction before the repository transaction, FulcrumFS guarantees that every file the database references stays accessible in the repository (even after a crash or power loss), and every file the database no longer references is eventually deleted so storage never leaks.
Details of each component are provided below:
Supported Runtimes: .NET 10.0+
Libraries may be in the following states:
- Internal: Source code (and possibly a nuget package) is available but the library is intended for internal use at this time.
- Preview: Library is available for public preview but the APIs may not be fully documented and the API surface is subject to change without notice.
- Public: Library is intended for public use with a fully documented and stable API surface.
You are welcome to use any libraries or code in this repository that you find useful and feedback/contributions are appreciated regardless of library state.
About Singulink
We are a small team of engineers and designers dedicated to building beautiful, functional and well-engineered software solutions. We offer very competitive rates as well as fixed-price contracts and welcome inquiries to discuss any custom development / project support needs you may have.
This package is part of our Singulink Libraries collection. Visit https://github.com/Singulink to see our full list of publicly available libraries and other open-source projects.
Components
FulcrumFS
Main library that enables transactional file storage and processing, providing a foundation for building reliable file repositories on top of any file system. The FulcrumFS package contains the interfaces, processing pipeline and shared types; the local file system repository implementation is in FulcrumFS.Local.
Features:
✔️ Commit-then-cleanup coordination with transactional databases
✔️ No distributed transactions required
✔️ Validate, pre-process, and post-process files during storage and retrieval
✔️ Generate and manage file variants (e.g. alternate formats, resolutions, thumbnails)
✔️ Durable, with graceful recovery from crashes, power failures and disconnected storage volumes
✔️ Runs on local file systems; shared and highly available storage coming soon
✔️ Scales to millions of files per repository while maintaining good file system performance characteristics
✔️ Provides direct FileStream access for efficient, low-overhead file I/O
✔️ Stored files remain browsable in standard file managers (e.g. File Explorer, Finder)
✔️ Fully compatible with file system features like encryption and compression
✔️ Designed to work seamlessly with existing backup, redundancy, replication, and storage tools
Application code works against the IFileRepo interface rather than the local FileRepo directly. The local implementation in FulcrumFS.Local stores files in a directory tree; a repository server, with a primary and a synchronously replicated secondary reached over HTTP, is planned as a second implementation, so an application can move from a single machine to shared, highly available storage without changing how it uses the repository. See the Repository Abstraction article.
FulcrumFS.Core
A lightweight library that exposes the FileId type, repository paths, and the standalone FileFormat content validation API. It can be used independently of the main FulcrumFS library; for example, in a client application that needs to convert file IDs received from a service to a resulting path, or to pre-validate user-selected files in a front-end before uploading them to a service that hosts a FulcrumFS repository.
The FileFormat API ships with built-in singletons for common formats (Jpeg, Png, Pdf, Mp4, Mkv, Docx, Zip, and many more), factory methods for text formats (TextAscii, TextUnicode, TextEncoding) and content-agnostic types (AnyContent), and is extensible so you can derive your own FileFormat for custom formats. Validation returns a FileFormatValidationResult (valid or invalid with an error message) rather than throwing, so callers can handle outcomes however they prefer.
The main FulcrumFS library uses FileFormat via the FileFormatValidationProcessor, which integrates validation into a processing pipeline and converts invalid results to FileProcessingExceptions.
FulcrumFS.Documents
Extension package that adds document to PDF conversion capabilities to file processing pipelines, allowing PDF variants (e.g. for previewing or archiving) to be generated from word processing documents, spreadsheets and presentations.
Document conversion is provided by LibreOffice, which must be installed on the host: see the Document PDF Conversion guide.
FulcrumFS.Images
Extension package that adds customizable image processing capabilities to file processing pipelines, including validation, thumbnail generation, resizing, format conversion and metadata stripping.
Image processing is provided by the fantastic ImageSharp library.
FulcrumFS.Local
The local file system repository implementation: FileRepo stores a repository in a directory tree and implements IFileRepo. Applications that store their files on a local file system install this package, which brings in the main FulcrumFS library.
FulcrumFS.Pdf
Extension package that adds PDF image extraction capabilities to file processing pipelines, rendering PDF pages to images (e.g. for thumbnail generation).
PDF rendering is provided by PDFium via the PDFtoImage library.
FulcrumFS.Videos
Extension package that adds customizable video processing capabilities to file processing pipelines, including validation, thumbnail generation, resizing, format conversion, metadata stripping, bitrate limiting and audio stripping.
Video processing is provided by FFmpeg and FFprobe, from a build you supply that includes the libraries the package uses. The Setting Up FFmpeg guide covers what the build needs, with scripts that check an existing build or build a complete one.
Usage
The canonical pattern: open a database transaction and a file-repository transaction, process and add the file, save the resulting FileId on the database row that owns it, then commit the database first and the file repository second.
The pipeline below uses FileProcessingPipelineSelector to route by extension - PDFs are validated, images are re-encoded with a 256x256 JPEG thumbnail variant, and videos are normalized to H.264/AAC MP4 with a JPEG thumbnail variant. Any other extension is rejected up front by the selector.
using FulcrumFS;
using Singulink.IO;
var baseDir = DirectoryPath.ParseAbsolute(@"C:\AppData\MyApp\Files");
using var repo = new FileRepo(baseDir, RepoFormatVersion.Version2);
// PDFs: validate the content matches the extension.
var pdfPipeline = new FileFormatValidationProcessor(new FileFormatValidationOptions(FileFormat.Pdf));
// Shared 256x256 JPEG thumbnail processor (accepts JPEG/PNG sources, always outputs JPEG).
var thumbnailProcessor = new ImageProcessor(new ImageProcessingOptions {
Formats = [
new(ImageFormat.Jpeg),
new(ImageFormat.Png, resultFormat: ImageFormat.Jpeg),
],
Resize = new(ImageResizeMode.FitDown, 256, 256),
});
// Images: accept JPEG/PNG and produce the shared thumbnail variant.
var imagePipeline = new ImageProcessor(new ImageProcessingOptions {
Formats = [new(ImageFormat.Jpeg), new(ImageFormat.Png)],
})
.WithVariant("thumbnail", thumbnailProcessor);
// Videos: normalize to H.264/AAC MP4 and produce a thumbnail variant by chaining
// the video frame extractor into the shared thumbnail processor.
var videoPipeline = new VideoProcessor(VideoProcessingOptions.StandardizedH264AACMP4)
.WithVariant("thumbnail", new FileProcessingPipeline(
new VideoFrameExtractionProcessor(VideoFrameExtractionProcessingOptions.Standard),
thumbnailProcessor,
));
// Route by file type. Unsupported extensions are rejected with a FileProcessingException.
var pipeline = new FileProcessingPipelineSelector(pdfPipeline, imagePipeline, videoPipeline);
// Open transactions for both the database and the file repository.
await using var dbTxn = await myDb.BeginTransactionAsync();
await using var fileTxn = await repo.BeginTransactionAsync();
// Process and add the file to the repo.
using var stream = File.OpenRead(@"C:\incoming\upload.png");
var added = await fileTxn.AddAsync(stream, leaveOpen: false, pipeline);
// Save the FileId on the database row that owns the file.
await myDb.Documents.AddAsync(new Document {
Name = "upload.png",
FileId = added.FileId, // FileId type implicitly converts to Guid
});
// Commit the database FIRST, then the file repository.
await dbTxn.CommitAsync();
await fileTxn.CommitAsync();
// Read the main file (and enumerate any auto-generated variants) at any time using the stored FileId.
var info = await repo.GetGroupAsync(added.FileId);
using var readStream = info.MainFile.Open(FileAccessPattern.Sequential);
// info.VariantFiles contains the "thumbnail" entry for images and videos.
The commit ordering is what guarantees consistency. If a crash occurs after dbTxn.CommitAsync() but before fileTxn.CommitAsync(), the added file is still on disk and reachable through the FileId stored in the database. If a crash occurs before dbTxn.CommitAsync(), the file is left in an indeterminate state on disk with no database reference to it, and the next clean operation will reclaim it.
The same holds when an exception skips fileTxn.CommitAsync(). Disposing a file transaction that was neither committed nor rolled back does not discard its files, because the database commit may have taken effect even though the call threw (a connection lost before the server's answer arrived, for example). The files are left indeterminate, and the next clean operation with a resolution callback keeps the ones a database row references and reclaims the rest. When you know the database did not commit, call fileTxn.RollbackAsync() to discard them at once.
Repository Layout
Files are stored in standard directories under the repository base directory and remain fully browsable in any file manager:
{base}/
fulcrumfs.info Repository marker / version info
fulcrumfs.lock Held by the active FileRepo instance
files/
{yy}/{zz}/{file-id}/ Sharded by FileId bytes
$main.pdf Main file
thumbnail.jpg Optional variants
scaled.jpg
temp/ In-progress work files
cleanup/ Pending delete / indeterminate markers
Each FileId maps deterministically to a sharded subdirectory under files/, keeping any one directory at a manageable file count even when the repository contains millions of entries. The main file is always stored as $main plus its original extension and variants live alongside it as named siblings (thumbnail.jpg, scaled.mp4, etc.).
Periodic FileRepoCleaner.CleanAsync calls (e.g. on a nightly schedule) reclaim files left indeterminate by failed or interrupted commits and by transactions that were disposed without a commit, along with files marked for deferred deletion past their grace period.
Why Not Just Store Files Directly?
A few common alternatives and how FulcrumFS compares:
Storing blobs in the database. Simple to reason about, but blob storage inflates database size, complicates backups, and limits I/O patterns to whatever the database driver supports. FulcrumFS keeps file bytes out of the database and gives callers a direct FileStream for reads and processing.
Writing files to the file system directly and tracking them in the database. Works until something fails midway. Files orphan on disk when an insert rolls back, or database rows reference files that were never flushed when a crash occurs between the two writes. FulcrumFS handles both cases: orphans from failed commits are cleaned up automatically, and a committed database row is guaranteed to find its file on disk.
Distributed transactions (e.g. MSDTC, XA). Solves the consistency problem but at significant cost: coordinator overhead on every commit, extra crash-recovery state on the database that must be resolved before the database can be reopened, and a hard dependency on every participating resource manager supporting the protocol. FulcrumFS reaches the same end-to-end guarantee using only a defined commit order and a periodic cleanup pass, with no coordinator and no crash-recovery state to resolve outside of normal database recovery.
Further Reading
Please head over to the project documentation site to view articles, examples and the fully documented API.
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
-
net10.0
- Microsoft.IO.RecyclableMemoryStream (>= 3.0.1)
- Singulink.Enums (>= 3.0.0)
- Singulink.FulcrumFS.Core (>= 2.3.0)
- Singulink.IO.FileSystem (>= 3.2.1)
- Singulink.Threading (>= 3.0.0)
NuGet packages (5)
Showing the top 5 NuGet packages that depend on Singulink.FulcrumFS:
| Package | Downloads |
|---|---|
|
Singulink.FulcrumFS.Images
FulcrumFS extension that adds customizable image processing capabilities, including validation, thumbnail generation, resizing, format conversion and metadata stripping, for both original images and their derived variants. |
|
|
Singulink.FulcrumFS.Videos
FulcrumFS extension that adds customizable video processing capabilities, including validation, thumbnail generation, resizing, format conversion, metadata stripping, bitrate limiting, and audio stripping, for both original videos and their derived variants. |
|
|
Singulink.FulcrumFS.Pdf
FulcrumFS extension that adds PDF image extraction capabilities, rendering PDF pages to images (e.g. for generating thumbnails) for both original PDF documents and their derived variants. |
|
|
Singulink.FulcrumFS.Documents
FulcrumFS extension that adds document to PDF conversion capabilities using LibreOffice, allowing PDF variants (e.g. for previewing or archiving) to be generated from word processing documents, spreadsheets and presentations. |
|
|
Singulink.FulcrumFS.Local
Local file system implementation of the FulcrumFS repository. |
GitHub repositories
This package is not used by any popular GitHub repositories.
| Version | Downloads | Last Updated |
|---|---|---|
| 2.3.0 | 119 | 11/12/2026 |
| 2.2.0 | 141 | 11/9/2026 |
| 2.1.0 | 183 | 10/15/2026 |
| 2.0.2 | 162 | 10/10/2026 |
| 2.0.1 | 151 | 10/7/2026 |
| 1.0.6 | 196 | 8/28/2026 |
| 1.0.5 | 144 | 8/28/2026 |
| 1.0.4 | 152 | 8/28/2026 |
| 1.0.3 | 159 | 8/28/2026 |
| 1.0.2 | 217 | 8/28/2026 |
| 1.0.1 | 150 | 8/27/2026 |
| 1.0.0 | 168 | 8/20/2026 |
| 1.0.0-preview9 | 190 | 8/20/2026 |
| 1.0.0-preview8 | 158 | 8/19/2026 |
| 1.0.0-preview7 | 214 | 7/22/2026 |
| 1.0.0-preview6 | 155 | 7/11/2026 |
| 1.0.0-preview5 | 159 | 7/11/2026 |
| 1.0.0-preview4 | 164 | 7/10/2026 |