Excalibur.Data.DataProcessing
10.0.0-alpha.15
See the version list below for details.
dotnet add package Excalibur.Data.DataProcessing --version 10.0.0-alpha.15
NuGet\Install-Package Excalibur.Data.DataProcessing -Version 10.0.0-alpha.15
<PackageReference Include="Excalibur.Data.DataProcessing" Version="10.0.0-alpha.15" />
<PackageVersion Include="Excalibur.Data.DataProcessing" Version="10.0.0-alpha.15" />
<PackageReference Include="Excalibur.Data.DataProcessing" />
paket add Excalibur.Data.DataProcessing --version 10.0.0-alpha.15
#r "nuget: Excalibur.Data.DataProcessing, 10.0.0-alpha.15"
#:package Excalibur.Data.DataProcessing@10.0.0-alpha.15
#addin nuget:?package=Excalibur.Data.DataProcessing&version=10.0.0-alpha.15&prerelease
#tool nuget:?package=Excalibur.Data.DataProcessing&version=10.0.0-alpha.15&prerelease
Excalibur.Data.DataProcessing
Data processing utilities for the Excalibur framework.
Installation
dotnet add package Excalibur.Data.DataProcessing
Features
- Batch processing
- ETL operations
- Data transformation
- Bulk operations
Tenancy
The data task queue is a global, type-wide sweep: it runs outside any tenant scope, and its enqueue and
drain surfaces (IDataOrchestrationManager.AddDataTaskForRecordTypeAsync, the pending-task drain) accept
no tenant identity. If your IRecordFetcher records are tenant-owned, your fetcher must apply the tenant
predicate itself -- the queue applies none.
License
This project is multi-licensed under:
See LICENSE for details.
Database schema
This package does not create its tables. The DDL ships inside the .nupkg under scripts/, and
you apply it — so a schema change to your database is always something you did deliberately, never
something a version upgrade did to you while you were not looking.
| Script |
|---|
scripts/001_CreateDataProcessingSchema.sql |
Apply them in filename order before starting the application, and again — for any new ones — before starting a new version after an upgrade.
The scripts/ folder lives inside the package, not in your build output. To read it:
# the package folder, then the scripts inside it
dotnet nuget locals global-packages --list
ls ~/.nuget/packages/excalibur.data.dataprocessing/<version>/scripts/
The folder is authoritative. If any list of scripts — including this one — disagrees with what is actually in the package you installed, the package is right.
Task ownership and recovery
The SQL Server orchestrator requires a factory that returns a new, closed connection. It claims each task with a database-scoped session application lock, rereads its eligibility, and retains the connection until processing and asynchronous scope cleanup finish. Competing workers skip owned tasks. Each checkpoint, failure update, and deletion verifies ownership on that session. Claim connections disable pooling so a lost grant or release response cannot strand an application lock in the pool; this adds one physical connection per claimed task, not per record.
A lost database session fences subsequent task-state writes. It cannot undo external handler effects already underway. Handlers must remain idempotent and tolerate replay and, after connection loss, overlapping external effects. All workers must use the ownership-aware implementation; older workers do not honor these locks. Drain tasks outside ambient transactions; enqueue may participate in a business transaction.
DispatcherTimeoutMilliseconds requests cancellation for each claimed task, including scope cleanup. Cancellation is cooperative: a handler that ignores it can delay completion. The manager joins processing before releasing ownership. An observed timeout consumes a failed attempt; host cancellation does not. Attempts counts observed failures, not process crashes. CompletedCount counts successful processing attempts and may include replayed records, rather than distinct records.
Repeated failed hosted cycles degrade health and become unhealthy at UnhealthyThreshold; a successful cycle resets the failure streak. Exceeding shutdown drain time reports the service as stopped and unhealthy even if noncooperative work has not yet finished.
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
-
net10.0
- Ben.Demystifier (>= 0.4.1)
- CloudNative.CloudEvents (>= 2.8.0)
- CloudNative.CloudEvents.SystemTextJson (>= 2.8.0)
- Cronos (>= 0.12.0)
- Dapper (>= 2.1.72)
- Excalibur.Data (>= 10.0.0-alpha.15)
- Excalibur.Dispatch (>= 10.0.0-alpha.15)
- Excalibur.Dispatch.Abstractions (>= 10.0.0-alpha.15)
- Excalibur.Hosting (>= 10.0.0-alpha.15)
- FluentValidation (>= 12.1.1)
- FluentValidation.DependencyInjectionExtensions (>= 12.1.1)
- IdentityModel (>= 7.0.0)
- JsonNet.ContractResolvers (>= 2.0.0)
- Medo.Uuid7 (>= 3.2.0)
- MemoryPack (>= 1.21.4)
- Microsoft.Extensions.Caching.Abstractions (>= 10.0.10)
- Microsoft.Extensions.Caching.Memory (>= 10.0.10)
- Microsoft.Extensions.Configuration (>= 10.0.10)
- Microsoft.Extensions.Configuration.Abstractions (>= 10.0.10)
- Microsoft.Extensions.Configuration.Binder (>= 10.0.10)
- Microsoft.Extensions.Configuration.CommandLine (>= 10.0.10)
- Microsoft.Extensions.Configuration.EnvironmentVariables (>= 10.0.10)
- Microsoft.Extensions.Configuration.Json (>= 10.0.10)
- Microsoft.Extensions.DependencyInjection (>= 10.0.10)
- Microsoft.Extensions.DependencyInjection.Abstractions (>= 10.0.10)
- Microsoft.Extensions.Diagnostics.HealthChecks (>= 10.0.10)
- Microsoft.Extensions.Diagnostics.HealthChecks.Abstractions (>= 10.0.10)
- Microsoft.Extensions.Hosting (>= 10.0.10)
- Microsoft.Extensions.Hosting.Abstractions (>= 10.0.10)
- Microsoft.Extensions.Http (>= 10.0.10)
- Microsoft.Extensions.Logging (>= 10.0.10)
- Microsoft.Extensions.Logging.Abstractions (>= 10.0.10)
- Microsoft.Extensions.Logging.Console (>= 10.0.10)
- Microsoft.Extensions.ObjectPool (>= 10.0.10)
- Microsoft.Extensions.Options (>= 10.0.10)
- Microsoft.Extensions.Options.ConfigurationExtensions (>= 10.0.10)
- Microsoft.Extensions.Options.DataAnnotations (>= 10.0.10)
- Microsoft.IdentityModel.Tokens (>= 8.17.0)
- Polly (>= 8.6.6)
- System.IdentityModel.Tokens.Jwt (>= 8.17.0)
- System.Threading.RateLimiting (>= 10.0.7)
NuGet packages (1)
Showing the top 1 NuGet packages that depend on Excalibur.Data.DataProcessing:
| Package | Downloads |
|---|---|
|
Excalibur.Jobs.DataProcessing
Data processing job types for the Excalibur Jobs framework. PRE-RELEASE: the public API is not frozen and may change between pre-release builds. Not recommended for production use. Known issues: https://docs.excalibur-dispatch.dev/docs/known-issues |
GitHub repositories
This package is not used by any popular GitHub repositories.
| Version | Downloads | Last Updated |
|---|---|---|
| 10.0.0-alpha.19 | 42 | 10/27/2026 |
| 10.0.0-alpha.18 | 57 | 10/25/2026 |
| 10.0.0-alpha.17 | 66 | 10/23/2026 |
| 10.0.0-alpha.16 | 56 | 10/21/2026 |
| 10.0.0-alpha.15 | 59 | 10/11/2026 |
| 10.0.0-alpha.14 | 67 | 10/2/2026 |
| 10.0.0-alpha.13 | 67 | 9/29/2026 |
| 10.0.0-alpha.12 | 71 | 9/25/2026 |
| 10.0.0-alpha.11 | 80 | 9/15/2026 |
| 10.0.0-alpha.10 | 89 | 9/6/2026 |
| 10.0.0-alpha.9 | 90 | 8/31/2026 |
| 10.0.0-alpha.8 | 94 | 8/14/2026 |
| 10.0.0-alpha.7 | 96 | 8/13/2026 |
| 10.0.0-alpha.6 | 94 | 8/11/2026 |
| 10.0.0-alpha.5 | 85 | 8/10/2026 |
| 10.0.0-alpha.4 | 83 | 8/10/2026 |
| 3.0.0-alpha.216 | 97 | 6/30/2026 |
| 3.0.0-alpha.215 | 87 | 6/23/2026 |
| 3.0.0-alpha.214 | 94 | 6/23/2026 |
| 3.0.0-alpha.208 | 83 | 6/11/2026 |
Release notes and versioning policy: https://docs.excalibur-dispatch.dev/docs/migration/version-upgrades