JsonPit 4.3.0

JsonPit

JsonPit change requests and release notes are centralized in the RAIkeep doc/ directory under JsonPit_... filenames; they are not stored separately in this child repository.

Stores JsonPits, json files with a value history, across machines/servers ("eventually persistent").

Start Here

If you want to use JsonPit 4.3.0 from NuGet in another service or agent workflow, start with GettingStarted.md.

That guide now covers:

• package setup for JsonPit, RaiUtils, and OsLibCore
• current Pit and PitItem usage patterns
• the recommended long-lived in-memory server usage pattern
• querying and enumeration patterns that are actually supported
• persistence and synchronized-storage expectations
• a practical PersonPit example for OTW / AfricaStage style backend work

JsonPit's durable recovery events and the strictly read-only CLI inspection path are documented in the pits audit operational manual.

4.3.0

• Carries accepted CR024 deterministic cleanup of owned PID-specific process flags into the coordinated v4.3.0 line.
• Finalizers remain strictly free of filesystem and recovery-publication I/O; crashed processes retain the established TTL and maintenance behavior.
• Current release notes: JsonPit_RELEASE_NOTES_4.3.0.md

4.2.11 (superseded before publication)

• This prepared line was not published; its CR024 changes are carried by v4.3.0.
• Owned flags are removed through RaiFile.rm() after the existing durability boundary; foreign flags and Master.flag are untouched.
• Read-only and writable pits share the same cleanup, while finalizers remain strictly free of filesystem and recovery-publication I/O.
• Crashed processes still leave flags for TTL detection and explicit maintenance pruning.
• Current release notes: JsonPit_RELEASE_NOTES_4.2.11.md

4.2.10

• Adds opt-in recovery-event compaction through PitMaintenanceOptions.ArchiveEvents.
• Preview reports the exact immutable Events_yyyyMMdd-HHmm_to_yyyyMMdd-HHmm.zip archive, UTC range, eligible count, and deferred evidence without mutation.
• Apply creates the archive directly in the existing pit Events directory, validates every filename and byte payload, then removes only the validated loose copies one by one.
• PitAudit.Read(...) transparently combines loose and archived events; PitAudit.Inspect(...) also exposes invalid/archive-conflict diagnostics and deduplicates by event identity.
• Existing archives are never overwritten, replaced, appended to, or staged through Os.TempDir.
• Current release notes: JsonPit_RELEASE_NOTES_4.2.10.md

4.2.9

• Implements accepted incident corrective action CR022: report-only Pit.Maintain(...) checks the non-creating canonical parent and performs zero filesystem mutation when the pit is absent.
• Missing maintenance targets return an explicit deferred result without creating pit directories, flags, receipts, events, or canonical files.
• Canonical and receipt persistence continues through the hardened OsLibCore in-place cloud pathname contract.
• Aligns fallback dependencies on OsLibCore 4.2.9 and RaiUtils 4.2.9.
• Current release notes: JsonPit_RELEASE_NOTES_4.2.9.md
• Mandatory storage contract: Cloud-Storage-In-Place-Invariant.md

4.2.8

• Implements accepted CR021 durable ReceiptFile cleanup evidence so the ten-minute change-file grace survives restart and master transfer without refresh.
• Adds explicit Pit.Maintain(...), PitMaintenanceOptions, and PitMaintenanceResult APIs for report-only inventory and separately authorized cleanup, process-window pruning, and legacy-extension repair.
• Retires eligible artifacts change-file first and receipt second after current-master and canonical-accounting revalidation; malformed or unverifiable evidence remains untouched.
• Aligns fallback dependencies on OsLibCore 4.2.8 and RaiUtils 4.2.8.
• Current release notes: JsonPit_RELEASE_NOTES_4.2.8.md

4.2.7

• Aligns JsonPit with the coordinated seven-package RAIkeep 4.2.7 release implementing accepted CR020.
• Aligns fallback dependencies on OsLibCore 4.2.7 and RaiUtils 4.2.7; persistence behavior is unchanged.
• Current release notes: JsonPit_RELEASE_NOTES_4.2.7.md

4.2.6

• Aligns JsonPit with the coordinated seven-package RAIkeep 4.2.6 release implementing accepted CR019.
• Aligns fallback dependencies on OsLibCore 4.2.6 and RaiUtils 4.2.6; persistence behavior is unchanged.
• Current release notes: JsonPit_RELEASE_NOTES_4.2.6.md

4.2.5

• Aligns JsonPit with the coordinated seven-package RAIkeep 4.2.5 release implementing accepted CR017.
• Existing GetAt(...), timestamped pit export, history ordering, tombstone, deletion-wall, and resurrection semantics are the authoritative projection engine behind pits export --at.
• Aligns fallback dependencies on OsLibCore 4.2.5 and RaiUtils 4.2.5.
• Current release notes: JsonPit_RELEASE_NOTES_4.2.5.md

4.2.4

• Aligns JsonPit with the coordinated seven-package RAIkeep 4.2.4 release implementing accepted CR016.
• Preserves the CR015 nested tombstone and explicit nested-deletion contracts unchanged.
• Aligns fallback dependencies on OsLibCore 4.2.4 and RaiUtils 4.2.4.
• 4.2.4 release notes: JsonPit_RELEASE_NOTES_4.2.4.md

4.2.3

• Implements accepted CR015 nested property tombstones at arbitrary depth.
• Adds PitItem.DeletePropertyPath(...) without changing literal dotted-name behavior in DeleteProperty(...).
• Projected reads and exports omit tombstoned nested properties and recursively prune newly empty parent containers while preserving append-only history.
• Aligns on OsLibCore 4.2.3 and RaiUtils 4.2.3.
• Makes UseLocalRAIkeepSources=false an explicit package-only boundary so release validation cannot silently resolve sibling projects.
• FileSystemWatcher callbacks and debounce work now use weak ownership so an abandoned Pit can be collected and its canonical path reopened.
• The Pit finalizer still performs no recovery publication, watcher disposal, or filesystem I/O.
• The remote synchronization suite now waits for authority-record contents rather than only a materialized Master.flag pathname.
• Keeps the WWWA-based quick-start section in GettingStarted.md for cloud-path pit creation and sample JSON5 seeding.
• The supported cloud-backed provider claim is Dropbox, OneDrive, GoogleDrive, and ICloudDrive.
• PitItem.Id is now the canonical framework identifier.
• Legacy payloads that still contain Name without Id are normalized internally by copying Name into Id, while preserving Name.
• Future use of Name as an application-defined custom field remains supported.
• PitItem.DeleteProperty(...) now projects top-level null tombstones as absent attributes instead of leaking a permanent null shadow.
• Remote-sync workflows continue to align with OsLib's configurable metadata propagation delay handling, including the mkdir polymorphism package line update in OsLib.
• No JsonPit API changes were required beyond the 3.12.0 line; this release refreshes the aligned package baseline and packaged docs.
• Live docs and release-note pointers were refreshed for the 4.2.3 release line, and this README is packaged with the NuGet release.

namespace

JsonPit

classes

ItemsBase: Base container holding a key identifier for item groups.

• ItemsBase: Key

JsonPitBase: Common base for pits with config, flags, and persistence helpers.

• JsonPitBase: ReadOnly, Backup, RunningOnMaster, MasterUpdatesAvailable, TryReleaseProcessWindow, ChangeDir, JsonFile

TimestampedValue: Value with an attached timestamp and round-trip string format.

• TimestampedValue: Value, Time, ToString

MasterFlagFile: Flag file used to track master ownership and last update time.

• MasterFlagFile: Originator, Time, Update

ProcessFlagFile: Flag file used to track the current process and last update time.

• ProcessFlagFile: Process, Update, CurrentProcessId, CurrentFlagName, IsOwnedByCurrentProcess, TryReleaseCurrentProcess
• Activity filename: {MachineName}-{Subscriber}-{PID}.flag; the PID makes ownership process-specific.
• Explicit release verifies the current process identity and removes the exact flag through RaiFile.rm(), including cloud disappearance waiting.
• Process activity windows and master writer tickets are separate; releasing the former never releases the latter.

PitItem: JSON-backed item with metadata and change tracking.

• PitItem: Id, Modified, Deleted, Note, SetProperty, Merge, DeleteProperty, DeletePropertyPath

PitItemExtensions: Helpers for comparing items and aligning timestamps.

• PitItemExtensions: Equals, isLike, aligned

PitItems: History stack of PitItem versions for a single key.

• PitItems: Push, Peek, Get, Merge, Count

Pit: JsonPit file container with item history and persistence.

• Pit: Add, Get, GetAt, Delete, Save, MergeChanges, Keys

Maintenance lifecycle

Normal writable Pit construction with autoload: true preserves JsonPit's original master opening performs cleanup behavior. Construction loads the canonical pit and calls MergeChanges() (applying Maintain(...)): the exact current master merges valid change files, persists any newly accounted history, creates or reuses immutable cleanup receipts, and retires eligible change-file / receipt pairs.

Cleanup is deliberately a later-pass protocol. A newly created receipt starts a ten-minute grace period, and deletion can occur only during a subsequent maintenance pass after that grace has elapsed. JsonPit does not install a background timer. Consequently, a service that opens a pit once and keeps it open may need to call Pit.Maintain(new PitMaintenanceOptions { Apply = true }) periodically; otherwise the next normal master opening performs the later pass.

Expired process-window flags are a separate diagnostic-retention concern. They are inventoried by maintenance but are never deleted implicitly when a master opens a pit. Their removal requires explicit apply, pruning, and age options. The pits CLI can combine ordinary WWWA maintenance and process-flag pruning in one invocation:

pits maintain --wwwa -c OneDrive -r AIA \
  --apply \
  --prune-process-flags \
  --older-than 01:00:00 \
  --json

The combined command already performs ordinary change/receipt maintenance; a second pits maintain --apply call is unnecessary. Recovery event files are a durable audit trail and are not pruned by this operation.

Event-file compaction is a separate explicit operation. Setting ArchiveEvents = true previews the exact immutable archive; adding Apply = true creates it in the existing Events directory and retires only the loose copies whose exact bytes were validated in the archive:

var preview = pit.Maintain(new PitMaintenanceOptions { ArchiveEvents = true });
var applied = pit.Maintain(new PitMaintenanceOptions
{
    Apply = true,
    ArchiveEvents = true
});

The archive name is based on the oldest and newest validated event content times in UTC, for example Events_20260804-0118_to_20260910-1643.zip. Existing same-name archives are immutable: identical content is reusable, while corrupt or different content is reported and all loose evidence is retained. Audit reads remain logically unchanged because PitAudit combines validated loose and archived events without extracting files.

cloud root convention

JsonPit resolves cloud-backed storage locations through OsLib, but the current approach is to read an explicit configured root from Os.Config.Cloud rather than relying on a preferred-root helper.

For Ubuntu development machines, especially when Google Drive is mounted through rclone, GNOME integration, or a team-specific mount path, prefer explicit configuration instead of probe-only discovery.

Recommended shared contract:

• Use RAIkeep.json5 to point the supported provider roots Cloud.Dropbox, Cloud.OneDrive, Cloud.GoogleDrive, and Cloud.ICloudDrive at the active synchronized mounts.
• Keep that file at ~/.config/RAIkeep.json5.
• Reuse the same PascalCase keys as OsLib.

That keeps JsonPit aligned with OsLib in .NET today and with the upcoming Python OsLib, RaiUtils, and JsonPit packages later.

Item: Base item with modified tracking and dirty state management.

• Item: Id, Modified, Deleted, Delete, Valid, Validate, Invalidate

@see GettingStarted.md for the practical onboarding guide, or check the unit tests for lower-level API examples.

Foldable class and contract documentation is available in API.md.

release notes

• Latest release notes: JsonPit_RELEASE_NOTES_4.3.0.md