Scrubkit.Tool
1.6.0
Prefix Reserved
See the version list below for details.
dotnet tool install --global Scrubkit.Tool --version 1.6.0
dotnet new tool-manifest
dotnet tool install --local Scrubkit.Tool --version 1.6.0
#tool dotnet:?package=Scrubkit.Tool&version=1.6.0
nuke :add-package Scrubkit.Tool --version 1.6.0
Scrubkit.Tool
The scrubkit command-line tool — run Scrubkit
from any shell or CI, no code required. Point it at a folder and it extracts text + metadata
from PDFs, Office / OpenDocument, email, EPUB, and text / image files, optionally redacts PII
and secrets, and writes a CSV / JSON / JSON Lines / Parquet table. Everything runs on your
machine — no network calls.
Install
dotnet tool install --global Scrubkit.Tool
Use
scrubkit scan <folder> [options]
# Extract a folder to CSV on stdout
scrubkit scan ./docs
# Redact PII + secrets and write JSON Lines for a RAG / embedding pipeline
scrubkit scan ./docs --redact --format jsonl --out docs.jsonl
# Aggressive redaction, only PDFs and email, with a content hash per file
scrubkit scan ./docs --redact=aggressive --include .pdf,.eml --hash
# Columnar output for analytics
scrubkit scan ./data --format parquet --out data.parquet
# Incremental: only output files changed since the last run, and update the manifest
scrubkit scan ./docs --since state.txt --manifest state.txt --out delta.jsonl
Options
| Option | Description |
|---|---|
--format <fmt> |
csv, json, jsonl, or parquet. Default: inferred from --out's extension, else csv. |
--out <file> |
Write to a file instead of stdout. Required for parquet. |
--redact[=<level>] |
Redact PII + secrets. Level standard (default) or aggressive. Omit to extract without redacting. |
--no-recurse |
Only the top folder (default: recurse all nested). |
--hash |
Compute a SHA-256 content hash per file. |
--include <exts> |
Comma-separated extension filter, e.g. --include .pdf,.docx. |
--since <manifest> |
Incremental: only output files changed since <manifest> (a missing file = first run). Skips unchanged files. |
--manifest <file> |
Write a manifest of this scan to <file> (for a later --since). |
--max-files <n> |
Stop after n files (0 = no limit). |
--max-bytes <n> |
Skip files larger than n bytes (0 = no limit). |
--max-text <n> |
Clip extracted text to n characters (0 = no clip). |
--local-time |
Emit local-time timestamps (with offset) instead of UTC. |
-h, --help |
Show help. |
-v, --version |
Show the version. |
The table goes to stdout (or --out); progress and a summary go to stderr, so you can
pipe the output cleanly. Exit code is 0 on success, 1 on a usage or I/O error.
Scrubbing is best-effort, not a compliance tool.
Links
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net8.0 is compatible. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. net9.0 was computed. net9.0-android was computed. net9.0-browser was computed. net9.0-ios was computed. net9.0-maccatalyst was computed. net9.0-macos was computed. net9.0-tvos was computed. net9.0-windows was computed. net10.0 was computed. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
This package has no dependencies.
1.5.0 — The tool release. New Scrubkit.Tool: a dotnet tool (scrubkit scan <folder>) that extracts, redacts PII + secrets, and writes CSV/JSON/JSON Lines/Parquet from any shell or CI. Secret detection in StandardRedactor: PEM private keys, JWTs, connection strings, and AWS/GCP/GitHub/Slack keys (Standard); credential assignments and high-entropy tokens (Aggressive). RAG kit: a Chunker over FileRecord.Text plus JSON Lines output on TableWriter for records and chunks. Full changelog: https://github.com/jjopensoftworks-blip/Scrubkit/blob/main/CHANGELOG.md