SimpleBase 4.2.0
dotnet add package SimpleBase --version 4.2.0
NuGet\Install-Package SimpleBase -Version 4.2.0
<PackageReference Include="SimpleBase" Version="4.2.0" />
<PackageVersion Include="SimpleBase" Version="4.2.0" />
<PackageReference Include="SimpleBase" />
paket add SimpleBase --version 4.2.0
#r "nuget: SimpleBase, 4.2.0"
#addin nuget:?package=SimpleBase&version=4.2.0
#tool nuget:?package=SimpleBase&version=4.2.0
SimpleBase
This is my own take for exotic base encodings like Base32, Base58 and Base85. I started to write it in 2013 as coding practice and kept it as a small pet project. I suggest anyone who wants to brush up their coding skills to give those encoding problems a shot. They turned out to be more challenging than I expected. To grasp the algorithms I had to get a pen and paper to see how the math worked.
Features
- Base32: RFC 4648, BECH32, Crockford, z-base-32, Geohash, FileCoin and Extended Hex (BASE32-HEX) flavors with Crockford character substitution, or any other custom flavors.
- Base58: Both the standard encoding (Bitcoin (BTC), Ripple (XRP), Flickr, and custom alphabets) and Monero (XMR) Base58 algorithms are supported. Also provides Base58Check and Avalanche CB58 encoding helpers.
- Base85: Ascii85, Z85 and custom flavors. IPv6 encoding/decoding support.
- Base16: UpperCase, LowerCase and ModHex flavors. An experimental hexadecimal
encoder/decoder just to see how far I
can take the optimizations compared to .NET's implementations. It's quite
fast now. It could also be used as a replacement for
SoapHexBinary.Parse
although .NET hasConvert.FromHexString()
method since .NET 5. - One-shot memory buffer based APIs for simple use cases.
- Stream-based async APIs for more advanced scenarios.
- Lightweight: No dependencies.
- Support for big-endian CPUs like IBM s390x (zArchitecture).
- Thread-safe.
- Simple to use.
NuGet
To install it from NuGet:
Install-Package SimpleBase
Usage
Base32
Encode a byte array:
using SimpleBase;
byte[] myBuffer;
string result = Base32.Crockford.Encode(myBuffer, padding: true);
// you can also use "ExtendedHex" or "Rfc4648" as encoder flavors
Decode a Base32-encoded string:
using SimpleBase;
string myText = ...
byte[] result = Base32.Crockford.Decode(myText);
Base58
Encode a byte array:
byte[] myBuffer = ...
string result = Base58.Bitcoin.Encode(myBuffer);
// you can also use "Ripple" or "Flickr" as encoder flavors
Decode a Base58-encoded string:
string myText = ...
byte[] result = Base58.Bitcoin.Decode(myText);
Encode a Base58Check address:
byte[] address = ...
byte version = 1; // P2PKH address
string result = Base58.Bitcoin.EncodeCheck(address, version);
Decode a Base58Check address:
string address = ...
Span<byte> buffer = new byte[maxAddressLength];
if (Base58.Bitcoin.TryDecodeCheck(address, buffer, out byte version, out int numBytesWritten));
buffer = buffer[..numBytesWritten]; // use only the written portion of the buffer
Avalanche CB58 usage is pretty much the same except it doesn't have a separate
version field. Just use EncodeCb58
and TryDecodeCb58
methods instead. For
encoding:
byte[] address = ...
byte version = 1;
string result = Base58.Bitcoin.EncodeCb58(address);
For decoding:
string address = ...
Span<byte> buffer = new byte[maxAddressLength];
if (Base58.Bitcoin.TryDecodeCb58(address, buffer, out int numBytesWritten));
buffer = buffer[..numBytesWritten]; // use only the written portion of the buffer
Base85
Encode a byte array to Ascii85 string:
byte[] myBuffer = ...
string result = Base85.Ascii85.Encode(myBuffer);
// you can also use Z85 as a flavor
Decode an encoded Ascii85 string:
string encodedString = ...
byte[] result = Base85.Ascii85.Decode(encodedString);
Both "zero" and "space" shortcuts are supported for Ascii85. Z85 is still vanilla.
Base16
Encode a byte array to hex string:
byte[] myBuffer = ...
string result = Base16.EncodeUpper(myBuffer); // encode to uppercase
// or
string result = Base16.EncodeLower(myBuffer); // encode to lowercase
To decode a valid hex string:
string text = ...
byte[] result = Base16.Decode(text); // decodes both upper and lowercase
Stream Mode
Most encoding classes also support a stream mode that can work on streams, be
it a network connection, a file or whatever you want. They are ideal for
handling arbitrarily large data as they don't consume memory other than a small
buffer when encoding or decoding. Their syntaxes are mostly identical. Text
encoding decoding is done through a TextReader
/TextWriter
and the rest is
read through a Stream
interface. Here is a simple code that encodes a file to
another file using Base85 encoding:
using (var input = File.Open("somefile.bin"))
using (var output = File.Create("somefile.ascii85"))
using (var writer = new TextWriter(output)) // you can specify encoding here
{
Base85.Ascii85.Encode(input, writer);
}
Decode works similarly. Here is a Base32 file decoder:
using (var input = File.Open("somefile.b32"))
using (var output = File.Create("somefile.bin"))
using (var reader = new TextReader(input)) // specify encoding here
{
Base32.Crockford.Decode(reader, output);
}
Asynchronous Stream Mode
You can also encode/decode streams in asynchronous fashion:
using (var input = File.Open("somefile.bin"))
using (var output = File.Create("somefile.ascii85"))
using (var writer = new TextWriter(output)) // you can specify encoding here
{
await Base85.Ascii85.EncodeAsync(input, writer);
}
And the decode:
using (var input = File.Open("somefile.b32"))
using (var output = File.Create("somefile.bin"))
using (var reader = new TextReader(input)) // specify encoding here
{
await Base32.Crockford.DecodeAsync(reader, output);
}
TryEncode/TryDecode
If you want to use an existing pre-allocated buffer to encode or decode without causing a GC allocation every time, you can make use of TryEncode/TryDecode methods which receive input, output buffers as parameters.
Encoding is like this:
byte[] input = new byte[] { 1, 2, 3, 4, 5 };
int outputBufferSize = Base58.Bitcoin.GetSafeCharCountForEncoding(input);
var output = new char[outputBufferSize];
if (Base58.Bitcoin.TryEncode(input, output, out int numCharsWritten))
{
// there you go
}
and decoding:
string input = "... some bitcoin address ...";
int outputBufferSize = Base58.Bitcoin.GetSafeByteCountForDecoding(output);
var output = new byte[outputBufferSize];
if (Base58.Bitcoin.TryDecode(input, output, out int numBytesWritten))
{
// et voila!
}
Benchmark Results
Small buffer sizes are used (64 characters). They are closer to real life applications. Base58 performs really bad in decoding of larger buffer sizes, due to polynomial complexity of numeric base conversions.
BenchmarkDotNet v0.14.0, Windows 11 (10.0.26100.3915) AMD Ryzen 9 5950X, 1 CPU, 32 logical and 16 physical cores .NET SDK 8.0.408 [Host] : .NET 8.0.15 (8.0.1525.16413), X64 RyuJIT AVX2 DefaultJob : .NET 8.0.15 (8.0.1525.16413), X64 RyuJIT AVX2
Encoding (64 byte buffer)
Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
---|---|---|---|---|---|---|---|---|
DotNet_Base64 | 28.08 ns | 0.377 ns | 0.334 ns | 1.00 | 0.02 | 0.0119 | 200 B | 1.00 |
SimpleBase_Base16_UpperCase | 84.19 ns | 1.715 ns | 2.230 ns | 3.00 | 0.09 | 0.0167 | 280 B | 1.40 |
SimpleBase_Base32_CrockfordWithPadding | 155.99 ns | 1.870 ns | 1.749 ns | 5.56 | 0.09 | 0.0138 | 232 B | 1.16 |
SimpleBase_Base85_Z85 | 148.10 ns | 1.071 ns | 1.002 ns | 5.27 | 0.07 | 0.0110 | 184 B | 0.92 |
SimpleBase_Base58_Bitcoin | 45.40 ns | 0.761 ns | 0.594 ns | 1.62 | 0.03 | 0.0091 | 152 B | 0.76 |
SimpleBase_Base58_Monero | 202.59 ns | 0.600 ns | 0.532 ns | 7.22 | 0.08 | 0.0119 | 200 B | 1.00 |
Decoding (80 character string)
Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
---|---|---|---|---|---|---|---|---|
DotNet_Base64 | 105.74 ns | 2.107 ns | 2.427 ns | 1.00 | 0.03 | 0.0052 | 88 B | 1.00 |
SimpleBase_Base16_UpperCase | 49.20 ns | 0.239 ns | 0.212 ns | 0.47 | 0.01 | 0.0038 | 64 B | 0.73 |
SimpleBase_Base32_Crockford | 125.15 ns | 0.410 ns | 0.383 ns | 1.18 | 0.03 | 0.0048 | 80 B | 0.91 |
SimpleBase_Base85_Z85 | 252.55 ns | 0.938 ns | 0.832 ns | 2.39 | 0.05 | 0.0105 | 176 B | 2.00 |
SimpleBase_Base58_Bitcoin | 5,415.10 ns | 36.000 ns | 33.674 ns | 51.24 | 1.20 | 0.0076 | 176 B | 2.00 |
SimpleBase_Base58_Monero | 127.32 ns | 0.966 ns | 0.903 ns | 1.20 | 0.03 | 0.0105 | 176 B | 2.00 |
Notes
I'm sure there are areas for improvement. I didn't want to go further in optimizations which would hurt readability and extensibility. I might experiment on them in the future.
Test suite for Base32 isn't complete, I took most of it from RFC4648. Base58 really lacks a good spec or test vectors needed. I had to resort to using online converters to generate preliminary test vectors.
Base85 tests are also makseshift tests based on what output Cryptii produces. Contribution to missing test cases are greatly appreciated.
It's interesting that I wasn't able to reach .NET Base64's performance with Base16 with a straightforward managed code despite that it's much simpler. I was only able to match it after I converted Base16 to unsafe code with good independent interleaving so CPU pipeline optimizations could take place. Still not satisfied though. Is .NET's Base64 implementation native? Perhaps.
Thanks
Thanks to all contributors (most up to date is on the GitHub sidebar) who provided patches, and reported bugs.
Chatting about this pet project with my friends @detaybey, @vhallac, @alkimake and @Utopians at one of our friend's birthday encouraged me to finish this. Thanks guys.
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net8.0 is compatible. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. net9.0 was computed. net9.0-android was computed. net9.0-browser was computed. net9.0-ios was computed. net9.0-maccatalyst was computed. net9.0-macos was computed. net9.0-tvos was computed. net9.0-windows was computed. |
-
net8.0
- System.Memory (>= 4.6.3)
NuGet packages (53)
Showing the top 5 NuGet packages that depend on SimpleBase:
Package | Downloads |
---|---|
Makaretu.Dns
DNS data model with serializer/deserializer for the wire and master file format. |
|
KubeOps
This is an operator sdk written in c#. It enables a developer to create a custom controller for CRDs (CustomResourceDefinitions) that runs on kubernetes. |
|
Ipfs.Core
Core objects and interfaces for IPFS. The InterPlanetary File System is the permanent web. It is a new hypermedia distribution protocol, addressed by content and identities. IPFS enables the creation of completely distributed applications. It aims to make the web faster, safer, and more open. |
|
SubstrateNetApi
Just another Substrate .NET API, written in NETStandard2.0 to provide maximum compatibility for Unity3D. |
|
Nethermind.Libp2p.Core
A libp2p implementation for .NET |
GitHub repositories (7)
Showing the top 7 popular GitHub repositories that depend on SimpleBase:
Repository | Stars |
---|---|
stratumauth/app
📱 Two-Factor Authentication (2FA) client for Android + Wear OS
|
|
unosquare/passcore
A self-service password management tool for Active Directory
|
|
TeslaFly01/SmartSqlT
🔥🔥🔥 SmartSQL 是一款方便、快捷的数据库文档查询、导出工具!该工具从最初支持CHM文档格式开始,通过不断地探索开发、集思广益和不断改进,又陆续支持Word、Excel、PDF、Html、Xml、Json、MarkDown等文档格式的导出。同时支持SqlServer、MySql、PostgreSQL、SQLite等多种数据库的文档查询和导出功能。
|
|
slowscript/warpinator-windows
An unofficial implementation of Warpinator for Windows
|
|
BasisVR/Basis
Basis is an open-source social framework for VR and Desktop usage.
|
|
richardschneider/net-ipfs-mount
Mount the InterPlanetary File System as a mapped drive on Windows
|
|
bizanc/Bizanc.io.Core
Bizanc Blockchain
|
Version | Downloads | Last updated |
---|---|---|
4.2.0 | 1,186 | 3 days ago |
4.0.2 | 258,194 | 7 months ago |
4.0.1 | 12,970 | 8 months ago |
4.0.0 | 1,162,047 | 11/10/2022 |
3.1.0 | 706,329 | 5/24/2021 |
3.0.3 | 1,404 | 5/24/2021 |
3.0.2 | 120,569 | 12/11/2020 |
3.0.1 | 94,421 | 2/14/2020 |
3.0.0 | 19,338 | 12/24/2019 |
2.1.0 | 530,195 | 1/21/2020 |
2.0.0 | 26,433 | 10/12/2019 |
1.8.0 | 100,470 | 3/20/2019 |
1.7.1 | 52,470 | 12/4/2018 |
1.6.1 | 15,574 | 7/13/2018 |
1.4.1 | 5,293 | 5/30/2018 |
1.3.1 | 842,955 | 7/27/2017 |
1.3.0 | 1,753 | 7/26/2017 |
1.2.0 | 40,360 | 5/19/2016 |
1.1.1 | 78,860 | 5/18/2016 |
1.1.0 | 2,315 | 5/16/2016 |
## New features
- Monero Base58 algorithm support with `MoneroBase58` class. It can be accessed as `Base58.Monero`
## Improvements
- Eliminate some memory allocations
## Fixes
- Throw `ArgumentOutOfRangeException` with correct parameters in `Base32.DecodeInt64()`