Yarhl 4.0.0

.NET 6.0

dotnet add package Yarhl --version 4.0.0

NuGet\Install-Package Yarhl -Version 4.0.0

This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.

<PackageReference Include="Yarhl" Version="4.0.0" />

For projects that support PackageReference, copy this XML node into the project file to reference the package.

paket add Yarhl --version 4.0.0

The NuGet Team does not provide support for this client. Please contact its maintainers for support.

#r "nuget: Yarhl, 4.0.0"

#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.

// Install Yarhl as a Cake Addin
#addin nuget:?package=Yarhl&version=4.0.0

// Install Yarhl as a Cake Tool
#tool nuget:?package=Yarhl&version=4.0.0

The NuGet Team does not provide support for this client. Please contact its maintainers for support.

Yarhl, A format ResearcH Library

Yarhl logo

Yarhl is a set of libraries that helps to implement and convert file formats It empowers you with...

♻️ ... APIs to easily convert between custom formats.
📚 ... guidelines to implement and test custom format converters.
🔢 ... advance binary and text reading / writing, encoding and serialization.
📃 ... standard formats implementation like PO for translations.
📂 ... virtual file system to unpack and pack containers efficiently.
🔌... plugin API to find formats and converters in .NET assemblies.

Usage

The project has the following .NET libraries (NuGet packages via nuget.org). The libraries only support .NET LTS versions: .NET 6.0 and .NET 8.0.

- Yarhl.FileFormat: format conversion APIs.
- Yarhl.FileSystem: virtual file system.
- Yarhl.IO: streams, binary and text reading / writing.
- Yarhl.Media.Text: translation formats and converters (Po), table replacer.
- Yarhl.Media.Text.Encoding: euc-jp and token-escaped encodings.
- Yarhl.Plugins: load nearby .NET assemblies and find type implementations.
- Yarhl.Plugins.FileFormat: find formats and converters from loaded assemblies.

[!NOTE]
Are you planning to try a preview version?
Check-out the GitHub project readme for details on how to setup the NuGet preview feed for SceneGate projects.

Quick demo

Yarhl allows you to work with different file formats with an unified API for conversion into binary (serialization / deserialization). Let's try to create a new translatable file format PO from Yarhl.Media.Text and save it into disk.

[!code-csharpDemo PO]

It's frequent to find formats that are containers. Yarhl allows to have a virtual file system to work with its content without having to extract it into disk (saving space and time). For instance, let's open a game file from Nintendo DS that contains thousand of files. Then we will navigate through its files, unpacking, decompressing and finally converting one file into PO. We can use the following libraries for this task:

Ekona: support of NDS game file system.
LayTea: support of formats from Professor Layton games.

[!code-csharpDemo containers]

Showcase

Some cool projects built with Yarhl:

Texim: experimental API for image file formats.
Ekona: support Nintendo DS file formats.
Lemon: support Nintendo 3DS file formats.
LayTea: modding tools for Professor Layton games.
Attack of Friday Monsters tools: modding tools for Attack of the Friday Monsters game.
Metatron: translation framework for Shin Megami Tensei saga games.

License

The software is licensed under the terms of the MIT license.

Product	Compatible and additional computed target framework versions.
.NET	net6.0 is compatible. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 is compatible. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed.

Product

.NET

Compatible target framework(s)

Included target framework(s) (in package)

Learn more about Target Frameworks and .NET Standard.

net6.0
- Microsoft.IO.RecyclableMemoryStream (>= 3.0.0)
- System.Text.Encoding.CodePages (>= 8.0.0)
net8.0
- Microsoft.IO.RecyclableMemoryStream (>= 3.0.0)
- System.Text.Encoding.CodePages (>= 8.0.0)

NuGet packages (9)

Showing the top 5 NuGet packages that depend on Yarhl:

Package	Downloads
Yarhl.Media Yarhl plugin with support of text converters.	7.0K
LibNep Lib for load Neptunia games file formats	1.7K
Yarhl.Media.Text Library with Yarhl converters for text formats.	580
SceneGate.Ekona Library for NDS file formats.	383
Texim Extensible library for image file format and conversings based on Yarhl.	271

GitHub repositories (1)

Showing the top 1 popular GitHub repositories that depend on Yarhl:

Repository	Stars
MeltyPlayer/FinModelUtility Model viewer and command-line tools for extracting models from various GCN/3DS/PC games en-masse.	109

Version	Downloads	Last updated
4.0.0	904	1/31/2024
3.1.0	1,572	12/23/2020
3.0.0	1,111	7/26/2020
3.0.0-alpha07	654	8/25/2019
3.0.0-alpha06	592	7/30/2019
3.0.0-alpha05	478	7/25/2019
3.0.0-alpha04	486	7/19/2019
3.0.0-alpha03	462	6/23/2019
3.0.0-alpha02	530	4/14/2019
3.0.0-alpha01	518	3/5/2019
2.0.0.1	1,069	2/17/2019
2.0.0	853	1/30/2019
2.0.0-alpha.2	399	12/8/2018
2.0.0-alpha.1	416	12/7/2018

## v4.0.0 (January 31, 2024)

New stable release! 🚀 🎉
There are new features, bug fixes but also some architecture changes and **[a new documentation website](https://scenegate.github.io/Yarhl/)** 🤩
The core concept of the framework is the same but there are some breaking changes in APIs and technology replacements (plugins API).

These changes have been extensively tested in the new SceneGate projects (UI tool, Ekona, Lemon) and modding projects. Releasing v4.0 is a new milestone in SceneGate that allows us to start shipping the first versions of these projects.

## Summary

- 📚 Documentation! [checkout in the project site](https://scenegate.github.io/Yarhl/)
- 🔢 Enhancements in the IO namespace bringing compatibility with the native .NET `Stream` type
- ♻️ New converters API that allows initialization via standard class constructors
- 📋 Cloneable formats
- 📃 New simple base class for custom encoding implementations
- 🔌 New native and safer plugin API. No longer based on MEF.
- 🆕 .NET 6.0 and 8.0 support with nullable annotations. .NET Framework is not supported anymore.

### Breaking changes

- **Drop support of .NET Framework**
- **Rename library Yarhl.Media to Yarhl.Media.Text**
- **Remove APIs `ConvertTo`, `TransformTo` and `DataStream.ReadFormat<T>()`**
- **Obsolete `IInitializer<T>`. Use the constructor instead.**
- **`DataStream` does NOT throw `EndOfStreamException` when reading bytes.**
- `ReadByte` returns `-1` when it reaches the end instead of throwing an exception.
- `DataReader` keeps the behavior and it will throw `EndOfStreamException` when reading any data type.
- Move `DataStream.Length` setter to its own method `SetLength(long)`.
- Rename `TextReader` and `TextWriter` to `TextDataReader` and `TextDataWriter`.
- Removed `IStream` interface in favor of `Stream`.
- `TextDataReader` will not return `null` string if it starts reading from the end already. It will throw `EndOfStreamException`.
- `DataStream.Seek` now requires a negative argument to go back positions when `origin` is `SeekOrigin.End`.
- Binary object (de)serialization removed from `DataReader` and `DataWriter` and moved into new types `BinaryDeserializer` and `BinarySerializer`.
- The attribute `BinarySerializable` is no longer required and it's removed.
- The attribute `BinaryOrder` is mandatory for properties in projects running in .NET 6.0
- The attribute property `ReadAs` and `WriteAs` is now `UnderlyingType`
- The attribute for enums is optional. Its defined underlying type will be used.

As part of this release we had [49 issues](https://github.com/SceneGate/Yarhl/milestone/5?closed=1) closed.

## What's Changed

- **Formats and converters**
- ✨🔥 Implement new converter API by @pleonex in https://github.com/SceneGate/Yarhl/pull/191
- ✨ Implement ICloneableFormat by @Kaplas80 in https://github.com/SceneGate/Yarhl/pull/168
- ✨ Enhance debugging view by @pleonex in https://github.com/SceneGate/Yarhl/pull/209
- **IO**
- ✨ Refactor and improvements in binary object (de)serialization by @pleonex in https://github.com/SceneGate/Yarhl/pull/208
- ✨ Implement DataStream inheritance with Stream and support of Readers and Writers with Stream by @pleonex in https://github.com/SceneGate/Yarhl/pull/161
- 👕 Rename TextReader and TextWriter to TextDataReader and TextDataWriter by @pleonex in https://github.com/SceneGate/Yarhl/pull/162
- ✨🔥 Add Stream argument constructor to DataStream and remove IStream interface by @pleonex in https://github.com/SceneGate/Yarhl/pull/170
- ✨ Implement slice for DataStream by @pleonex in https://github.com/SceneGate/Yarhl/pull/204
- ✨ Implement factories to create DataStreams and Nodes from array and standard streams by @pleonex in https://github.com/SceneGate/Yarhl/pull/173
- 🐎 Improve memory consumption of DataStream.WriteTo by @pleonex in https://github.com/SceneGate/Yarhl/pull/183
- ✨ Use DataStream type in Stream properties for readers and writers and explicitly set CanTimeout to false by @pleonex in https://github.com/SceneGate/Yarhl/pull/172
- ✨🐛 Stream overloads in BinaryFormat and NodeFactory and fix threading issue by @pleonex in https://github.com/SceneGate/Yarhl/pull/163
- ✨ Implement constructor to create BinaryFormat from a file by @pleonex in https://github.com/SceneGate/Yarhl/pull/205
- 🐛 Fix behavior of DataStream.Seek when SeekOrigin is End by @Kaplas80 in https://github.com/SceneGate/Yarhl/pull/177
- 🐛 Comparing stream to itself returns false by @pleonex in https://github.com/SceneGate/Yarhl/pull/199
- 🐛 WriteTo truncates output file by @pleonex in https://github.com/SceneGate/Yarhl/pull/203
- **FileSystem**
- ✨ Implement node renaming by @pleonex in https://github.com/SceneGate/Yarhl/pull/200
- ✨ Implement extension methods to transform node collections by @pleonex in https://github.com/SceneGate/Yarhl/pull/201
- ✨ ChangeFormat fluent style returning itself by @pleonex in https://github.com/SceneGate/Yarhl/pull/202
- ✨ Advanced filter in FromDirectory by @Kaplas80 in https://github.com/SceneGate/Yarhl/pull/166
- ✨ Support opening nodes from Windows symlinks by @pleonex in https://github.com/SceneGate/Yarhl/pull/180
- 🐛 Set FileOpenMode at NodeFactory.FromFile by @Kaplas80 in https://github.com/SceneGate/Yarhl/pull/165
- 🐛 Fix NodeFactory.FromDirectory by @Kaplas80 in https://github.com/SceneGate/Yarhl/pull/188
- **Media.Text**
- 👕 Rename Yarhl.Media to Yarhl.Media.Text by @pleonex in https://github.com/SceneGate/Yarhl/pull/187
- ✨ Implement simple base class for custom encoding implementations by @pleonex in https://github.com/SceneGate/Yarhl/pull/181
- 🐛 Fix Binary2Po converter not reading from the beginning of the stream by @Kaplas80 in https://github.com/SceneGate/Yarhl/pull/169
- **Plugins**
- ❇️ Re-implementation of plugins API by @pleonex in https://github.com/SceneGate/Yarhl/pull/206
- **Documentation**
- ✨📚 New documentation website and project name re-meaning by @pleonex in https://github.com/SceneGate/Yarhl/pull/192
- 🐛 Fix "Yarhl in a nutshell" link by @Timo654 in https://github.com/SceneGate/Yarhl/pull/182
- 🐛 Fixed a small error in the Docs by @priverop in https://github.com/SceneGate/Yarhl/pull/184
- **Build system**
- ✨ Annotate API reference types for nullable awareness by @pleonex in https://github.com/SceneGate/Yarhl/pull/176
- 🔥 Deprecate .NET Framework by @pleonex in https://github.com/SceneGate/Yarhl/pull/178
- ✨ Support .NET 8.0 by @pleonex in https://github.com/SceneGate/Yarhl/pull/198
- ✨ Update build system to build and test for .NET 6 by @pleonex in https://github.com/SceneGate/Yarhl/pull/174
- ✨ New build system and warning clean-up by @pleonex in https://github.com/SceneGate/Yarhl/pull/197
- ⬆️ Bump build system and use Cake v2.1 by @pleonex in https://github.com/SceneGate/Yarhl/pull/179
- ⬆️ Migrate to Cake 3.0 and latest .NET SDK by @pleonex in https://github.com/SceneGate/Yarhl/pull/190
- ⬆️ Update dependencies and fix code coverage by @pleonex in https://github.com/SceneGate/Yarhl/pull/171
- 🔧 Update build system for Ubuntu 20.04 by @pleonex in https://github.com/SceneGate/Yarhl/pull/167
- 🔧 Disable auto-comment on release by @pleonex in https://github.com/SceneGate/Yarhl/pull/160

**Full Changelog**: https://github.com/SceneGate/Yarhl/compare/v3.1.0...v4.0.0