EdnParser 0.2.0

Extensible Data Notation Parser for C#

This is a .Net Core 2.0 implementation of an Extensible Data Notation (EDN) parser. It currently does not do EDN generation.

Quickstart

Create a new parser with Parser parser = new Parser(); and then pass in your EDN string to the Parse function.

using EdnParser;
using System.Linq;

...

String edn = "[db/connect {:host env/server :port env/port :user env/user :password env/password}]\n[db/list :tables]";
Parser parser = new Parser();
var parsed = parser.Parse(edn);
foreach (var val in parsed)
{
    foreach (var elem in val)
    {
        Console.Write(elem);
        Console.Write(" ");
    }
    Console.WriteLine();
}

Output:

db/connect {:host env/server, :port env/port, :user env/user, :password env/password}
db/list :tables

Parsing

Examples for all the parsing methods discussed here can be found in the ExampleProject.

There are three main ways to parse with this library:

• Fully parse an EDN string
• Partially parse an EDN string
• Parse an EDN stream

Fully Parsing

When fully parsing, the parser assumes that you are providing a complete EDN string and will throw an error if the string is not a valid EDN string. It will then return a list of all EDN tokens back to you that you can handle.

This is done by calling Parse on a parser.

Usage:

String edn = "[db/connect {:host env/server :port env/port :user env/user :password env/password}]\n[db/list :tables]";
Parser parser = new Parser();
var parsed = parser.Parse(edn);
foreach (var val in parsed)
{
    foreach (var elem in val)
    {
        Console.Write(elem);
        Console.Write(" ");
    }
    Console.WriteLine();
}

Partially Parsing

When partially parsing, the parser assumes that the string is incomplete so part of it may be invalid. It will then do a greedy parse of the string to get as many valid tokens as possible and then assumes the rest of the string is simple incomplete. It will return a PartialParse which has the properties ParsedTokens (a list of parsed tokens) and Unparsed (a string of what could not be parsed).

Please note that since this is greedy, it will try to match whatever it is given even if it doesn't accurately represent the full EDN data. This means that if you have the string "1 4\ntrue false" but break it into the pieces "1 4\n tr" and "ue false" and ran it through the partial parsing, you would get the integers 1,4 and the symbol "tr" for the first half and the symbols "ue" and "false" for the second half with no unparsed strings. For this reason, it's recommended that you split your code up on some form of whitespace, such as newlines.

You can do a partial parse by calling ParseAsMuchAsPossible on a parser.

Usage:

Parser parser = new Parser();
String edn = "[db/connect \n{:host env/server :port env/port\n :user env/user :password env/password}]\n[db/list :tables]\n[unparsable";
var lines = edn.Split("\n");
var input = "";

foreach(var line in lines)
{
    var parsed = parser.ParseAsMuchAsPossible(input + line);
    foreach (var val in parsed.ParsedTokens)
    {
        foreach (var elem in val)
        {
            Console.Write(elem);
            Console.Write(" ");
        }
        Console.WriteLine();
    }
    input = parsed.Unparsed;
}
Console.WriteLine("Unparsable: " + input);

Output:

db/connect {:host env/server, :port env/port, :user env/user, :password env/password}
db/list :tables
Unparsable: [unparsable

Parse a Stream

The parser also gives you the option to parse a stream. This option is great if you have data that is too big to fit entirely in memory or if you want to process data as it comes in. This parsing mode is different than the others in that it doesn't aggregate everything into a single list but it instead calls a function every time it finds a complete token. The reason for this is that it is designed to be used for when you either don't have enough memory to fit your data, so it can't create a list that would fit in memory, or you need to process data as it comes in (say, from a WebSocket) so you don't want to wait until everything is done transmitting. The stream parser works with an internal buffer and carries over any unparsable input from the last time it pulled. It won't parse a token unless it is followed by some sort of delimiter (white space, starting bracket, etc) which means it will properly parse "5 tr|ue false" as the integer 5, followed by the booleans true and false.

Parsing by a stream is done by calling StreamParse or StreamParseAsync on a parser. StreamParse will synchronously parse a stream in the current thread while StreamParseAsync will launch a Task that will run the parsing. If either of these parsers encounter an exception while parsing (such as the stream closing prematurely) then it will call the error handler you provide (if present) and return.

Both of these functions take in required arguments of the StreamReader to use for parsing and an Action that will be called every time a new token is processed. They optionally take in an unparsed handler which is called after the stream is empty if there is any unparsed input; an error handler function which will be called with the current error and the last unprocessed input (not including what is in the buffer); the maximum unprocessed string length limit (by default there no limit), and the buffer size (defaults to 2048).

If you provide a maximum unprocessed string length limit and the parser exceeds that limit, then it will call your error handler with an InvalidDataException and quit parsing.

Usage:

Parser parser = new Parser();
using (StreamReader sr = new StreamReader(stream, Encoding.UTF8)) {
    Action<EdnValue> processEdnToken = (EdnValue val) =>
    {
        foreach (var elem in val)
        {
            Console.Write(elem);
            Console.Write(" ");
        }
        Console.WriteLine();
    };
    Action<String> handleUnprocessedInput = (String unprocessed) => Console.WriteLine("Unparsable: " + unprocessed);
    task = parser.StreamParseAsync(sr, processEdnToken, handleUnprocessedInput);
    task.Wait();
}

Generating Strings

To generate a string from EDN values, use the Generator static class' function CreateStringFrom. Below is an example:

// from a single value
string res = Generator.CreateStringFrom(
    new EdnVector((new EdnValue[] {
        new EdnInteger(0),
        new EdnInteger(1),
        new EdnFloat(2.3),
        new EdnKeyword(new EdnParser.Values.Keyword("hello"))
    })
));
// res = "[0 1 2.3 :hello]"

To do it for a list of values:

// from a list of values
string res = Generator.CreateStringFrom(
    (new EdnValue[] {
        new EdnInteger(0),
        new EdnInteger(1),
        new EdnFloat(2.3),
        new EdnKeyword(new EdnParser.Values.Keyword("hello"))
    }).ToList()
);
// res = "0\n1\n2.3\n:hello"

To include Windows-style carriage returns:

Generator.CreateStringFrom(
    (new EdnValue[] {
        new EdnInteger(0),
        new EdnInteger(1),
        new EdnFloat(2.3),
        new EdnKeyword(new EdnParser.Values.Keyword("hello"))
    }).ToList(),
    true
);
// res = "0\r\n1\r\n2.3\r\n:hello"