SJP.AvroTool 0.1.1

Avro Tools

A collection of tools to work with Apache Avro in C#.

Description

The intention of this project is to provide a pure C# implementation of an Avro IDL compiler. Additionally, although the Avro GitHub project does contain a code generator for C#, it contains rather verbose code. This project generates human-readable output via a Roslyn-based code generator.

One other benefit of this project is avoiding the pre-requisite for a Java runtime.

Features

• Compile Avro IDL to an Avro Protocol.
• Compile Avro IDL to Avro Schema.
• Generate C# classes for protocols and schemas.
• Supports additional logical types compared to reference compiler. Note that these may not be usable in practice but can be compiled to compatible Avro Protocol/Schema. The following additional logical types are supported in IDL:
  • uuid
  • time-micros
  • timestamp-micros
  • local-timestamp-ms
  • local-timestamp-micros
  • duration

Installation

Install as a .NET tool:

dotnet tool install --global SJP.AvroTool

Usage

Most of the documentation is provided by the tool itself (outside of the language specifications).

$ avrotool --help

USAGE:
    avrotool [OPTIONS] <COMMAND>

OPTIONS:
    -h, --help       Prints help information
    -v, --version    Prints version information

COMMANDS:
    idl <IDL_FILE>                      Generates a JSON protocol file from an Avro IDL file
    idl2schemata <IDL_FILE>             Extract JSON schemata of the types from an Avro IDL file
    codegen <INPUT_FILE> <NAMESPACE>    Generates C# code for a given Avro IDL, protocol or schem

Examples

Compile IDL to an Avro Protocol

$ cat sample.avdl
protocol TestProtocol {
  record TestRecord {
    string FirstName;
    string LastName;
  }

  void Ping();
}
$ avrotool idl sample.avdl
Generated /home/sjp/repos/AvroTools/TestProtocol.avpr
$ cat TestProtocol.avpr
{
  "protocol": "TestProtocol",
  "types": [
    {
      "type": "record",
      "fields": [
        {
          "name": "FirstName",
          "type": "string"
        },
        {
          "name": "LastName",
          "type": "string"
        }
      ],
      "name": "TestRecord"
    }
  ],
  "messages": {
    "Ping": {
      "request": [],
      "response": "null"
    }
  }
}

Compile IDL to Avro Schema

$ cat sample.avdl
protocol TestProtocol {
  record TestRecord {
    string FirstName;
    string LastName;
  }

  enum TestEnum {
    A,
    B,
    C
  }

  void Ping();
}
$ avrotool idl2schemata sample.avdl
Generated /home/sjp/repos/AvroTools/TestRecord.avsc
Generated /home/sjp/repos/AvroTools/TestEnum.avsc

$ cat TestRecord.avsc
{
    "type": "record",
    "name": "TestRecord",
    "fields": [
        {
            "name": "FirstName",
            "type": "string"
        },
        {
            "name": "LastName",
            "type": "string"
        }
    ]
}

$ cat TestEnum.avsc
{
    "type": "enum",
    "name": "TestEnum",
    "symbols": [
        "A",
        "B",
        "C"
    ]
}

Generate C# code for Avro Protocol and Schema

$ cat sample.avdl
protocol TestProtocol {
  record TestRecord {
    string FirstName;
    string LastName;
  }

  void Ping();
}

$ avrotool codegen sample.avdl Test.Code.Namespace
Generated /home/sjp/repos/AvroTools/TestProtocol.cs
Generated /home/sjp/repos/AvroTools/TestRecord.cs

// Contents of files omitted for brevity