VersaTul.Collection.Streamers 1.0.34

VersaTul Collection Streamers

VersaTul Collection Streamers is a library for turning collections or forward-only IDataReader sources into reusable export streams such as CSV, tab-delimited text, JSON, and JSONL. It also works with display attributes to manipulate the properties on the objects in the collection.

Features

• Convert from a data-reader to other file formats such as CSV, TAB or JSON
• Stream large exports directly to disk without first buffering the full file in memory
• Use display attributes to format or rename the properties
• Send streamers via email attachments
• Save streamers to physical files on disk
• Compress streamers for transport over the network
• Convert any collection to an IDataReader

Installation

To use VersaTul Collection Streamers, first install it using nuget:

PM> NuGet\Install-Package VersaTul.Collection.Streamers -Version latest

Usage

The library provides several interfaces and classes to create and manipulate streamers. Here are some of the main ones:

• IStreamer: A base interface that represents the data contained in the stream and the functionality that can be applied to the data.
• IStreamCreator: A base interface that represents the functionality for creating streamers.
• IDataReaderStreamCreator: Adds support for binding an existing IDataReader to a streamer.
• IFileWritableStreamer: Adds direct-to-disk writing through WriteToFile(...).
• ICsvStreamer: A specific stream type interface that represents the functionality for creating csv streamers.
• ITabStreamer: A specific stream type interface that represents the functionality for creating tab streamers.
• IJsonStreamer: A specific stream type interface that represents the functionality for creating json streamers.
• BaseStreamer: A concrete implementation of the core streamer contracts, providing common functionality for collection-backed and IDataReader-backed exports.
• CsvStreamer: A concrete implementation of the ICsvStreamer interface.
• JsonStreamer: A concrete implementation of the IJsonStreamer interface.
• TabStreamer: A concrete implementation of the ITabStreamer interface.
• IMailTransporter: Represents a set of functionality to send streamers via email attachments.
• IStreamFileConverter: Represent a set of conversion techniques that can be applied to a streamer. For plain file output, this now prefers WriteToFile(...) when the streamer supports it.
• ICompressTransport: Represents a set of functionality to reduce streamers for transport over the network.
• CollectionReaderExtensions: Contains the ToReader method that can be used to convert any collection to an IDataReader.

Example: Collection To CSV File

using VersaTul.Collection.Streamers;
using VersaTul.Handler.File;
using VersaTul.Object.Converters;
using VersaTul.Utilities;

namespace CollectionStreamers
{
    class Program
    {
        static void Main(string[] args)
        {
            var people = GetPeople(1000);

            var directoryWrapper = new DirectoryWrapper();
            var utility = new CommonUtility();
            var flattener = new Flattener();
            var fileUtil = new FileUtility(directoryWrapper, directoryWrapper);

            var csvStreamer = new CsvStreamer(utility, fileUtil, flattener);

            var filePath = ((IFileWritableStreamer)csvStreamer.Create(people, "people"))
                .WriteToFile("C:\\your\\file\\path\\here\\");

            Console.WriteLine(filePath);
        }

        private static IEnumerable<Person> GetPeople(int amount)
        {
            var people = new List<Person>(amount);
            var names = new[] { "John Doe", "Jane Smith", "Susan Williams", "Mike Burger", "Joe Williams", "Timmy Smith", "Lisa Ray", "Stanley Smith", "Sam Jones", };

            for (int i = 0; i < amount; i++)
            {
                people.Add(new Person
                {
                    Age = i + 10,
                    Name = CommonUtil.RandomSampler(names),
                    AccountBalance = (100.99m * i),
                    BestFriend = CommonUtil.RandomSampler(people)
                });
            }

            return people;
        }
    }

    // Data Model
    public class Person
    {
        public int Age { get; set; }
        public string? Name { get; set; }
        public decimal AccountBalance { get; set; }
        public IEnumerable<Person>? Friends { get; set; }
        public Person? BestFriend { get; set; }
    }

    // Helper class
    public static class CommonUtil
    {
        public static T? RandomSampler<T>(IList<T> source)
        {
            var max = source.Count;
            if (max == 0) return default;
            var rand = new Random();
            var position = rand.Next(max);
            return source[position];
        }
    }
}

Example: Stream SQL Reader Rows Directly To Disk

using System.Data;
using VersaTul.Collection.Streamers;
using VersaTul.Collection.Streamers.Contracts;
using VersaTul.Handler.File;
using VersaTul.Object.Converters;
using VersaTul.Utilities;

IDataReader reader = dataService.ExecuteReader(command);

var directoryWrapper = new DirectoryWrapper();
var utility = new CommonUtility();
var flattener = new Flattener();
var fileUtil = new FileUtility(directoryWrapper, directoryWrapper);
var csvStreamer = new CsvStreamer(utility, fileUtil, flattener);

var filePath = ((IFileWritableStreamer)csvStreamer.Create(reader, "orders-export"))
    .WriteToFile("C:\\exports");

Example: Keep Using In-Memory Streams

var csvStreamer = new CsvStreamer(utility, fileUtil, flattener);

using var fileStream = csvStreamer
    .Create(people, "people")
    .GetFileStream();

Notes

• Create(IDataReader, ...) lets you bind a forward-only reader without first materializing a collection.
• WriteToFile(...) is the preferred path for very large exports because rows can be written directly to disk.
• GetFileStream() remains available for callers that still need an in-memory MemoryStream.
• FileConverter.Save() now uses the direct file-writing path automatically for non-compressed output when the streamer supports it.

License

This project is licensed under the MIT License - see the LICENSE file for details.