DataLinq.Snowflake 1.3.0

DataLinq.Snowflake

LINQ-native Snowflake integration for DataLinq.NET.

dotnet add package DataLinq.Snowflake --version 1.3.0

Free dev tier included — 1,000 rows, no license key, no credit card. The core DataLinq.NET package (streaming, SUPRA pattern, Cases, EF Core) is Apache 2.0 free and a dependency.

📖 Full Documentation | DataLinq.NET on GitHub | 🌐 Product Website

Features

• Native LINQ Translation - Write C# LINQ, execute Snowflake SQL
• Streaming Results - Row-by-row processing with IAsyncEnumerable
• Type Safety - Strong typing with automatic column mapping
• SQL Injection Prevention - Parameterized queries by default
• O(1) Memory Writes - Native streaming via PUT + COPY INTO
• Cases Pattern - Multi-output conditional routing
• Auto-UDF — Custom methods in Where/Select/OrderBy/GroupBy auto-translate to Snowflake UDFs (static, instance, lambda, entity-param)
• ForEach — Server-side iteration via stored procedures with static field sync-back
• Pull() Escape Hatch - Switch to client-side streaming for edge cases

Quick Start

using DataLinq.SnowflakeQuery;

// Connect to Snowflake
await using var context = Snowflake.Connect(
    account: "xy12345.us-east-1",
    user: "myuser",
    password: "mypass",
    database: "MYDB",
    warehouse: "COMPUTE_WH"
);

// Query with LINQ (server-side SQL)
var orders = await context.Read.Table<Order>("orders")
    .Where(o => o.Amount > 1000)
    .OrderByDescending(o => o.OrderDate)
    .Take(100)
    .ToList();

// Client-side processing requires explicit Pull()
await context.Read.Table<Order>("orders")
    .Where(o => o.Status == "Active")    // Server-side SQL
    .Pull()                               // ← Switch to client
    .ForEach(o => Console.WriteLine(o))   // Client-side C#
    .Do();

// Update specific columns only (compile-time safe expression)
await records.MergeTable(context, "ORDERS", o => o.OrderId,
    updateOnly: o => new { o.Status, o.UpdatedAt });

Nested Objects (VARIANT)

Access Snowflake VARIANT columns with natural C# property syntax:

// Model with nested properties
public class Order {
    public int Id { get; set; }
    
    [Variant]  // Marks column as VARIANT
    public OrderData Data { get; set; }
}

// Query nested properties - translates to colon syntax
var parisOrders = await context.Read.Table<Order>("ORDERS")
    .Where(o => o.Data.Customer.City == "Paris")
    .ToList();
// SQL: WHERE data:customer:city = 'Paris'

Auto-UDF — Custom Methods in LINQ Queries

Use your own C# methods directly in Where/Select/OrderBy — they auto-translate to Snowflake UDFs:

// Static method → auto-registered as Snowflake UDF
static string Classify(decimal amount) => amount > 1000 ? "HIGH" : "LOW";

var results = context.Read.Table<Order>("ORDERS")
    .Select(o => new { o.Id, Tier = Classify(o.Amount) })
    .ToList();
// SQL: SELECT "ID", "UDF_Classify"("AMOUNT") AS "Tier" FROM ORDERS

// Works with Where too
var highValue = context.Read.Table<Order>("ORDERS")
    .Where(o => Classify(o.Amount) == "HIGH")
    .ToList();

Instance methods, lambda closures, and entity-parameter methods are also supported. See the full documentation.

Write Operations

Snowflake uses native IAsyncEnumerable streaming - O(1) memory, no config needed:

// Bulk insert (streams via PUT + COPY INTO)
await records.WriteTable(context, "ORDERS");
await records.WriteTable(context, "ORDERS", createIfMissing: true);
await records.WriteTable(context, "ORDERS", overwrite: true);
await records.WriteTable(context, "ORDERS", createIfMissing: true, overwrite: true);

// Upsert (merge) on key — all columns updated
await records.MergeTable(context, "ORDERS", o => o.OrderId);

// Upsert — only update specific columns (compile-time safe expression)
await records.MergeTable(context, "ORDERS", o => o.OrderId,
    updateOnly: o => new { o.Status, o.UpdatedAt });

// Multi-column single-property shorthand
await records.MergeTable(context, "ORDERS", o => o.OrderId,
    updateOnly: o => o.Status);

ForEach — Server-Side Iteration with Sync-Back

Your C# code is translated to a Java stored procedure, deployed and executed inside Snowflake — no data leaves the server:

// Static fields — accumulate on Snowflake, sync back to C#
static long RowCount = 0;
static double TotalAmount = 0.0;

static void CountAndSum(Order o)
{
    RowCount++;
    TotalAmount += (double)o.Amount;
}

// ForEach is LAZY — nothing executes until you trigger it
var deferred = context.Read.Table<Order>("ORDERS")
    .Where(o => o.IsActive)
    .ForEach(CountAndSum);

// Trigger execution — stored procedure runs on Snowflake
var count = await deferred.Count();

// Fields are synced back automatically
Console.WriteLine($"Processed {RowCount} rows, total: {TotalAmount}");

Pipeline: C# lambda → IL analysis → Java stored procedure → Snowflake execution → sync back to C# static fields. Collections (List<T>, arrays) are not synchronized — use scalar accumulators.

Test Coverage

Requirements

• .NET 8.0+
• DataLinq.NET 1.0.0+

Support & Issues

📧 Contact: support@get-datalinq.net
🐛 Report Issues: github.com/improveTheWorld/DataLinq.NET/issues

License

Free Tier (No Setup Required)

DataLinq.Snowflake works out of the box with no license and no configuration. The free tier allows up to 1,000 rows per query — exceeding this throws a LicenseException. No environment variables, no debugger detection, no opt-in needed. Just install and run.

Production License

For unlimited rows, obtain a license at:

• 🌐 Pricing: https://get-datalinq.net/pricing
• 📧 Contact: support@get-datalinq.net

Set your license key as an environment variable (auto-detected at runtime):

# PowerShell
$env:DATALINQ_LICENSE_KEY="your-license-key"

# Bash/Linux/macOS
export DATALINQ_LICENSE_KEY="your-license-key"

# Docker / Kubernetes
ENV DATALINQ_LICENSE_KEY=your-license-key

Security: The license key is never in source code. Set it in your deployment environment (CI/CD secrets, Azure Key Vault, AWS Secrets Manager, etc.)