TabuLynx.Model.Extractor 0.7.0

TabuLynx.Model.Extractor

A powerful .NET library for extracting metadata and model structures from Analysis Services Tabular Models, including Power BI datasets, SSAS, and Microsoft Fabric. Built on top of TabuLynx.Query.Executor for robust DMV (Dynamic Management Views) querying capabilities.

✨ Features

• Comprehensive Model Extraction: Extract complete tabular model metadata including tables, columns, measures, relationships, and hierarchies
• DMV-Based Extraction: Leverages Analysis Services Dynamic Management Views for accurate metadata retrieval
• Rich Model Objects: Strongly-typed model classes representing all aspects of tabular models
• LLM-Optimized Models: Generate AI/LLM-friendly representations of your data models
• JSON Serialization: Built-in JSON export capabilities for model metadata
• Dependency Injection Ready: Seamless integration with .NET dependency injection
• Multi-Platform Support: Works with Power BI, SSAS, and Microsoft Fabric

📦 Installation

dotnet add package TabuLynx.Model.Extractor

🏗️ Dependencies

This package requires and builds upon:

• TabuLynx.Query.Executor - Core query execution engine using ADOMD.NET
• TabuLynx.Core - Core TabuLynx functionality and interfaces
• .NET 8.0 - Latest .NET runtime

Important: TabuLynx.Model.Extractor depends on TabuLynx.Query.Executor to execute DMV queries against Analysis Services. You must configure a query executor before using the model extractor.

🚀 Quick Start

1. Basic Setup with Local Power BI

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using TabuLynx.Core.Configuration;
using TabuLynx.Query.Executor;
using TabuLynx.Model.Extractor;
using TabuLynx.Model.Extractor.Interfaces;

var host = Host.CreateDefaultBuilder(args)
    .ConfigureServices((context, services) =>
    {
        // Configure TabuLynx options
        services.Configure<TabuLynxOptions>(context.Configuration.GetSection("TabuLynx"));
        
        // Add query executor (required dependency)
        services.AddAdomdQueryExecutorForLocalPowerBI();
        
        // Add model extractor
        services.AddDmvModelExtractor();
    })
    .Build();

var modelExtractor = host.Services.GetRequiredService<IModelExtractor>();
var model = await modelExtractor.ExtractModelAsync();

Console.WriteLine($"Extracted model with {model.Tables.Count} tables");

2. Configuration (appsettings.json)

{
  "TabuLynx": {
    "ConnectionString": "Provider=MSOLAP;Data Source=localhost:{port};"
  }
}

3. ASP.NET Core Integration

using TabuLynx.Core.Configuration;
using TabuLynx.Query.Executor;
using TabuLynx.Model.Extractor;
using TabuLynx.Model.Extractor.Interfaces;

var builder = WebApplication.CreateBuilder(args);

// Configure services
builder.Services.Configure<TabuLynxOptions>(
    builder.Configuration.GetSection("TabuLynx"));

// Add query executor (choose based on your target platform)
builder.Services.AddAdomdQueryExecutorForLocalPowerBI();
// OR: builder.Services.AddAdomdQueryExecutorForSsasServer();
// OR: builder.Services.AddAdomdQueryExecutorForFabric();

// Add model extractor
builder.Services.AddDmvModelExtractor();

var app = builder.Build();

app.MapGet("/model", async (IModelExtractor extractor) =>
{
    var model = await extractor.ExtractModelAsync();
    return Results.Ok(model);
});

app.MapGet("/model/json", async (IModelExtractor extractor) =>
{
    var model = await extractor.ExtractModelAsync();
    return Results.Content(model.ToJson(), "application/json");
});

app.Run();

📊 Model Structure

The extracted TabularModel contains:

Tables and Columns

var model = await modelExtractor.ExtractModelAsync();

foreach (var table in model.Tables)
{
    Console.WriteLine($"Table: {table.Name}");
    Console.WriteLine($"  Description: {table.Description}");
    Console.WriteLine($"  Hidden: {table.IsHidden}");
    Console.WriteLine($"  Columns: {table.Columns.Count}");
    
    foreach (var column in table.Columns)
    {
        Console.WriteLine($"    - {column.Name} ({column.DataType})");
        if (column.IsKey) Console.WriteLine("      [Key Column]");
        if (!string.IsNullOrEmpty(column.Expression)) 
            Console.WriteLine($"      Expression: {column.Expression}");
    }
}

Measures

foreach (var measure in model.Measures)
{
    Console.WriteLine($"Measure: {measure.Name}");
    Console.WriteLine($"  Expression: {measure.Expression}");
    Console.WriteLine($"  Table: {measure.TableName}");
    Console.WriteLine($"  Hidden: {measure.IsHidden}");
}

Relationships

foreach (var relationship in model.Relationships)
{
    Console.WriteLine($"Relationship: {relationship.Name}");
    Console.WriteLine($"  From: {relationship.FromTable}.{relationship.FromColumn}");
    Console.WriteLine($"  To: {relationship.ToTable}.{relationship.ToColumn}");
    Console.WriteLine($"  Type: {relationship.CrossFilteringBehavior}");
}

🔧 Advanced Usage

Export to JSON

var model = await modelExtractor.ExtractModelAsync();

// Export full model to JSON
var jsonModel = model.ToJson(indented: true);
await File.WriteAllTextAsync("model.json", jsonModel);

// Create LLM-optimized version
var llmModel = model.ToLlmTabularModel(new TabularModel.LlmTabularModelOptions
{
    IncludeHiddenTables = false,
    IncludeHiddenColumns = false,
    IncludeHiddenMeasures = false
});
var llmJson = llmModel.ToJson();
await File.WriteAllTextAsync("llm-model.json", llmJson);

Custom Model Analysis

public class ModelAnalyzer
{
    private readonly IModelExtractor _modelExtractor;

    public ModelAnalyzer(IModelExtractor modelExtractor)
    {
        _modelExtractor = modelExtractor;
    }

    public async Task<ModelStats> AnalyzeModelAsync()
    {
        var model = await _modelExtractor.ExtractModelAsync();
        
        return new ModelStats
        {
            TotalTables = model.Tables.Count,
            VisibleTables = model.Tables.Count(t => !t.IsHidden),
            TotalColumns = model.Tables.SelectMany(t => t.Columns).Count(),
            CalculatedColumns = model.CalculatedColumns.Count,
            TotalMeasures = model.Measures.Count,
            VisibleMeasures = model.Measures.Count(m => !m.IsHidden),
            Relationships = model.Relationships.Count,
            Hierarchies = model.Hierarchies.Count
        };
    }
}

public class ModelStats
{
    public int TotalTables { get; set; }
    public int VisibleTables { get; set; }
    public int TotalColumns { get; set; }
    public int CalculatedColumns { get; set; }
    public int TotalMeasures { get; set; }
    public int VisibleMeasures { get; set; }
    public int Relationships { get; set; }
    public int Hierarchies { get; set; }
}

Multi-Environment Configuration

public void ConfigureServices(IServiceCollection services, IConfiguration configuration)
{
    var environment = configuration["Environment"];
    
    services.Configure<TabuLynxOptions>(configuration.GetSection("TabuLynx"));
    
    // Configure query executor based on environment
    switch (environment?.ToLower())
    {
        case "development":
            services.AddAdomdQueryExecutorForLocalPowerBI();
            break;
        case "staging":
            services.AddAdomdQueryExecutorForSsasServer();
            break;
        case "production":
            services.AddAdomdQueryExecutorForFabric();
            break;
    }
    
    // Add model extractor (works with any query executor)
    services.AddDmvModelExtractor();
}

🔍 DMV Queries Used

The model extractor uses the following Analysis Services DMV queries to extract metadata:

• $SYSTEM.TMSCHEMA_TABLES - Table definitions
• $SYSTEM.TMSCHEMA_COLUMNS - Column definitions
• $SYSTEM.TMSCHEMA_MEASURES - Measure definitions
• $SYSTEM.TMSCHEMA_RELATIONSHIPS - Relationship definitions
• $SYSTEM.TMSCHEMA_HIERARCHIES - Hierarchy definitions

🎯 Use Cases

Business Intelligence Documentation

// Generate model documentation
var model = await modelExtractor.ExtractModelAsync();
var documentation = GenerateModelDocumentation(model);

Data Catalog Integration

// Extract metadata for data catalog
var model = await modelExtractor.ExtractModelAsync();
var catalogEntries = model.Tables.Select(t => new CatalogEntry
{
    Name = t.Name,
    Type = "Table",
    Description = t.Description,
    Columns = t.Columns.Select(c => c.Name).ToList()
});

AI/ML Model Training

// Create LLM-friendly model representation
var model = await modelExtractor.ExtractModelAsync();
var llmModel = model.ToLlmTabularModel();
var prompt = $"Analyze this data model: {llmModel.ToJson()}";

🛠️ Troubleshooting

Common Issues

Missing Query Executor: Ensure you've registered a query executor service before adding the model extractor:

// Required: Add query executor first
services.AddAdomdQueryExecutorForLocalPowerBI();

// Then: Add model extractor
services.AddDmvModelExtractor();

DMV Access Denied: Verify that your connection has permissions to query system DMVs. This typically requires:

• Read permissions on the Analysis Services database
• Access to system catalog views

Empty Model Results: Ensure the target Analysis Services instance is running and accessible via the configured connection string.

🤝 Contributing

Contributions, issues, and feature requests are welcome! Please feel free to check the issues page.

📄 License

This project is licensed under the terms specified in the LICENSE file.