FoundationaLLM.Client.Management 0.9.6

There is a newer prerelease version of this package available.
See the version list below for details.
dotnet add package FoundationaLLM.Client.Management --version 0.9.6
                    
NuGet\Install-Package FoundationaLLM.Client.Management -Version 0.9.6
                    
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="FoundationaLLM.Client.Management" Version="0.9.6" />
                    
For projects that support PackageReference, copy this XML node into the project file to reference the package.
<PackageVersion Include="FoundationaLLM.Client.Management" Version="0.9.6" />
                    
Directory.Packages.props
<PackageReference Include="FoundationaLLM.Client.Management" />
                    
Project file
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.
paket add FoundationaLLM.Client.Management --version 0.9.6
                    
#r "nuget: FoundationaLLM.Client.Management, 0.9.6"
                    
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
#addin nuget:?package=FoundationaLLM.Client.Management&version=0.9.6
                    
Install FoundationaLLM.Client.Management as a Cake Addin
#tool nuget:?package=FoundationaLLM.Client.Management&version=0.9.6
                    
Install FoundationaLLM.Client.Management as a Cake Tool

FoundationaLLM Management Client

The FoundationaLLM Management Client is a .NET client library that simplifies the process of interacting with the FoundationaLLM Management API. The client library provides a set of classes and methods that allow you to interact with the FoundationaLLM Management API in a more intuitive way.

This library contains two primary classes:

  • ManagementRESTClient: A class that provides a set of methods for interacting with the FoundationaLLM Management API using REST. This is considered the low-level client and provides direct access to all Management API endpoints.
  • ManagementClient: A class that provides a set of methods for interacting with the FoundationaLLM Management API using a higher-level abstraction. This class is designed to simplify the process of interacting with the Management API by providing a more intuitive interface. It does not contain all the methods available in the ManagementRESTClient class, but it provides a more user-friendly way to interact with the Management API.

These two classes are mutually exclusive, and you should choose one based on your requirements. If you need direct access to all Management API endpoints, use the ManagementRESTClient class. If you need a more user-friendly interface, use the ManagementClient class.

Getting started

If you do not have FoundationaLLM deployed, follow the Quick Start Deployment instructions to get FoundationaLLM deployed in your Azure subscription.

Install the NuGet package:

dotnet add package FoundationaLLM.Client.Management

Manual service instantiation

Complete the following steps if you do not want to use dependency injection:

  1. Create a new instance of the ManagementRESTClient and ManagementClient classes:

    var managementUri = "<YOUR_MANAGEMENT_API_URL>"; // e.g., "https://myfoundationallmmanagementapi.com"
    var instanceId = "<YOUR_INSTANCE_ID>"; // Each FoundationaLLM deployment has a unique (GUID) ID. Locate this value in the FoundationaLLM Management Portal or in Azure App Config (FoundationaLLM:Instance:Id key)
    
    var credential = new AzureCliCredential(); // Can use any TokenCredential implementation, such as ManagedIdentityCredential or AzureCliCredential.
    var options = new APIClientSettings // Optional settings parameter. Default timeout is 900 seconds.
    {
        Timeout = TimeSpan.FromSeconds(600)
    };
    
    var managementRestClient = new ManagementRESTClient(
        managementUri,
        credential,
        instanceId,
        options);
    var managementClient = new ManagementClient(
        managementUri,
        credential,
        instanceId,
        options);
    
  2. Make a request to the Management API with the ManagementRESTClient class:

    var status = await managementRestClient.Status.GetServiceStatusAsync();
    
  3. Make a request to the Management API with the ManagementClient class:

    await managementClient.DataSources.DeleteDataSourceAsync("<DATASOURCE_NAME>");
    // Purge the data source so we can reuse the name.
    await managementClient.DataSources.PurgeDataSourceAsync("<DATASOURCE_NAME>");
    

You can use the FoundationaLLM.Common.Authentication.DefaultAuthentication class to generate the TokenCredential. This class sets the AzureCredential property using the ManagedIdentityCredential when running in a production environment (production parameter of the Initialize method) and the AzureCliCredential when running in a development environment.

Example:

DefaultAuthentication.Initialize(false, "Test"); var credentials = DefaultAuthentication.AzureCredential;

Use dependency injection with a configuration file

Rather than manually instantiating the ManagementRESTClient and ManagementClient classes, you can use dependency injection to manage the instances. This approach is more flexible and allows you to easily switch between different implementations of the IManagementClient and IManagementRESTClient interfaces.

  1. Create a configuration file (e.g., appsettings.json) with the following content:

    {
        "FoundationaLLM": {
            "APIEndpoints": {
     	        "ManagementAPI": {
     	            "Essentials": {
                        "APIUrl": "https://localhost:63267/"
                    }
                }
            },
            "Instance": {
                "Id": "00000000-0000-0000-0000-000000000000"
            }
        }
    }
    
  2. Read the configuration file:

    var configuration = new ConfigurationBuilder()
        .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
        .Build();
    
  3. Use the ManagementClient extension method to add the ManagementClient and ManagementRESTClient to the service collection:

    var services = new ServiceCollection();
    var credential = new AzureCliCredential(); // Can use any TokenCredential implementation, such as ManagedIdentityCredential or AzureCliCredential.
    services.AddManagementClient(
        configuration[AppConfigurationKeys.FoundationaLLM_APIEndpoints_ManagementAPI_Essentials_APIUrl]!,
        credential,
        configuration[AppConfigurationKeys.FoundationaLLM_Instance_Id]!);
    
    var serviceProvider = services.BuildServiceProvider();
    
  4. Retrieve the ManagementClient and ManagementRESTClient instances from the service provider:

    var managementClient = serviceProvider.GetRequiredService<IManagementClient>();
    var managementRestClient = serviceProvider.GetRequiredService<IManagementRESTClient>();
    

Alternately, you can inject the ManagementClient and ManagementRESTClient instances directly into your classes using dependency injection.

public class MyService
{
    private readonly IManagementClient _managementClient;
    private readonly IManagementRESTClient _managementRestClient;

    public MyService(IManagementClient managementClient, IManagementRESTClient managementRestClient)
    {
        _managementClient = managementClient;
        _managementRestClient = managementRestClient;
    }
}

Use dependency injection with Azure App Configuration

If you prefer to retrieve the configuration settings from Azure App Configuration, you can use the Microsoft.Azure.AppConfiguration.AspNetCore or Microsoft.Extensions.Configuration.AzureAppConfiguration package to retrieve the configuration settings from Azure App Configuration.

  1. Connect to Azure App Configuration:

    var configuration = new ConfigurationBuilder()
        .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
        .AddEnvironmentVariables()
        .AddAzureAppConfiguration(options =>
        {
            options.Connect("<connection-string>");
            options.ConfigureKeyVault(kv =>
            {
                kv.SetCredential(Credentials);
            });
            options.Select(AppConfigurationKeyFilters.FoundationaLLM_Instance);
            options.Select(AppConfigurationKeyFilters.FoundationaLLM_APIEndpoints_ManagementAPI_Essentials);
        })
        .Build();
    

    If you have configured your local development environment, you can obtain the App Config connection string from an environment variable (Environment.GetEnvironmentVariable(EnvironmentVariables.FoundationaLLM_AppConfig_ConnectionString)) when developing locally.

  2. Use the ManagementClient extension method to add the ManagementClient and ManagementRESTClient to the service collection:

    var services = new ServiceCollection();
    var credential = new AzureCliCredential(); // Can use any TokenCredential implementation, such as ManagedIdentityCredential or AzureCliCredential.
    
    services.AddManagementClient(
        configuration[AppConfigurationKeys.FoundationaLLM_APIEndpoints_ManagementAPI_Essentials_APIUrl]!,
        credential,
        configuration[AppConfigurationKeys.FoundationaLLM_Instance_Id]!);
    
  3. Retrieve the ManagementClient and ManagementRESTClient instances from the service provider:

    var managementClient = serviceProvider.GetRequiredService<IManagementClient>();
    var managementRestClient = serviceProvider.GetRequiredService<IManagementRESTClient>();
    

Example projects

The Core.Examples test project contains several examples that demonstrate how to use the ManagementClient and ManagementRESTClient classes to interact with the Management API through a series of end-to-end tests.

FoundationaLLM: The platform for deploying, scaling, securing and governing generative AI in the enterprises 🚀

License

FoundationaLLM provides the platform for deploying, scaling, securing and governing generative AI in the enterprise. With FoundationaLLM you can:

  • Create AI agents that are grounded in your enterprise data, be that text, semi-structured or structured data.
  • Make AI agents available to your users through a branded chat interface or integrate the REST API to the AI agent into your application for a copilot experience or integrate the Agent API in a machine-to-machine automated process.
  • Experiment building agents that can use a variety of large language models including OpenAI GPT-4, Mistral and Llama 2 or any models pulled from the Hugging Face model catalog that provide a REST completions endpoint.
  • Centrally manage, configure and secure your AI agents AND their underlying assets including prompts, data sources, vectorization data pipelines, vector databases and large language models using the management portal.
  • Enable everyone in your enterprise to create their own AI agents. Your non-developer users can create and deploy their own agents in a self-service fashion from the management portal, but we don't get in the way of your advanced AI developers who can deploy their own orchestrations built in LangChain, Semantic Kernel, Prompt Flow or any orchestration that exposes a completions endpoint.
  • Deploy and manage scalable vectorization data pipelines that can ingest millions of documents to provide knowledge to your model.
  • Empower your users with as many task-focused AI agents as desired.
  • Control access to the AI agents and the resources they access using role-based access controls (RBAC).
  • Harness the rapidly evolving capabilities from Azure AI and Azure OpenAI from one integrated stack.

FoundationaLLM is not a large language model. It enables you to use the large language models of your choice (e.g., OpenAI GPT-4, Mistral, LLama 2, etc.)

FoundationaLLM deploys a secure, comprehensive and highly configurable copilot platform to your Azure cloud environment:

  • Simplifies integration with enterprise data sources used by agent for in-context learning (e.g., enabling RAG, CoT, ReAct and inner monologue patterns).
  • Provides defense in depth with fine-grain security controls over data used by agent and pre/post completion filters that guard against attack.
  • Hardened solution attacked by an LLM red team from inception.
  • Scalable solution load balances across multiple LLM endpoints.
  • Extensible to new data sources, new LLM orchestrators and LLMs.

Why is FoundationaLLM Needed?

Simply put we saw a lot of folks reinventing the wheel just to get a customized copilot or AI agent that was grounded and bases its responses in their own data as opposed to the trained parametric knowledge of the model. Many of the solutions we saw made for great demos, but were effectively toys wrapping calls to OpenAI endpoints- they were not something intended or ready to take into production at enterprise scale. We built FoundationaLLM to provide a continuous journey, one that was quick to get started with so folks could experiment quickly with LLM's but not fall off a cliff after that with a solution that would be insecure, unlicensed, inflexible and not fully featured enough to grow from the prototype into a production solution without having to start all over.

The core problems to deliver enterprise copilots or AI agents are:

  • Enterprise grade copilots or AI agents are complex and have lots of moving parts (not to mention infrastructure).
  • The industry has a skills gap when it comes to filling the roles needed to deliver these complex copilot solutions.
  • The top AI risks (inaccuracy, cybersecurity, compliance, explainability, privacy) are not being mitigated by individual tools.
  • Delivery of a copilot or AI agent solution is time consuming, expensive and frustrating when starting from scratch.

Documentation

Get up to speed with FoundationaLLM by reading the documentation. This includes deployment instructions, quickstarts, architecture, and API references.

Getting Started

FoundationaLLM provides a simple command line driven approach to getting your first deployment up and running. Basically, it's two commands. After that, you can customize the solution, run it locally on your machine and update the deployment with your customizations.

Follow the Quick Start Deployment instructions to get FoundationaLLM deployed in your Azure subscription.

Reporting Issues and Support

If you encounter any issues with FoundationaLLM, please open an issue on GitHub. We will respond to your issue as soon as possible. Please use the Labels (bug, documentation, general question, release x.x.x) to categorize your issue and provide as much detail as possible to help us understand and resolve the issue.

Product Compatible and additional computed target framework versions.
.NET net8.0 is compatible.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed.  net9.0 was computed.  net9.0-android was computed.  net9.0-browser was computed.  net9.0-ios was computed.  net9.0-maccatalyst was computed.  net9.0-macos was computed.  net9.0-tvos was computed.  net9.0-windows was computed.  net10.0 was computed.  net10.0-android was computed.  net10.0-browser was computed.  net10.0-ios was computed.  net10.0-maccatalyst was computed.  net10.0-macos was computed.  net10.0-tvos was computed.  net10.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
0.9.7-rc115 0 7 hours ago
0.9.7-rc114 20 15 hours ago
0.9.7-rc113 23 16 hours ago
0.9.7-rc112 20 17 hours ago
0.9.7-rc111 26 17 hours ago
0.9.7-rc110 30 a day ago
0.9.7-rc109 29 2 days ago
0.9.7-rc108 27 2 days ago
0.9.7-rc107 28 3 days ago
0.9.7-rc106 32 3 days ago
0.9.7-rc105 35 3 days ago
0.9.7-rc104 51 3 days ago
0.9.7-rc103 52 5 days ago
0.9.7-rc102 55 5 days ago
0.9.7-rc101 48 5 days ago
0.9.7-rc100 78 7 days ago
0.9.7-beta159 127 10 days ago
0.9.7-beta158 157 14 days ago
0.9.7-beta157 212 17 days ago
0.9.7-beta156 197 18 days ago
0.9.7-beta155 128 24 days ago
0.9.7-beta154 126 24 days ago
0.9.7-beta153 130 25 days ago
0.9.7-beta152 128 a month ago
0.9.7-beta151 145 a month ago
0.9.7-beta150 148 a month ago
0.9.7-beta149 150 a month ago
0.9.7-beta148 130 a month ago
0.9.7-beta147 175 a month ago
0.9.7-beta146 178 a month ago
0.9.7-beta145 92 2 months ago
0.9.7-beta144 117 2 months ago
0.9.7-beta143 127 2 months ago
0.9.7-beta142 120 2 months ago
0.9.7-beta141 110 2 months ago
0.9.7-beta140 156 2 months ago
0.9.7-beta139 153 2 months ago
0.9.7-beta138 153 2 months ago
0.9.7-beta137 142 2 months ago
0.9.7-beta136 142 2 months ago
0.9.7-beta135 140 2 months ago
0.9.7-beta134 138 2 months ago
0.9.7-beta133 144 2 months ago
0.9.7-beta132 140 2 months ago
0.9.7-beta131 150 2 months ago
0.9.7-beta130 143 2 months ago
0.9.7-beta129 152 2 months ago
0.9.7-beta128 147 2 months ago
0.9.7-beta127 138 2 months ago
0.9.7-beta126 144 2 months ago
0.9.7-beta125 457 2 months ago
0.9.7-beta124 469 2 months ago
0.9.7-beta123 462 2 months ago
0.9.7-beta122 463 2 months ago
0.9.7-beta121 461 2 months ago
0.9.7-beta120 463 2 months ago
0.9.7-beta119 474 2 months ago
0.9.7-beta118 472 2 months ago
0.9.7-beta117 485 2 months ago
0.9.7-beta116 479 2 months ago
0.9.7-beta115 395 2 months ago
0.9.7-beta114 256 2 months ago
0.9.7-beta113 76 2 months ago
0.9.7-beta112 102 2 months ago
0.9.7-beta111 141 2 months ago
0.9.7-beta110 138 2 months ago
0.9.7-beta109 139 2 months ago
0.9.7-beta108 136 2 months ago
0.9.7-beta107 136 2 months ago
0.9.7-beta106 146 2 months ago
0.9.7-beta105 142 3 months ago
0.9.7-beta104 145 3 months ago
0.9.7-beta103 147 3 months ago
0.9.7-beta102 151 3 months ago
0.9.7-beta101 204 3 months ago
0.9.7-beta100 200 3 months ago
0.9.6 202 3 months ago
0.9.6-rc100 88 3 months ago
0.9.5 108 3 months ago
0.9.5-rc102 87 3 months ago
0.9.5-rc101 82 3 months ago
0.9.5-rc100 104 3 months ago
0.9.4 105 3 months ago
0.9.3 108 3 months ago
0.9.3-rc018 93 3 months ago
0.9.3-rc017 103 4 months ago
0.9.3-rc016 97 4 months ago
0.9.3-rc015 95 4 months ago
0.9.3-rc014 89 4 months ago
0.9.3-rc013 94 4 months ago
0.9.3-rc012 102 4 months ago
0.9.3-rc011 92 4 months ago
0.9.3-rc010 95 4 months ago
0.9.3-rc009 101 4 months ago
0.9.3-rc008 98 4 months ago
0.9.3-rc007 95 4 months ago
0.9.3-rc006 89 4 months ago
0.9.3-rc005 93 4 months ago
0.9.3-rc004 91 4 months ago
0.9.3-rc003 95 4 months ago
0.9.3-rc002 88 4 months ago
0.9.3-rc001 92 4 months ago
0.9.2 106 4 months ago
0.9.2-rc007 87 4 months ago
0.9.2-rc006 95 4 months ago
0.9.2-rc005 95 4 months ago
0.9.2-rc004 95 4 months ago
0.9.2-rc003 90 4 months ago
0.9.2-rc002 90 4 months ago
0.9.2-rc001 91 4 months ago
0.9.2-a001 97 4 months ago
0.9.1 102 4 months ago
0.9.1-rc131 86 4 months ago
0.9.1-rc130 87 4 months ago
0.9.1-rc129 78 4 months ago
0.9.1-rc128 85 4 months ago
0.9.1-rc127 92 4 months ago
0.9.1-rc126 91 4 months ago
0.9.1-rc125 92 4 months ago
0.9.1-rc124 87 4 months ago
0.9.1-rc123 85 4 months ago
0.9.1-rc122 86 5 months ago
0.9.1-rc121 80 5 months ago
0.9.1-rc120 88 5 months ago
0.9.1-rc118 91 5 months ago
0.9.1-rc117 81 5 months ago
0.9.1-rc116 86 5 months ago
0.9.1-rc115 105 5 months ago
0.9.1-rc114 87 5 months ago
0.9.1-rc113 95 5 months ago
0.9.1-rc112 99 5 months ago
0.9.1-rc111 98 5 months ago
0.9.1-rc110 87 5 months ago
0.9.1-rc109 93 5 months ago
0.9.1-rc108 92 5 months ago
0.9.1-rc107 101 5 months ago
0.9.1-rc106 98 5 months ago
0.9.1-rc105 94 5 months ago
0.9.1-rc104 99 5 months ago
0.9.1-rc100 96 5 months ago
0.9.1-alpha4 106 5 months ago
0.9.1-alpha3 95 5 months ago
0.9.0-rc3 88 6 months ago
0.9.0-rc2 95 6 months ago
0.9.0-alpha5 89 6 months ago
0.9.0-alpha1 90 6 months ago
0.8.4 113 6 months ago
0.8.3 129 8 months ago
0.8.2 144 9 months ago
0.8.2-alpha2 112 8 months ago
0.8.1 146 9 months ago
0.8.1-alpha2 111 8 months ago