Aerx.QdrantClient.Http 1.17.10

.NET SDK for Qdrant vector database HTTP API

.NET SDK for Qdrant vector database.

Getting started

Creating a client

A client that will connect to Qdrant HTTP API on http://localhost:6334 can be instantiated as follows

var client = new QdrantHttpClient("localhost");

Additional constructor overloads provide more control over how the client is configured. The following example configures a client to use the different Qdrant port and an api key:

var client = new QdrantHttpClient(
    httpAddress: "localhost", 
    port: 1567, 
    apiKey : "my-secret-api-key"
);

Dependency Injection

To register Qdrant HTTP client in the dependency injection container use the following code:

services.AddQdrantHttpClient(options =>
{
    options.HttpAddress = "localhost";
    options.Port = 6334;
    options.ApiKey = "my-secret-api-key";
});

This will register QdrantHttpClient as a singleton service.

If you wish to register IQdrantHttpClient instead (supported from version 1.15.13 of this library), pass the registerAsInterface argument to the AddQdrantHttpClient method:

services.AddQdrantHttpClient(options =>
    {
        options.HttpAddress = "localhost";
        options.Port = 6334;
        options.ApiKey = "my-secret-api-key";
    },
    registerAsInterface: true
);

Register multiple clients

To register multiple Qdrant clients in the dependency injection container use named clients feature:

services.AddQdrantHttpClient(
    Configuration,
    clientConfigurationSectionName: "ConfigSecrtion_1",
    clientName: "My client 1"
);

services.AddQdrantHttpClient(
    Configuration,
    clientConfigurationSectionName: "ConfigSecrtion_2",
    clientName: "My client 2"
);

Note that you can use different configuration sections for each client.

Then inject IQdrantHttpClientFactory and create clients by name:

public class MyService
{
    private readonly IQdrantHttpClient _qdrantHttpClient;

    public MyService(IQdrantHttpClientFactory qdrantHttpClientFactory)
    {
        _qdrantHttpClient = qdrantHttpClientFactory.CreateClient("My client 1");
    }
}

Be sure to call services.AddQdrantHttpClient for each named client and use the same clientName when creating the client from the factory.

!!! When using multiple named clients feature don't inject IQdrantHttpClient directly as it will lead to ambiguity. !!!

Observability: metrics and tracing

Tracing and compression in the constructor

When creating a QdrantHttpClient directly you can control tracing and compression via constructor parameters:

var client = new QdrantHttpClient(
    host: "localhost",
    port: 6334,
    apiKey: "my-secret-api-key",
    disableTracing: false,    // set to true to disable underlying HTTP client and Qdrant-specific activity tracing
    enableCompression: true,  // set to true to enable gzip/deflate request-response compression
    tracer: myTracer          // optional OpenTelemetry Tracer instance. If not passed telemetry spans won't be written
);

The same parameters are available on the Uri-based overload:

var client = new QdrantHttpClient(
    httpAddress: new Uri("http://localhost:6334"),
    apiKey: "my-secret-api-key",
    disableTracing: false,
    enableCompression: true,
    tracer: myTracer
);

Tracing and metrics in QdrantClientSettings

When using dependency injection configure the following properties on QdrantClientSettings:

services.AddQdrantHttpClient(options =>
{
    options.HttpAddress = "http://localhost:6334";
    options.ApiKey = "my-secret-api-key";
    options.DisableTracing = false;
    options.DisableMetrics = false;
    options.EnableCompression = true;
});

Enabling diagnostics (metrics listener)

Call EnableQdrantHttpClientDiagnostics on the application builder to wire up the OpenTelemetry metrics listener. Metrics are not written when QdrantClientSettings.DisableMetrics is true.

app.EnableQdrantHttpClientDiagnostics(); // uses the default client name

// or, for a named client:
app.EnableQdrantHttpClientDiagnostics(clientName: "My client 1");

OpenTelemetry metrics

The client publishes the following OpenTelemetry metrics under the meter Aer.QdrantClient.Http.Metrics:

Register the meter when configuring your OpenTelemetry pipeline:

services.AddOpenTelemetry()
    .WithMetrics(builder =>
    {
        builder.AddMeter(QdrantHttpClientDiagnosticConstants.MetricsMeterName);
    });

OpenTelemetry tracing

Client spans are published to the activity source Aer.QdrantClient.Http. Each span is named qdrant.http <MethodName> and carries the following attributes:

Register the source when configuring your OpenTelemetry pipeline:

services.AddOpenTelemetry()
    .WithTracing(builder =>
    {
        builder.AddSource(QdrantHttpClientDiagnosticConstants.TracingActivitySourceName);
    });

Tracing only emits spans when there is an active parent Activity in the current execution context. If disableTracing / DisableTracing is true or no Tracer instance is provided, tracing is skipped entirely.

Using IQdrantClientFactory with tracing and compression

IQdrantClientFactory exposes AddClientConfiguration and CreateClient overloads that accept the same disableTracing, enableCompression, and tracer parameters:

Register the factory:

services.AddQdrantClientFactory();

Add client configurations at runtime and create clients:

public class MyService
{
    private readonly IQdrantHttpClient _client;

    public MyService(IQdrantClientFactory factory)
    {
        factory.AddClientConfiguration(
            clientName: "my-client",
            host: "localhost",
            port: 6334,
            apiKey: "my-secret-api-key",
            disableTracing: false,
            enableCompression: true,
            tracer: myTracer
        );

        _client = factory.CreateClient("my-client");
    }
}

You can also create a one-off client directly from IQdrantClientFactory without pre-registering it:

var client = factory.CreateClient(
    httpAddress: new Uri("http://localhost:6334"),
    apiKey: "my-secret-api-key",
    disableTracing: false,
    enableCompression: true,
    tracer: myTracer
);

To obtain a raw HttpClient pre-configured for Qdrant (useful when overriding the default transport):

var httpClient = factory.CreateApiClient(
    httpAddress: new Uri("http://localhost:6334"),
    apiKey: "my-secret-api-key",
    disableTracing: false,
    enableCompression: true
);

Working with collections

Once a client has been created, create a new collection

var collectionCreationResult = await _qdrantHttpClient.CreateCollection(
    "my_collection",
    new CreateCollectionRequest(VectorDistanceMetric.Dot, vectorSize: 100, isServeVectorsFromDisk: true)
    {
        OnDiskPayload = true
    },
    cancellationToken
);

Insert vectors into a collection

var upsertPoints = Enumerable.Range(0, 100).Select(
    i => new UpsertPointsRequest.UpsertPoint(
        PointId.Integer((ulong) i),
        Enumerable.Range(0, 128)
            .Select(_ => float.CreateTruncating(Random.Shared.NextDouble()))
            .ToArray(),
        new TestPayload()
        {
            Integer = 123,
            Text = "test"
        }
    )
).ToList();

var upsertPointsResult = await _qdrantHttpClient.UpsertPoints(
    "my_collection",
    new UpsertPointsRequest()
    {
        Points = upsertPoints
    },
    cancellationToken
);

Search for similar vectors

var queryVector = Enumerable.Range(0, 128)
    .Select(_ => float.CreateTruncating(Random.Shared.NextDouble()))
    .ToArray()

// return the 5 closest points
var searchResult = await _qdrantHttpClient.SearchPoints(
    "my_collection",
    new SearchPointsRequest(
        queryVector,
        limit: 5)
    {
        WithVector = false,
        WithPayload = PayloadPropertiesSelector.All
    },
    cancellationToken
);

Search for similar vectors with filtering condition

 var searchResult = await _qdrantHttpClient.SearchPoints(
    "my_collection",
    new SearchPointsRequest(
        queryVector,
        limit: 5)
    {
        WithVector = false,
        WithPayload = PayloadPropertiesSelector.All,
        Filter = 
            Q.Must(
                Q.BeInRange("int_field", greaterThanOrEqual: 0)
            )
            +
            !(
                Q.MatchValue("int_field_2", 1567)
                &
                Q.MatchValue("text_field", "test")
            )
    },
    cancellationToken
);

Search for similar vectors with filtering condition on typed payload

Here we are using typed builders for building filters for typed payload.

 var searchResult = await _qdrantHttpClient.SearchPoints(
    "my_collection",
    new SearchPointsRequest(
        queryVector,
        limit: 5)
    {
        WithVector = false,
        WithPayload = PayloadPropertiesSelector.All,
        Filter = 
            Q.Must(
                Q<TestPayload>.BeInRange(p => p.Integer, greaterThanOrEqual: 0)
            )
            |
            !Q.Must(
                Q<TestPayload>.MatchValue(p => p.Integer, 1)
            )
    },
    cancellationToken
);

Building collections

Conditions are built using Q (from Qdrant or Query) and Q<TPayload> condition builders. Top level filter should contain only Must, MustNot or Should condition groups. Result if any call on Q or Q<TPayload> is implicitly convertible to QdrantFilter, that is accepted everywhere the filter is expected, for ease of use. QdrantFilter can be created directly using QdrantFilter.Create() factory method.

Non-generic condition builder Q

Non-generic condition builder methods other than Must, MustNot, Should and Nested accept string payload field name as the first parameter.

Q.Must(
    Q.MatchValue("test_key", 123),
    Q.MatchValue("test_key_2", "test_value")
)

Filters can be nested.

Q.Should(
    Q.MustNot(Q.MatchValue("test_key", 123)),
    Q.MatchValue("test_key_2", "test_value")
)

Generic condition builder Q<T>

If the type of the payload is known beforehand the generic version of condition builder Q<TPayload> can be used to avoid typos in payload field names. Q<TPayload> has the same methods but with payload field selector expression as the first parameter.

If the payload is as defined as follows

public class TestPayload : Payload
{
    public string Text { get; set; }

    [JsonPropertyName("int")]
    public int? IntProperty { get; set; }

    public NestedClass Nested { get; set; }

    public class NestedClass
    {
        public string Name { get; set; }

        public double Double { set; get; }

        public int Integer { get; set; }
    }
}

Filter can access its structure to derive the payload filed names. Property renames in payload json are supported through standard JsonPropertyNameAttribute. Property nesting is also suppoerted in this case json dot-notation path will be constructed.

Q.Should(
    Q<TestPayload>.MatchValue(p=>p.IntProperty, 123),
    Q<TestPayload>.MatchValue(p=>p.Nested.Name, "test_value")
)

Filter combination operators

In addition to combining filters explicitly, the more terse combination is possible through the use of operators + , |, & and !.

• + - combines top level filter conditions to simplify filter building.

  Q.Must(
      Q.BeInRange("int_field", greaterThanOrEqual: 0)
  )
  +
  Q.MustNot(
      Q.MatchValue("int_field_2", 1567)
  )
  

  Which is equivalent to the following filter json

  {
      "must": [
          {
              "key": "int_field",
              "range": {
                  "gte": 0
              }
          }
      ],
      "must_not": [
          {
              "key": "int_field_2",
              "match": {
                  "value": 1567
              }
          }
      ]
  }
  

• | combines two conditions using Should condition group. Nested Should groups are automatically unwrapped.

  Q.Must(
      Q.BeInRange("int_field", greaterThanOrEqual: 0)
  )
  |
  Q.Should(
      Q.MatchValue("int_field_2", 1567)
  )
  

  Which is equivalent to the following filter json

  {
      "should": [
          {
              "must": [
                  {
                      "key": "int_field",
                      "range": {
                          "gte": 0
                      }
                  }
              ]
          },
          {
              "key": "int_field_2",
              "match": {
                  "value": 1567
              }
          }
      ]
  }
  

• & combines two conditions using Must condition group. Nested Must groups are automatically unwrapped.

  Q.MustNot(
      Q.BeInRange("int_field", greaterThanOrEqual: 0)
  )
  &
  Q.Must(
      Q.MatchValue("int_field_2", 1567)
  )
  

  Which is equivalent to the following filter json

  {
      "must": [
          {
              "must_not": [
                  {
                      "key": "int_field",
                      "range": {
                          "gte": 0
                      }
                  }
              ]
          },
          {
              "key": "int_field_2",
              "match": {
                  "value": 1567
              }
          }
      ]
  }
  

• ! wraps condition in MustNot condition group. Negates nested Must and MustNot condition groups.

  Q.Should(
      Q<TestPayload>.MatchValue(p=>p.IntProperty, 123),
      !Q<TestPayload>.MatchValue(p=>p.Nested.Name, "test_value")
  )
  +
  !Q.Must(
      Q<TestPayload>.MatchValue(p=>p.IntProperty, 345)
  )
  

  Which is equivalent to the following filter json

  {
      "should": [
          {
              "key": "int_property",
              "match": {
                  "value": 123
              }
          },
          {
              "must_not": [
                  {
                      "key": "nested.name",
                      "match": {
                          "value": "test_value"
                      }
                  }
              ]
          }
      ],
      "must_not": [
          {
              "key": "int_property",
              "match": {
                  "value": 345
              }
          }
      ]
  }