ThrottlingTroll 1.0.0

ThrottlingTroll

Yet another take on rate limiting/throttling in ASP.NET.

<img alt="Nuget" src="https://img.shields.io/nuget/v/ThrottlingTroll.ThrottlingTroll?label=current%20version">

Features

• Ingress throttling, aka let your service automatically respond with 429 TooManyRequests to some obtrusive clients. Implemented as an ASP.NET Core Middleware.
• Egress throttling, aka limit the number of calls your code is making against some external endpoint. Implemented as an HttpClient DelegatingHandler, which produces 429 TooManyRequests response (without making the actual call) when a limit is exceeded.
• Storing rate counters in a distributed cache, making your throttling policy consistent across all your computing instances. Both Microsoft.Extensions.Caching.Distributed.IDistributedCache and StackExchange.Redis are supported.
• Propagating 429 TooManyRequests from egress to ingress, aka when your service internally makes an HTTP request which results in 429 TooManyRequests, your service can automatically respond with same 429 TooManyRequests to its calling client.
• Dynamically configuring rate limits, so that those limits can be adjusted on-the-go, without restarting the service.

How to configure

Quick example of an ingress config setting:

  "ThrottlingTrollIngress": {
    "Rules": [
      {
        "UriPattern": "/api/values",
        "RateLimit": {
          "Algorithm": "FixedWindow",
          "PermitLimit": 5,
          "IntervalInSeconds": 10
        }
      }
    ]
  }

ThrottlingTroll's configuration (both for ingress and egress) is represented by ThrottlingTrollConfig class. It contains a list of rate limiting Rules and some other settings.

Each Rule defines a pattern that HTTP requests should match. A pattern can include the following properties (all are optional):

• UriPattern - a Regex pattern to match request URI against. Empty string or null means any URI. Note that this value is treated as a Regex, so symbols that have special meaning in Regex language must be escaped (e.g. to match a query string specify \\?abc=123 instead of ?abc=123).
• Method - request's HTTP method. E.g. POST. Empty string or null means any method.
• HeaderName - request's HTTP header to check. If specified, the rule will only apply to requests with this header set to HeaderValue.
• HeaderValue - value for HTTP header identified by HeaderName. The rule will only apply to requests with that header set to this value. If HeaderName is specified and HeaderValue is not - that matches requests with any value in that header.
• IdentityId - request's custom Identity ID. If specified, the rule will only apply to requests with this Identity ID. When specifying IdentityId you will also need to provide a custom identity extraction routine via IdentityIdExtractor setting.

If any of the above properties is empty or not specified, this means matching any request.

Then each Rule must specify the rate limiting algorithm to be applied for matched requests and its parameters.

The following algorithms are currently supported:

• FixedWindow. No more than PermitLimit requests are allowed in IntervalInSeconds. Example:

  "ThrottlingTrollIngress": {
    "Rules": [
      {
        "RateLimit": {
          "Algorithm": "FixedWindow",
          "PermitLimit": 5,
          "IntervalInSeconds": 10
        }
      }
    ]
  }

• SlidingWindow. No more than PermitLimit requests are allowed in IntervalInSeconds, but that interval is split into NumOfBuckets. The main benefit of this algorithm over FixedWindow is that if a client constantly exceedes PermitLimit, it will never get any valid response and will always get 429 TooManyRequests. Example:

  "ThrottlingTrollIngress": {
    "Rules": [
      {
        "RateLimit": {
          "Algorithm": "SlidingWindow",
          "PermitLimit": 5,
          "IntervalInSeconds": 15,
          "NumOfBuckets": 3
        }
      }
    ]
  }

Requests that should be whitelisted (exempt from the above Rules) can be specified via WhiteList property:

  "ThrottlingTrollIngress": {

    "WhiteList": [
      "/api/healthcheck",
      "api-key=my-unlimited-api-key"
    ]
  },

When using the same instance of a distributed cache for multiple different services you might also need to specify a value for UniqueName property. That value will be used as a prefix for cache keys, so that those multiple services do not corrupt each other's rate counters.

Rate limits and other settings can be configured:

• Statically, via appsettings.json.
• Programmatically, at service startup.
• Dynamically, by providing a callback, which ThrottlingTroll will periodically call to get updated values.

See more examples for all of these options below.

How to use for Ingress Throttling

To configure via appsettings.json

1. Add the following section to your config file:

  "ThrottlingTrollIngress": {

    "Rules": [
        ... here go rate limiting rules and limits...
    ],

    "WhiteList": [
        ... here go whitelisted URIs...
    ]
  },

2. Add the following call to your startup code:

app.UseThrottlingTroll();

To configure programmatically

Use the following configuration method at startup:

app.UseThrottlingTroll(options =>
{
    options.Config = new ThrottlingTrollConfig
    {
        Rules = new[]
        {
            new ThrottlingTrollRule
            {
                UriPattern = "/api/values",
                LimitMethod = new SlidingWindowRateLimitMethod
                {
                    PermitLimit = 5,
                    IntervalInSeconds = 10,
                    NumOfBuckets = 5
                }
            },

            // add more rules here...
        }
    };
});

To configure dynamically

Specify a callback that loads rate limits from some shared persistent storage and a time interval to periodically call it:

app.UseThrottlingTroll(options =>
{
    options.GetConfigFunc = async () =>
    {
        var myThrottlingRules = await LoadThrottlingRulesFromDatabase();

        return new ThrottlingTrollConfig
        {
            Rules = myThrottlingRules
        };
    };

    options.IntervalToReloadConfigInSeconds = 10;
});

NOTE: if your callback throws an exception, ThrottlingTroll will get suspended (will not apply any rules) until the callback succeeds again.

How to use for Egress Throttling

To configure via appsettings.json

1. Add the following section to your config file:

  "ThrottlingTrollEgress": {

    "Rules": [
        ... here go rate limiting rules and limits...
    ],

    "WhiteList": [
        ... here go whitelisted URIs...
    ],

    "PropagateToIngress": true
  },

2. Use the following code to configure a named HttpClient at startup:

builder.Services.AddHttpClient("my-throttled-httpclient").AddThrottlingTrollMessageHandler();

3. Get an instance of that HttpClient via IHttpClientFactory:

var throttledHttpClient = this._httpClientFactory.CreateClient("my-throttled-httpclient");

To configure programmatically

Create an HttpClient instance like this:

var myThrottledHttpClient = new HttpClient
(
    new ThrottlingTrollHandler
    (
        new ThrottlingTrollEgressConfig
        {
            Rules = new[]
            {
                new ThrottlingTrollRule
                {
                    UriPattern = "/some/external/url",
                    LimitMethod = new SlidingWindowRateLimitMethod
                    {
                        PermitLimit = 5,
                        IntervalInSeconds = 10,
                        NumOfBuckets = 5
                    }
                },
            },

            PropagateToIngress = true
        }
    )
);

NOTE: normally HttpClient instances should be created once and reused.

To configure dynamically

1. Use the following code to configure a named HttpClient at startup:

builder.Services.AddHttpClient("my-throttled-httpclient").AddThrottlingTrollMessageHandler(options =>
{
    options.GetConfigFunc = async () =>
    {
        var myThrottlingRules = await LoadThrottlingRulesFromDatabase();

        return new ThrottlingTrollConfig
        {
            Rules = myThrottlingRules
        };
    };

    options.IntervalToReloadConfigInSeconds = 10;
});

2. Get an instance of that HttpClient via IHttpClientFactory:

var throttledHttpClient = this._httpClientFactory.CreateClient("my-throttled-httpclient");

Supported Rate Counter Stores

By default ThrottlingTroll will store rate counters in memory, using MemoryCacheCounterStore (which internally uses System.Runtime.Caching.MemoryCache).

Other supported options are:

• DistributedCacheCounterStore. Uses IDistributedCache instance taken from DI container.
• RedisCounterStore. Specifically designed to work with Redis. Prefer this one over DistributedCacheCounterStore + Distributed Redis Cache.

You can also create your custom Counter Store by implementing the ICounterStore interface.

How to specify a Rate Counter Store to be used

Either put a desired ICounterStore implementation into DI container:

builder.Services.AddSingleton<ICounterStore>(
    provider => new DistributedCacheCounterStore(provider.GetRequiredService<IDistributedCache>())
);

Or provide it via UseThrottlingTroll() method:

app.UseThrottlingTroll(options =>
{
    options.CounterStore = new RedisCounterStore(app.Services.GetRequiredService<IConnectionMultiplexer>());
});

Samples

Here is a sample project, that demonstrates all the above concepts.

Contributing

Is very much welcomed.