Advanced API Guardrails: Implementing Rate Limiting, Distributed Caching, and Circuit Breakers in .NET Core

Three days before a holiday sale, your e-commerce API crashes. Not because of a bug in your business logic, not because of slow queries, but because you got hammered with more traffic than you’ve ever seen. Your thread pool exhausted. Your database connection pool dried up. Your cloud bill spiked. Your on-call engineer’s phone wouldn’t stop ringing.

The code itself was fine. The database was fine. What failed was the lack of guardrails. This is where Advanced API Guardrails come in — the production-grade defenses that stop a traffic spike from becoming an outage.

This post walks through three production-grade defenses that the modern .NET ecosystem gives you natively—no custom wheel-reinvention required. We’ll cover native rate limiting, distributed caching that won’t stampede your database, and circuit breakers that stop cascading failures before they happen. More importantly, we’ll talk about why you need each one, what breaks when you don’t have it, and how to actually implement it in a way that survives contact with production. Knowing what’s already built into your Microsoft solutions stack before reaching for a third-party tool saves both time and operational complexity.

1. Traffic Control: Native Rate Limiting in ASP.NET Core

The Production Pain Point

The first layer of Advanced API Guardrails is traffic control: native rate limiting in ASP.NET Core. A few years back, we had an API endpoint that calculated shipping costs. Simple operation—look up package dimensions, get carrier rates, return a number. We didn’t rate-limit it.

Then Black Friday hit. A competitor’s website went down (definitely not us), and suddenly their traffic diverted to every other e-commerce platform including ours. Our API started getting 500 requests per second from a single IP. Not bots—just traffic. Our app server was fine with the throughput. The database was fine. But something broke: the thread pool.

ASP.NET Core runs a thread pool. Each HTTP request grabs a thread. Each database query grabs a connection. Under sustained load, you hit saturation. Requests start queuing. New requests arrive faster than threads can finish. The queue grows. Eventually, the thread pool gives up and throws ThreadStateException. Your API returns 503s. Users see errors. Ops pages you at 3 AM.

We could have written a custom middleware with a thread-safe dictionary tracking request counts per IP. But that’s in-memory only—if you run two server instances behind a load balancer, each instance has its own counter. An attacker can round-robin requests across instances and bypass your 100-req-per-minute limit entirely. Or worse, your in-memory counter grows unbounded, consuming gigabytes of RAM by end-of-day, until the GC freaks out and you spike latency to 50 seconds.

This is the itch that native rate limiting scratches.

The Modern Solution: Built Into .NET 7+

Since .NET 7, ASP.NET Core ships with Microsoft.AspNetCore.RateLimiting—a first-class middleware that understands multiple rate-limiting algorithms and gives you fine-grained control over rejection behavior. It doesn’t require Redis (though you can wire it up if you need distributed state). It’s efficient, it’s standardized, and it’s designed for the patterns that actually appear in production.

Here’s what you’re getting under the hood:

Algorithm Choices: Fixed window, sliding window, token bucket, or concurrency limits. Each one has different performance characteristics and fairness properties. You pick based on your use case, not because someone on the internet said so.
Partitioning: You can slice your rate limits by API key, user ID, IP address, or any other dimension that makes sense for your domain. An authenticated user gets 10,000 reqs/day, an unauthenticated visitor gets 100.
Standardized Rejection: When a request exceeds quota, the middleware returns HTTP 429 with a Retry-After header that tells the client when they can retry. No custom hand-rolled rejection logic that breaks client libraries.
Lease Metadata: The rate limiter assigns a “lease” to each request. You can inspect that lease to get the retry time, remaining quota, and other useful telemetry.

Choosing the Right Algorithm

Before you configure anything, understand what you’re optimizing for:

Fixed Window: The simplest algorithm. “Allow 100 requests per 10-second window.” The window resets every 10 seconds on the clock. If I make 100 requests at second 9, and you’re counting per 10-second window starting at second 0, then my next request (at second 10) gets a fresh window to burn.

When to use: Internal APIs, background job triggers, cron-style webhooks where you know traffic will be bursty and synchronized.
Why it works: Trivial to implement, minimal memory overhead.
The catch: Vulnerable to boundary surge. If the window resets at 9:00:00 and again at 9:00:10, a client can send 200 requests in 5 milliseconds (100 at 9:00:09.999, then the window resets, 100 at 9:00:10.000). You’ve allowed twice your intended throughput in a short burst.

Sliding Window: A fancier version. “Allow 100 requests in any 10-second window.” If I made 60 requests in the past 8 seconds, I can only make 40 more before 10 seconds elapse.

When to use: Sensitive operations like login attempts, OTP resets, password changes. You want a truly rolling limit that can’t be gamed.
Why it works: No boundary surge. The limit is always enforced over the most recent N seconds.
The catch: More memory overhead (you track timestamps of recent requests, not just a counter). And it’s stricter—clients can’t burst even briefly without hitting quota.

Token Bucket: The workhorse of public APIs. Imagine a bucket that holds 100 tokens. Every 10 seconds, you add 20 tokens (refill rate). Each request consumes 1 token. If the bucket is full, you’ve got a burst capacity—you can make 100 requests immediately. After that, you refill at 20 requests per 10 seconds. If you make more requests than the refill rate, you wait.

When to use: Public REST APIs, third-party integrations, any scenario where you want to allow short-term bursts but enforce long-term averages.
Why it works: Balances fairness (long-term throughput is capped) with burstiness (short-term peaks are tolerated).
The catch: Requires state tracking (current token count per partition). Not suitable for extremely high-cardinality partitions (like per-user limits with a million users) unless you’re willing to pay the memory cost.

Concurrency Limiter: Not strictly a rate limit—it’s a count of concurrent requests. “Allow at most 5 PDF generation requests to run at the same time.” When one finishes, another queued request starts.

When to use: CPU-intensive or resource-intensive operations (file processing, report generation, image transforms).
Why it works: Prevents thread pool starvation and resource exhaustion on expensive operations.
The catch: Doesn’t protect against short, quick operations that just spam the endpoint. A concurrency limiter on a fast database query is less useful than token bucket.

Production Implementation: Token Bucket with Intelligent Rejection

Let’s build a rate limiter that handles real-world requirements:

Authenticated users (identified by API key or bearer token) get a higher quota.
Unauthenticated requests (or those from suspicious IPs) get a lower quota.
When a request is rejected, return 429 with a proper Retry-After header and a JSON error response.
Log circuit-breaker trips to your observability stack.

// Program.cs

builder.Services.AddRateLimiter(options =>
{
// Return 429 (Too Many Requests) when quota exceeded
options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;

// Custom rejection handler: return structured error + Retry-After
options.OnRejected = async (context, cancellationToken) =>
{
context.HttpContext.Response.ContentType = “application/json”;

// Try to extract Retry-After from the lease
if (context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
{
context.HttpContext.Response.Headers.RetryAfter =
((int)retryAfter.TotalSeconds).ToString();
}

// Return a structured error (RFC 7807 Problem Details)
var problemDetails = new ProblemDetails
{
Status = StatusCodes.Status429TooManyRequests,
Title = “Rate Limit Exceeded”,
Detail = “You have exceeded your rate limit quota. Please retry after the time specified in the Retry-After header.”,
Instance = context.HttpContext.Request.Path
};

await context.HttpContext.Response.WriteAsJsonAsync(problemDetails, cancellationToken);
};

// Define rate limit policies
options.AddPolicy(“PublicApiPolicy”, httpContext =>
{
// Extract the partition key: API key, user ID, or IP address
string? partitionKey = null;

// First, try to get an authenticated API key or user ID
if (httpContext.User?.FindFirst(“sub”)?.Value is { } userId)
{
// Authenticated user: higher limit
partitionKey = $”user-{userId}”;
}
else if (httpContext.Request.Headers[“X-Api-Key”] is { } apiKey && !string.IsNullOrEmpty(apiKey))
{
// API key auth: moderate limit
partitionKey = $”key-{apiKey}”;
}
else
{
// Fall back to IP address for anonymous requests: strict limit
partitionKey = $”ip-{httpContext.Connection.RemoteIpAddress}”;
}

// Return a token bucket limiter configured per partition
return RateLimitPartition.GetTokenBucketLimiter(partitionKey, _ =>
new TokenBucketRateLimiterOptions
{
TokenLimit = 100, // Burst capacity
QueueLimit = 10, // Allow 10 waiting requests
ReplenishmentPeriod = TimeSpan.FromSeconds(1),
TokensPerPeriod = 20, // Refill: 20 tokens/second = 1200/min
AutoReplenishment = true
});
});

// For admin/internal endpoints, use a more generous policy
options.AddPolicy(“AdminApiPolicy”, httpContext =>
{
string partitionKey = httpContext.User?.FindFirst(“sub”)?.Value
?? “anonymous-admin”;

return RateLimitPartition.GetTokenBucketLimiter(partitionKey, _ =>
new TokenBucketRateLimiterOptions
{
TokenLimit = 1000,
QueueLimit = 50,
ReplenishmentPeriod = TimeSpan.FromSeconds(1),
TokensPerPeriod = 500,
AutoReplenishment = true
});
});
});

var app = builder.Build();

// Middleware must be registered *before* routing/endpoints
app.UseRateLimiter();

app.MapGet(“/api/products”, GetProducts).RequireRateLimiting(“PublicApiPolicy”);
app.MapGet(“/admin/stats”, GetStats).RequireRateLimiting(“AdminApiPolicy”);

Identifying which traffic counts as suspicious in the first place is where regular security testing services earn their keep, feeding directly into how you tune these quotas.

Architectural Decisions and Trade-Offs

When do you need distributed rate limiting (e.g., with Redis)?

The built-in rate limiter is in-process. If you have two app servers behind a load balancer, each server tracks its own quota independently. For most public APIs, this is fine—the attacker’s requests get distributed naturally, and you get built-in burst tolerance.

But if you need strict global rate limiting (e.g., “exactly 100 requests per minute across all servers”), you’ll need to wire Redis into a custom policy. This adds latency and operational complexity. Most teams don’t need this.

Token bucket parameters: how tight should I make it?

Start conservative. If you over-throttle, users complain. If you under-throttle, you get the 3 AM page. We typically run:

TokenLimit = burst capacity * 1.5 (allow 50% overshoot in a tight burst)
TokensPerPeriod = expected sustained throughput
ReplenishmentPeriod = 1 second (one-second granularity is fine for most use cases)

Then monitor actual traffic patterns for a week. Adjust based on what you see. Some endpoints need stricter limits (auth, expensive operations). Others are cheap enough to be generous. That kind of ongoing tuning — watching real traffic and adjusting thresholds over time — is exactly what dedicated maintenance and support services are built to handle, rather than a one-time configuration.

Queue limits: should I queue rejected requests?

Yes, but conservatively. QueueLimit = 10 means if the server is at capacity, it’ll queue up to 10 more requests before rejecting the next one. This is useful for legitimate traffic spikes. But don’t set it too high or you’ll just delay the inevitable failure. Keep it at 5-15 depending on your request latency.

2. Shielding the Database: Distributed Caching Without the Stampede

The Production Pain Point

The second pillar of Advanced API Guardrails is shielding your database — distributed caching without the stampede. We had a product catalog endpoint. Thousands of concurrent users, each fetching product details. We were caching aggressively—30-minute TTL on every product.

Then a cache key expired at 2 PM on a Tuesday. 500 concurrent requests hit that key within the same millisecond (think cache stampede). Each one saw a cache miss. Each one issued a SELECT to SQL Server. The database got 500 identical queries at once.

Your connection pool has, say, 50 connections. These 500 queries get queued. They sit in the queue. Slow. Your application’s request thread pool also starts to starve (each thread is blocked waiting for a database response). New HTTP requests arrive. No threads available. Everything backs up.

The stampede lasted 45 seconds. During that time, the product endpoint latency spiked from 50ms to 8 seconds. Some clients gave up. Some retried (making it worse). One of your internal dashboards that polled this endpoint every 10 seconds got stuck waiting for responses. That dashboard’s thread pool got saturated too. It stopped updating. Some other internal service that depended on that dashboard’s health checks went down.

One expired cache key. Cascading failures everywhere.

The old workaround was to lock before cache misses: “Only one thread rebuilds the cache; others wait.” But rolling your own synchronization across HTTP requests is fragile. You need a distributed lock if you’re running multiple app servers. That’s complex. And if you mess up the timeout on the lock, you can deadlock the whole system.

The Modern Solution: HybridCache

.NET 8 introduced HybridCache, which elegantly solves this. It’s a two-tier caching strategy:

L1 (in-process): Fast, local IMemoryCache. Every instance has its own. Zero latency.
L2 (distributed): Shared IDistributedCache (usually Redis). Consistent across all instances. Slightly higher latency, but not bad.

Getting this distributed layer right — Redis sizing, failover, cross-region consistency — is exactly the kind of infrastructure work that falls under cloud application development services.

The key innovation: asynchronous locking on cache miss. When a cache key expires and multiple threads hit it simultaneously, HybridCache ensures that only one thread fetches the fresh data. The others wait for the result without hammering the data source.

Here’s how it works conceptually:

Thread A requests cache-key-123. Miss (cache is empty or expired).
Thread A acquires an async lock on cache-key-123.
Threads B, C, D also request cache-key-123 simultaneously. They see the miss and the lock. They wait.
Thread A fetches the data from SQL Server (the expensive operation).
Thread A stores the result in both L1 and L2 (local memory and Redis).
Threads B, C, D wake up. They hit L1 cache (now warm). Instant return.

This is called singleflight or request coalescing in distributed systems literature. It’s the right solution to the stampede problem.

Setting Up HybridCache

First, add the package:

dotnet add package Microsoft.Extensions.Caching.Hybrid

Then configure it in Program.cs:

builder.Services.AddHybridCache(options =>
{
// Limit the size of each cached entry to prevent memory exhaustion
options.MaximumPayloadBytes = 1024 * 1024; // 1 MB per entry

// Default expiration for all cached entries
options.DefaultEntryOptions = new HybridCacheEntryOptions
{
// How long before the entry expires globally (in Redis)
Expiration = TimeSpan.FromMinutes(30),

// How long to keep the entry in local L1 memory after it expires in L2
// This prevents constant Redis hits during the “grace period”
LocalCacheExpiration = TimeSpan.FromMinutes(5)
};
});

// If you want Redis as your L2 store, add it:
// builder.Services.AddStackExchangeRedisCache(options =>
// {
// options.Configuration = builder.Configuration.GetConnectionString(“Redis”);
// });

A few notes on the configuration:

MaximumPayloadBytes: HybridCache serializes entries to JSON before storing in L2. Large payloads slow down serialization and Redis I/O. 1 MB is reasonable; adjust based on your data.
Expiration vs. LocalCacheExpiration: The first is the global TTL (when the entry leaves Redis entirely). The second is how long the local copy persists after that. This is a grace period. If Redis is temporarily unreachable, local copies can still serve stale data for a brief window. Useful for resilience.

Repository Pattern with Stampede Protection

Here’s a real-world data access pattern:

public class CachedProductRepository
{
private readonly HybridCache _cache;
private readonly AppDbContext _dbContext;
private readonly ILogger<CachedProductRepository> _logger;

public CachedProductRepository(
HybridCache cache,
AppDbContext dbContext,
ILogger<CachedProductRepository> logger)
{
_cache = cache;
_dbContext = dbContext;
_logger = logger;
}

public async Task<ProductDto?> GetProductAsync(Guid id, CancellationToken cancellationToken)
{
// Construct a cache key that uniquely identifies this product
string cacheKey = $”product-{id:N}”; // “product-a1b2c3d4e5f6g7h8”

// The magic: GetOrCreateAsync handles the entire stampede-prevention dance
return await _cache.GetOrCreateAsync(
cacheKey,
async cancellationToken =>
{
// This factory function runs *only* on a cache miss
_logger.LogInformation(“Cache miss for product {ProductId}, fetching from database”, id);

var product = await _dbContext.Products
.Where(p => p.Id == id)
.Select(p => new ProductDto(p.Id, p.Name, p.Price, p.CreatedAt))
.FirstOrDefaultAsync(cancellationToken);

return product;
},
cancellationToken: cancellationToken
);
}

public async Task<List<ProductDto>> GetProductsByCategory(
string category,
CancellationToken cancellationToken)
{
string cacheKey = $”products-category-{category}”;

// For collections, use GetOrCreateAsync with explicit options
return await _cache.GetOrCreateAsync(
cacheKey,
async ct =>
{
_logger.LogInformation(“Cache miss for category {Category}, fetching from database”, category);

var products = await _dbContext.Products
.Where(p => p.Category == category && p.IsActive)
.Select(p => new ProductDto(p.Id, p.Name, p.Price, p.CreatedAt))
.ToListAsync(ct);

return products;
},
options: new HybridCacheEntryOptions
{
Expiration = TimeSpan.FromMinutes(15), // Category listings change less frequently
LocalCacheExpiration = TimeSpan.FromMinutes(3)
},
cancellationToken: cancellationToken
);
}
}

Invalidation: The Hard Part

Caching is easy. Invalidation is hard. When does data become stale and need to be purged?

Time-based expiration (TTL) is your friend here. Set a reasonable TTL based on how often data changes and how stale it’s acceptable to be. For product data, 30 minutes is often reasonable. For user preferences, maybe 5 minutes.

But if a user edits a product, you want that change reflected immediately, not in 30 minutes. This is where explicit invalidation comes in:

public async Task<ProductDto> UpdateProductAsync(
Guid id,
UpdateProductRequest request,
CancellationToken cancellationToken)
{
var product = await _dbContext.Products.FindAsync(id);

if (product == null)
throw new NotFoundException($”Product {id} not found”);

// Apply changes
product.Name = request.Name;
product.Price = request.Price;
product.UpdatedAt = DateTime.UtcNow;

await _dbContext.SaveChangesAsync(cancellationToken);

// Invalidate the cache for this product
string cacheKey = $”product-{id:N}”;
await _cache.RemoveAsync(cacheKey, cancellationToken);

// Also invalidate any category listings that might include this product
string categoryKey = $”products-category-{product.Category}”;
await _cache.RemoveAsync(categoryKey, cancellationToken);

return new ProductDto(product.Id, product.Name, product.Price, product.UpdatedAt);
}

Distributed Cache Outages: Graceful Degradation

What if Redis goes down? HybridCache has you covered. The L1 (local) cache continues to serve data. You’ll get cache misses more frequently (only L1 is available, no L2 sync), but you won’t crash. Your application degrades gracefully.

However, if you’re running multiple app servers and Redis is down, each server’s L1 cache will diverge. Server A might have a stale copy of product-123 while Server B has a fresh copy. This is an eventual consistency trade-off, but it beats a hard failure.

Pro tip: Log when Redis is unreachable. Set up an alert. Don’t ignore it. Distributed cache failures are often early signs of bigger infrastructure problems.

Choosing Cache Keys Wisely

Bad cache keys lead to collisions and subtle bugs. Use a consistent, deterministic format:

// Good: descriptive, includes all relevant dimensions
string key = $”product-{id:N}-include-inventory={includeInventory}”;

// Bad: ambiguous, could collide with other caches
string key = $”prod-{id}”;

// Bad: includes user input without normalization (leading to duplicate keys)
string key = $”product-search-{userQuery}”; // What if userQuery is ” laptop ” vs “laptop”?

Prefix your cache keys with a service or domain identifier if multiple services share a Redis instance:

string key = $”catalog:product-{id:N}”;
string key = $”orders:order-{orderId:N}”;

3. Preventing Cascading Failures: Circuit Breakers with Polly

The Production Pain Point

The third and final piece of Advanced API Guardrails: preventing cascading failures with circuit breakers. Our payment processing system calls an external payment gateway (Stripe, Square, whatever). The gateway is reliable 99.9% of the time. But that 0.1% kills us.

One Tuesday morning, the gateway had a minor outage—just 60 seconds. But in those 60 seconds, our API hammered it with retries. Each request timed out after 3 seconds. Each timeout triggered an automatic retry (because we had retry logic). Three retries * 3-second timeout = 9 seconds of waiting per request. Our request thread pool got saturated waiting for dead connections. New incoming requests had no threads available. HTTP 503s.

The gateway recovered at 10:01 AM. But by 10:05 AM, we still had a mountain of queued requests backed up from the outage. And our app server was still hammering the gateway with retry requests that had been sitting in the thread pool.

This is a cascading failure. One downstream service goes down, it takes down your service, which affects everything upstream.

The solution isn’t to retry harder or add more threads. It’s to stop calling the broken service and fail fast. This is the circuit breaker pattern.

How Circuit Breakers Work

Imagine a physical circuit breaker in your house. When current overloads, the breaker trips and stops the current flow. You don’t keep trying to flip the switch; the breaker has already made the decision: “Stop, something’s wrong.”

The software pattern works similarly:

Closed state (normal): Requests go through. If a request succeeds, great. If it fails, we note it.
Threshold reached: If failures exceed a threshold (e.g., 50% of recent requests failed), the breaker trips.
Open state (fail-fast): Subsequent requests fail immediately without calling the downstream service. No timeouts, no retries, no thread blocking. Just instant ServiceUnavailableException.
Half-Open state (testing): After some time (e.g., 30 seconds), the breaker enters half-open. It allows one test request through. If it succeeds, the breaker closes. If it fails, the breaker opens again.

This way, the moment the gateway comes back online, we detect it and resume normal operation. In the meantime, our thread pool isn’t blocked on dead connections.

Polly: Resilience Pipelines

Polly is the gold standard resilience library for .NET. Modern Polly (v8+) uses Resilience Pipelines, which combine multiple strategies (timeout, retry, circuit breaker) into a single composable unit.

First, add the package:

dotnet add package Polly
dotnet add package Polly.Extensions

Now define a resilience pipeline in Program.cs:

builder.Services.AddResiliencePipeline(“PaymentServicePipeline”, (builder) =>
{
// Strategy 1: Timeout protection
// If the downstream service doesn’t respond in 3 seconds, fail fast
builder.AddTimeout(TimeSpan.FromSeconds(3));

// Strategy 2: Retry with exponential backoff
// Retry up to 3 times, waiting 500ms, 1s, 2s (exponential)
builder.AddRetry(new HttpRetryStrategyOptions
{
MaxRetryAttempts = 3,
Delay = TimeSpan.FromMilliseconds(500),
BackoffType = DelayBackoffType.Exponential,
ShouldHandle = new PredicateBuilder<HttpResponseMessage>()
.Handle<HttpRequestException>()
.HandleResult(r => !r.IsSuccessStatusCode)
});

// Strategy 3: Circuit Breaker
// If 50% of requests fail over the last 30 seconds (with at least 8 samples),
// open the circuit for 15 seconds
builder.AddCircuitBreaker(new HttpCircuitBreakerStrategyOptions
{
FailureRatio = 0.5, // 50% failure = trip
SamplingDuration = TimeSpan.FromSeconds(30), // Over last 30 seconds
MinimumThroughput = 8, // Need at least 8 requests to evaluate
BreakDuration = TimeSpan.FromSeconds(15), // Stay open for 15 seconds
OnOpened = async args =>
{
// Log when the circuit opens
var logger = args.ServiceProvider.GetRequiredService<ILogger<Program>>();
logger.LogCritical(
“Circuit breaker OPENED for PaymentService. Reason: {Reason}”,
args.Outcome.Exception?.Message ?? “High failure rate”);
}
});
});

Let me break down those circuit breaker parameters:

FailureRatio (0.5): Trip the breaker if 50% or more of requests fail. This is configurable. For non-critical services, 50% is reasonable. For critical infrastructure, you might want 30%. For super-flaky services you’re trying to isolate, maybe 70%.
SamplingDuration (30 seconds): Evaluate the failure rate over the most recent 30 seconds. This prevents old failures from influencing the decision.
MinimumThroughput (8 requests): Don’t trip based on a tiny sample. If only 2 requests have gone through, even if both failed, don’t open. Wait for at least 8 requests before deciding.
BreakDuration (15 seconds): Once open, stay open for 15 seconds. After that, go half-open and test one request. This prevents rapid open-close-open-close cycles.

Using the Pipeline with HttpClient

Inject the pipeline into your service and use it to wrap HTTP calls:

public class PaymentServiceClient
{
private readonly HttpClient _httpClient;
private readonly ResiliencePipeline<HttpResponseMessage> _pipeline;
private readonly ILogger<PaymentServiceClient> _logger;

public PaymentServiceClient(
HttpClient httpClient,
ResiliencePipelineProvider<string> pipelineProvider,
ILogger<PaymentServiceClient> logger)
{
_httpClient = httpClient;
_pipeline = pipelineProvider.GetPipeline<HttpResponseMessage>(“PaymentServicePipeline”);
_logger = logger;
}

public async Task<PaymentResult> ProcessPaymentAsync(
PaymentRequest request,
CancellationToken cancellationToken)
{
try
{
// Execute the HTTP call wrapped in the resilience pipeline
// The pipeline handles timeout, retry, and circuit breaker
var response = await _pipeline.ExecuteAsync(
async ct =>
{
var json = JsonSerializer.Serialize(request);
var content = new StringContent(json, Encoding.UTF8, “application/json”);

return await _httpClient.PostAsync(“/v1/charges”, content, ct);
},
cancellationToken
);

if (!response.IsSuccessStatusCode)
{
var errorBody = await response.Content.ReadAsStringAsync(cancellationToken);
_logger.LogWarning(“Payment gateway returned {StatusCode}: {Error}”,
response.StatusCode, errorBody);

return new PaymentResult(success: false,
error: $”Payment gateway error: {response.StatusCode}”);
}

var resultJson = await response.Content.ReadAsStringAsync(cancellationToken);
var result = JsonSerializer.Deserialize<PaymentGatewayResponse>(resultJson);

return new PaymentResult(
success: true,
transactionId: result?.Id
);
}
catch (BrokenCircuitException ex)
{
// The circuit breaker is open. Fail gracefully.
_logger.LogError(“Payment service circuit breaker is open. Cannot process payment.”);
return new PaymentResult(success: false, error: “Payment service temporarily unavailable. Please try again.”);
}
catch (OperationCanceledException ex)
{
// Request timed out (exceeded 3-second timeout from the pipeline)
_logger.LogError(“Payment service request timed out”);
return new PaymentResult(success: false, error: “Payment service timeout. Please try again.”);
}
catch (HttpRequestException ex)
{
// Network error, DNS failure, SSL error, etc.
_logger.LogError(ex, “Payment service connection error”);
return new PaymentResult(success: false, error: “Payment service unreachable. Please try again.”);
}
}
}

Architectural Decisions

Timeout Duration: How Long Is Too Long?

Your timeout should be slightly longer than your expected p99 latency. If the payment gateway usually responds in 500ms but occasionally in 1.5 seconds, set your timeout to 2 or 3 seconds. If you timeout at 1 second, you’ll trigger unnecessary retries on slow-but-succeeding requests.

We typically start with 3 seconds for external API calls and adjust based on observed latencies.

Retry Strategy: When Is It Safe?

Only retry on idempotent operations. A payment charge is idempotent if you can safely retry it and have it succeed or fail in the same way. (Many modern payment gateways use idempotency keys for this reason.)

A write operation that modifies state (e.g., “delete this user”) is not idempotent—retrying could delete the user twice if the first request actually succeeded but the response timed out.

In the Polly configuration above, we retry on HttpRequestException (network errors) but not on all 5xx status codes. A 503 Service Unavailable is retriable. A 400 Bad Request is not—the request is malformed and retrying won’t fix it.

Circuit Breaker Thresholds: Tight vs. Loose

A tight circuit breaker (low failure ratio, low minimum throughput) opens quickly. It prevents cascading failures but might overreact to momentary blips.

A loose circuit breaker (high failure ratio, high minimum throughput) opens slowly. It tolerates blips but takes longer to protect you from real problems.

Start with what we’ve shown (50% failure, 8 minimum throughput) and adjust based on your service’s characteristics:

Critical infrastructure (payment, auth): Tighter (e.g., 30% failure ratio, 20 minimum throughput)
Best-effort services (analytics, recommendations): Looser (e.g., 70% failure ratio, 5 minimum throughput)

What About Multiple Downstream Services?

Create separate resilience pipelines for each service. You don’t want one flaky service dragging down your entire resilience strategy.

builder.Services.AddResiliencePipeline(“PaymentServicePipeline”, /* … */);
builder.Services.AddResiliencePipeline(“ShippingServicePipeline”, /* … */);
builder.Services.AddResiliencePipeline(“NotificationServicePipeline”, /* … */);

Putting It All Together: A Real System

Here’s how these Advanced API Guardrails work together in a real e-commerce checkout API. Here’s how these three patterns work together in a real e-commerce checkout API:

[ApiController]
[Route(“api/checkout”)]
public class CheckoutController : ControllerBase
{
private readonly OrderService _orderService;
private readonly CachedProductRepository _productRepository;
private readonly PaymentServiceClient _paymentClient;

public CheckoutController(
OrderService orderService,
CachedProductRepository productRepository,
PaymentServiceClient paymentClient)
{
_orderService = orderService;
_productRepository = productRepository;
_paymentClient = paymentClient;
}

[HttpPost(“process”)]
[RequireRateLimiting(“PublicApiPolicy”)]
public async Task<IActionResult> ProcessCheckout(
CheckoutRequest request,
CancellationToken cancellationToken)
{
// Rate limiting: Protect against abuse, DoS, traffic spikes
// Already applied by the [RequireRateLimiting] attribute

// Caching: Fetch product details from cache (avoids database stampede)
var products = new List<ProductDto>();
foreach (var lineItem in request.LineItems)
{
var product = await _productRepository.GetProductAsync(
lineItem.ProductId,
cancellationToken);

if (product == null)
return NotFound($”Product {lineItem.ProductId} not found”);

products.Add(product);
}

// Calculate order total
decimal total = request.LineItems
.Zip(products, (item, product) => item.Quantity * product.Price)
.Sum();

// Circuit Breaker: Call payment gateway with resilience guarantees
var paymentResult = await _paymentClient.ProcessPaymentAsync(
new PaymentRequest { Amount = total, Currency = “USD” },
cancellationToken);

if (!paymentResult.Success)
return BadRequest(new { error = paymentResult.Error });

// Persist the order
var order = await _orderService.CreateOrderAsync(
new CreateOrderRequest
{
UserId = User.FindFirst(“sub”)?.Value,
LineItems = request.LineItems,
Total = total,
PaymentTransactionId = paymentResult.TransactionId
},
cancellationToken);

return Ok(new { orderId = order.Id, total = order.Total });
}
}

In this flow:

Rate limiting gates the request. If the user has exceeded their quota, they get 429 immediately. This protects your infrastructure.
Caching fetches product details. Multiple concurrent users requesting the same product hit the cache. If the cache key expires, only one thread hits the database; others wait for the cached result. No stampede.
Circuit breaker calls the payment gateway. If the gateway is misbehaving (high failure rate), the circuit opens and we fail fast instead of threading through timeout-retry-timeout cycles. We stay responsive to the user and protect our thread pool.
If payment succeeds, we persist the order. This is a write operation—it’s not cached (caching reads, not writes).

A checkout flow that stays fast and responsive under load isn’t just an infrastructure win — it’s a direct customer experience win. For a broader look at this shift, see How Intelligent Systems Are Transforming Customer Experiences.

Monitoring and Observability

These guardrails only work if you can see them. Add structured logging and metrics:

// In your Polly circuit breaker configuration
builder.AddCircuitBreaker(new HttpCircuitBreakerStrategyOptions
{
// … other options …
OnOpened = async args =>
{
var logger = args.ServiceProvider.GetRequiredService<ILogger<Program>>();
var meter = args.ServiceProvider.GetRequiredService<IMeterProvider>();

logger.LogCritical(“Payment gateway circuit breaker opened”);

// Increment a counter metric for alerting
var counter = meter
.CreateMeter(“payment-gateway”)
.CreateCounter<int>(“circuit_breaker_trips”);
counter.Add(1);
}
});

Set up alerts in your monitoring system:

Rate limit rejections spiking: Indicates either traffic surge or potential attack.
Cache misses increasing: Could indicate memory pressure or too-aggressive TTL.
Circuit breaker trips: Indicates downstream service problems.

Wiring these alerts into your observability stack reliably — without gaps or alert fatigue — is where solid DevOps services and solutions make the difference.

Conclusion

Advanced API Guardrails — rate limiting, distributed caching, and circuit breakers — aren’t exotic patterns. They’re table stakes for APIs running in production. Advanced API Guardrails — rate limiting, distributed caching, and circuit breakers — aren’t exotic patterns. They’re table stakes for APIs running in production.

Start with what you need: Does traffic spike? Rate limit. Do you have a chatty database? Cache. Do you call flaky services? Circuit break.

Don’t over-engineer. Don’t assume you need all three from day one. But when you do need them—and you will—you’ll be glad you know how to implement them right.

The 3 AM page? It’s a lot quieter when you have good guardrails.

FAQs

Q1. What are the three core Advanced API Guardrails in .NET Core?
Native rate limiting (built into ASP.NET Core since .NET 7), distributed caching with HybridCache to prevent cache stampedes, and circuit breakers via Polly to stop cascading failures from downstream service outages.

Q2. Do I need Redis to implement rate limiting in ASP.NET Core?
No. The built-in Microsoft.AspNetCore.RateLimiting middleware works in-process without Redis. You only need Redis for strict, global rate limits shared exactly across multiple app server instances — most APIs don’t need this.

Q3. What’s the difference between a retry and a circuit breaker?
A retry re-attempts a failed request, which works for transient, idempotent failures. A circuit breaker stops sending requests entirely once a downstream service’s failure rate crosses a threshold, failing fast instead of piling up timeouts and retries that can exhaust your thread pool.

← Previous

Services

Microsoft Practices

Avio

Blog

Case Studies

Advanced API Guardrails: Implementing Rate Limiting, Distributed Caching, and Circuit Breakers in .NET Core

1. Traffic Control: Native Rate Limiting in ASP.NET Core

The Production Pain Point

The Modern Solution: Built Into .NET 7+

Choosing the Right Algorithm

Production Implementation: Token Bucket with Intelligent Rejection

Architectural Decisions and Trade-Offs

When do you need distributed rate limiting (e.g., with Redis)?

Token bucket parameters: how tight should I make it?

Queue limits: should I queue rejected requests?

2. Shielding the Database: Distributed Caching Without the Stampede

The Production Pain Point

The Modern Solution: HybridCache

Setting Up HybridCache

Repository Pattern with Stampede Protection

Invalidation: The Hard Part

Distributed Cache Outages: Graceful Degradation

Choosing Cache Keys Wisely

3. Preventing Cascading Failures: Circuit Breakers with Polly

The Production Pain Point

How Circuit Breakers Work

Polly: Resilience Pipelines

Using the Pipeline with HttpClient

Architectural Decisions

Timeout Duration: How Long Is Too Long?

Retry Strategy: When Is It Safe?

Circuit Breaker Thresholds: Tight vs. Loose

What About Multiple Downstream Services?

Putting It All Together: A Real System

Monitoring and Observability

Conclusion

FAQs