Response Caching

Response caching (also called cross-tenant deduplication) reduces costs and prevents duplicate requests by caching HTTP responses across all tenants. When multiple tenants test the same endpoint with identical configurations, Pingward serves cached responses instead of making redundant API calls.

Overview

What Is Response Caching?

Response caching captures the raw HTTP response from an API request and temporarily stores it in Redis. When another tenant executes an identical test within the cache window (default: 10 seconds), Pingward retrieves the cached response instead of making a new HTTP request.

Key benefits:

Cost reduction: 70-80% fewer HTTP requests in typical deployments
DDoS protection: Same endpoint never hit more than once per cache window
Faster execution: Cache retrieval takes <5ms vs 100-500ms for HTTP requests
API rate limit protection: Prevents triggering rate limits across tenants

How It Works

The cache operates at the worker level and is shared across all tenants:

Request signature generation: Each test request is converted into a unique cache key based on URL, method, headers, and body
Cache lookup: Before executing the HTTP request, workers check Redis for a cached response
Cache hit: If found, the cached response is returned and assertions are re-evaluated for the current tenant
Cache miss: If not found, the HTTP request is executed and the response is stored in Redis with a short TTL
Per-tenant evaluation: Each tenant's assertions are evaluated against the cached response independently

Important: The cache stores raw HTTP responses (status code, headers, body, response time), NOT evaluated test results. This allows each tenant to evaluate their own assertions against the same cached response.

Cache Flow

Test Execution Request
        ↓
Generate Cache Key (SHA256 of request signature)
        ↓
Check Redis Cache
        ↓
    Hit? ────Yes───→ Return Cached Response
        │                    ↓
        No              Re-evaluate Assertions
        ↓                    ↓
Execute HTTP Request    Store TestResult
        ↓
Cache Raw Response (if cacheable)
        ↓
Evaluate Assertions
        ↓
Store TestResult

Configuration

Response caching is disabled by default and must be explicitly enabled.

Enabling Caching

Edit your worker's appsettings.json:

json

{
  "Worker": {
    "Cache": {
      "Enabled": true,
      "RedisConnectionString": "localhost:6379",
      "DefaultTtlSeconds": 10,
      "MaxTtlSeconds": 60,
      "LogCacheHits": false
    }
  }
}

Configuration Options

Setting	Description	Default
`Enabled`	Master switch for caching	`false`
`RedisConnectionString`	Redis server connection string	`"localhost:6379"`
`DefaultTtlSeconds`	Default cache TTL in seconds	`10`
`MaxTtlSeconds`	Maximum cache TTL in seconds	`60`
`LogCacheHits`	Log cache hits/misses for debugging	`false`

Redis Setup

Local Development

bash

# Using Docker
docker run -d -p 6379:6379 --name pingward-redis redis:7-alpine

# Verify connection
redis-cli ping
# Should return: PONG

Production

For production deployments, use a managed Redis service:

Azure Cache for Redis:

json

{
  "RedisConnectionString": "your-cache.redis.cache.windows.net:6380,password=your-key,ssl=True,abortConnect=False"
}

AWS ElastiCache:

json

{
  "RedisConnectionString": "your-cache.abc123.0001.use1.cache.amazonaws.com:6379"
}

Redis Cloud:

json

{
  "RedisConnectionString": "redis-12345.c1.us-east-1-1.ec2.cloud.redislabs.com:12345,password=your-password"
}

Redis Security

For production environments:

Enable TLS/SSL: Add ssl=True to the connection string
Set a password: Configure requirepass in Redis and include it in the connection string
Network isolation: Place Redis in a private network, not publicly accessible
Encryption at rest: Enable encryption at rest if your Redis provider supports it

Cacheability Rules

Not all requests are cacheable. Pingward applies strict rules to ensure cached responses are safe and correct.

What Gets Cached

Only these requests are cached:

GET requests without authentication
Public endpoints that don't require credentials
Idempotent operations that return consistent results
Responses with cache-friendly headers (Cache-Control: public, max-age)

What NEVER Gets Cached

The following requests are never cached for security and correctness:

Rule	Reason
Authenticated requests (API Key, Basic Auth, Bearer, OAuth2)	User-specific responses that should not be shared
Write operations (POST, PUT, PATCH, DELETE)	Non-idempotent operations that modify state
Requests with user-specific headers (Authorization, Cookie, Session-ID)	User context prevents sharing
Responses with Set-Cookie headers	Session management requires fresh responses
Responses with Cache-Control: no-cache or private	API explicitly disables caching

Examples

Cacheable:

http

GET https://api.example.com/health

NOT cacheable (authenticated):

http

GET https://api.example.com/users/me
Authorization: Bearer token123

NOT cacheable (write operation):

http

POST https://api.example.com/orders
Content-Type: application/json
{"item": "product"}

Cache Key Generation

Cache keys are computed using SHA256 hashes of normalized request signatures. This ensures identical requests produce identical keys, even if they differ in formatting.

Signature Components

A cache key includes:

CacheKey = SHA256(
    Normalized URL +
    HTTP Method +
    Sorted Headers (filtered) +
    Normalized Body +
    Hashed Auth Config
)

Normalization Rules

URL Normalization

Convert to lowercase: https://API.com → https://api.com
Sort query parameters: ?b=2&a=1 → ?a=1&b=2
Remove trailing slashes: /users/ → /users

Header Normalization

Sort alphabetically by header name
Convert header names to lowercase
Filter time-based headers: Date, X-Request-Id, X-Correlation-Id
Filter auth headers: Authorization, Cookie, X-Auth-Token

Body Normalization

JSON: Parse and serialize with sorted keys, no whitespace
Non-JSON: Trim whitespace but keep as-is

Example:

json

// These produce the same cache key:
{"name": "test", "value": 123}
{ "value": 123, "name": "test" }
{"name":"test","value":123}

Authentication Hashing

Authentication credentials are hashed separately to prevent exposure in cache keys:

csharp

// Instead of including plaintext credentials in the cache key:
// ❌ "api_key=secret123"

// Credentials are hashed:
// ✅ SHA256("api_key=secret123") = "a1b2c3..."

This ensures:

Credentials never appear in cache keys
Different credentials produce different cache keys
Cache key collisions are impossible

Cache Key Exclusions

Assertions are NOT included in the cache key. This maximizes cache hit rates because:

Different tenants may have different assertions for the same endpoint
The cached response can be re-evaluated with each tenant's assertions
Cache hit rate increases from ~40% to 70-80%

Performance Benefits

Response caching dramatically reduces API costs and improves test execution speed.

Expected Metrics

Metric	Without Caching	With Caching	Improvement
Cache Hit Rate	0%	70-80%	-
Avg Response Time	200ms	50ms	75% faster
API Requests/Hour	1000	250	75% reduction
Monthly API Costs	$100	$25	75% savings

Real-World Example

Scenario: 10 tenants all monitor https://api.github.com/status

Without caching:

Each tenant makes a request every 60 seconds
10 tenants × 1440 checks/day = 14,400 requests/day
Cost: 14,400 API calls

With caching (10s TTL):

First tenant makes a real request
Next 9 tenants use cached response (within 10s window)
Only ~6 requests per minute across all tenants
6 × 1440 minutes = 8,640 requests/day
Cost: 8,640 API calls
Savings: 40% reduction

Cost Savings Calculator

Estimate your savings:

Current monthly API calls: ___________
Cache hit rate (typically 70%): × 0.70
Cacheable requests: ___________
Cost per 1M requests: $___________
Monthly savings: $___________

Monitoring

Track cache performance through the admin dashboard and logs.

Admin Dashboard

Navigate to Admin → Response Cache to view:

Overall hit rate: Percentage of requests served from cache
Total hits/misses: Cumulative cache statistics
Cache size: Number of cached entries and memory usage
Per-worker breakdown: Hit rates by worker and region

What to look for:

Hit rate above 50%: Healthy caching
Hit rate below 30%: Most tests may be authenticated or using write operations
Sudden drop in hit rate: Check if tests changed or Redis connection failed

Cache Metrics

Cache metrics are exposed per worker at:

GET /internal/cache/stats

Response:

json

{
  "workerId": "worker-eastus-001",
  "hits": 12450,
  "misses": 3120,
  "hitRate": 0.799,
  "totalKeys": 1523,
  "memoryUsageBytes": 12998400
}

Log Monitoring

Enable cache hit logging for debugging:

json

{
  "Worker": {
    "Cache": {
      "LogCacheHits": true
    }
  }
}

Log output:

[12:34:56 INF] Cache HIT: a1b2c3d4 (TestId: 12345)
[12:34:57 INF] Cache SET: e5f6g7h8 (TTL: 10s)
[12:34:58 INF] Cache MISS: i9j0k1l2 (TestId: 67890)

Note: Only enable LogCacheHits for debugging. High-traffic workers may generate excessive logs.

Security

Response caching is designed with security as a top priority.

Credential Protection

Credentials never appear in cache keys:

API keys are hashed using SHA256
Bearer tokens are hashed separately
Basic Auth credentials are hashed
OAuth2 client secrets are hashed

Example:

// Cache key does NOT contain:
"Authorization: Bearer sk_live_abc123"

// Cache key contains:
"auth_hash:a1b2c3d4e5f6..."

Tenant Isolation

Authenticated requests are NEVER cached:

Each tenant's authenticated requests hit the API directly
No cross-tenant data leakage is possible
User-specific responses remain private

Why authenticated requests aren't cached:

Security: Responses may contain user-specific data
Correctness: Different users may get different responses
Privacy: Tenant data must not be shared across tenants

Redis Encryption

At rest:

Use managed Redis services with encryption at rest
Azure Cache for Redis: Automatically encrypted
AWS ElastiCache: Enable encryption at rest in settings

In transit:

Always use TLS/SSL in production: ssl=True
Verify Redis certificate validation is enabled
Use private network connections when possible

Filtered Headers

These headers are automatically filtered from cached responses to prevent information leakage:

Set-Cookie: Session cookies not shared
Authorization: Auth tokens removed
Date: Timestamp removed (would reveal cache age)
X-Request-Id: Request tracing removed

Troubleshooting

Cache Not Working

Symptom: Hit rate is 0% or cache metrics show no hits.

Possible causes:

Cache is disabled
- Check appsettings.json: Enabled: true
- Restart worker after config changes
Redis connection failed
- Check Redis is running: redis-cli ping
- Verify connection string is correct
- Check network connectivity from worker to Redis
All tests use authentication
- Authenticated requests are never cached
- Review cacheability rules above
Tests use write operations
- POST, PUT, DELETE are never cached
- Only GET requests are cacheable

Fix:

bash

# Check Redis connectivity
redis-cli -h your-redis-host -p 6379 ping

# Check worker logs
grep "Cache" worker.log

# Verify cache is enabled
grep -A 5 '"Cache"' appsettings.json

Low Hit Rate

Symptom: Hit rate is below 30%.

Possible causes:

Tests use authentication
- Authenticated tests bypass cache
- Check if tests have API keys, bearer tokens, or basic auth
Tests have unique URLs
- Each unique URL creates a separate cache entry
- Dynamic query parameters reduce hit rate
Tests have variable headers
- Headers like Date or X-Request-Id vary per request
- These headers should be automatically filtered
Cache TTL too short
- Default TTL is 10 seconds
- If tests run >10s apart, cache expires

Fix:

json

{
  "Worker": {
    "Cache": {
      "DefaultTtlSeconds": 30
    }
  }
}

Redis Out of Memory

Symptom: Redis logs show memory errors or cache stops working.

Possible causes:

Too many cache entries
- Thousands of unique URLs cached
- Redis maxmemory limit reached
TTL too long
- Old entries not expiring
- Memory accumulates over time

Fix:

Set a memory policy in Redis:

redis

# Redis CLI
CONFIG SET maxmemory 256mb
CONFIG SET maxmemory-policy allkeys-lru

Or configure in redis.conf:

maxmemory 256mb
maxmemory-policy allkeys-lru

Recommended policies:

allkeys-lru: Evict least recently used keys
volatile-ttl: Evict keys with nearest expiry time

Cache Returning Stale Data

Symptom: Tests show old response data.

Possible causes:

TTL too long
- Cached responses are valid for too long
- API has changed but cache hasn't expired

Fix:

Reduce TTL:

json

{
  "Worker": {
    "Cache": {
      "DefaultTtlSeconds": 5
    }
  }
}

Or manually flush cache:

bash

redis-cli FLUSHDB

Per-Tenant Assertion Failures

Symptom: Different tenants get different test results for the same cached response.

This is expected behavior!

The cached response is shared across tenants
Each tenant re-evaluates their own assertions
Tenant A may have assertions that pass while Tenant B's fail

Example:

Cached Response: Status 200, Body: {"status": "ok", "version": "1.2.3"}

Tenant A Assertion: status code == 200
Result: ✅ PASS (cache hit)

Tenant B Assertion: body contains "version": "2.0.0"
Result: ❌ FAIL (cache hit, but assertion failed)

This is by design. Each tenant's assertions are evaluated independently against the same cached response.

Best Practices

Maximize Cache Hit Rate

Use public endpoints for health checks: Public APIs have higher cache hit rates
Standardize test configurations: Identical tests across tenants share cache
Avoid unnecessary headers: Remove headers that aren't required by the API
Use consistent URL formatting: https://api.com/users is different from https://api.com/users/

Balance TTL and Freshness

Short TTL (5-10s): More fresh data, slightly lower hit rate
Medium TTL (30-60s): Higher hit rate, less fresh data
Long TTL (>60s): Not recommended (stale data risk)

Recommended: Start with 10 seconds and adjust based on your needs.

Monitor Regularly

Check cache hit rate weekly
Alert if hit rate drops below 40%
Monitor Redis memory usage
Review cache logs when debugging test failures

Security Checklist

✅ Never cache authenticated requests
✅ Use TLS/SSL for Redis in production
✅ Set a Redis password (requirepass)
✅ Place Redis in a private network
✅ Enable encryption at rest
✅ Rotate Redis passwords quarterly

FAQ

Will caching affect test accuracy?

No. Each tenant's assertions are re-evaluated against the cached response. The cache only stores raw HTTP responses, not test results.

Can I disable caching for specific tests?

Currently, caching is controlled globally. Tests that use authentication or write operations are automatically excluded.

How long are responses cached?

Default: 10 seconds. Configurable via DefaultTtlSeconds (1-60 seconds).

Does caching work across regions?

No. Each worker region has its own Redis instance. This prevents cross-region latency issues.

What happens if Redis goes down?

The worker falls back to a no-op cache service. All requests execute normally without caching. No errors occur.

Can I see which tests are using cache?

Yes. Enable LogCacheHits: true to see cache hits/misses in worker logs. Each log entry includes the test ID.

Does caching affect response times in test results?

No. The cached response includes OriginalResponseTimeMs, which is the time from the original HTTP request. This ensures response time metrics remain accurate.

How much Redis memory do I need?

Estimate: 1KB per cached response on average.

1000 cached responses = ~1 MB
10,000 cached responses = ~10 MB
100,000 cached responses = ~100 MB

Recommended: Start with 256 MB and monitor usage.

Response Caching ​

Overview ​

What Is Response Caching? ​

How It Works ​

Cache Flow ​

Configuration ​

Enabling Caching ​

Configuration Options ​

Redis Setup ​

Local Development ​

Production ​

Redis Security ​

Cacheability Rules ​

What Gets Cached ​

What NEVER Gets Cached ​

Examples ​

Cache Key Generation ​

Signature Components ​

Normalization Rules ​

URL Normalization ​

Header Normalization ​

Body Normalization ​

Authentication Hashing ​

Cache Key Exclusions ​

Performance Benefits ​

Expected Metrics ​

Real-World Example ​

Cost Savings Calculator ​

Monitoring ​

Admin Dashboard ​

Cache Metrics ​

Log Monitoring ​

Security ​

Credential Protection ​

Tenant Isolation ​

Redis Encryption ​

Filtered Headers ​

Troubleshooting ​

Cache Not Working ​

Low Hit Rate ​

Redis Out of Memory ​

Cache Returning Stale Data ​

Per-Tenant Assertion Failures ​

Best Practices ​

Maximize Cache Hit Rate ​

Balance TTL and Freshness ​

Monitor Regularly ​

Security Checklist ​

FAQ ​

Will caching affect test accuracy? ​

Can I disable caching for specific tests? ​

How long are responses cached? ​

Does caching work across regions? ​

What happens if Redis goes down? ​

Can I see which tests are using cache? ​

Does caching affect response times in test results? ​

How much Redis memory do I need? ​

Related Topics ​

Response Caching

Overview

What Is Response Caching?

How It Works

Cache Flow

Configuration

Enabling Caching

Configuration Options

Redis Setup

Local Development

Production

Redis Security

Cacheability Rules

What Gets Cached

What NEVER Gets Cached

Examples

Cache Key Generation

Signature Components

Normalization Rules

URL Normalization

Header Normalization

Body Normalization

Authentication Hashing

Cache Key Exclusions

Performance Benefits

Expected Metrics

Real-World Example

Cost Savings Calculator

Monitoring

Admin Dashboard

Cache Metrics

Log Monitoring

Security

Credential Protection

Tenant Isolation

Redis Encryption

Filtered Headers

Troubleshooting

Cache Not Working

Low Hit Rate

Redis Out of Memory

Cache Returning Stale Data

Per-Tenant Assertion Failures

Best Practices

Maximize Cache Hit Rate

Balance TTL and Freshness

Monitor Regularly

Security Checklist

FAQ

Will caching affect test accuracy?

Can I disable caching for specific tests?

How long are responses cached?

Does caching work across regions?

What happens if Redis goes down?

Can I see which tests are using cache?

Does caching affect response times in test results?

How much Redis memory do I need?

Related Topics