Response Caching

Response caching returns a stored response when your project sends the same chat request again, instead of calling the model. A cache hit costs nothing and returns much faster.

Response caching is off by default and available on every plan.

1. Open your project and go to Settings → General.
2. Under Request handling, turn on Response caching.

Turning it on uses these defaults:

The switch only turns caching on or off; it does not change these values. Projects that already had caching tuned keep their existing settings.

Caching applies to chat requests (/v1/chat/completions and the SDK's chat) when all of these are true:

• The request's temperature is at or below the highest cacheable temperature (0.2 by default). A request without a temperature counts as 0.
• The request has no tools.
• The request does not use memory retrieval or knowledge bases.
• The request is not sent with passthrough / fast_lane.
• The request does not have the x-skip-cache: true header.

On the SDK's chat, only non-streaming requests without images are cached. On /v1/chat/completions, streaming requests are cached too (exact matching only), and requests to the auto / cencori-auto model are never cached.

Exact matching returns a stored response when the new request has the same model, temperature, max tokens, and messages (role and content, ignoring leading and trailing whitespace) as an earlier one in the same project.

Similar-request (semantic) matching returns a stored response when a non-streaming request is close in meaning to an earlier one, above a similarity threshold (0.95 by default). It is off by default.

Cached responses expire after the time to live (1 hour by default).

Responses served from the cache carry these headers:

Send the x-skip-cache: true header to always call the model:

curl https://api.cencori.com/v1/chat/completions \
  -H "Authorization: Bearer $CENCORI_API_KEY" \
  -H "x-skip-cache: true" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Hello"}]}'

The legacy text completions endpoint (/api/ai/completions) keeps its own exact-match cache for non-streaming requests. It runs whether or not Response caching is on.

• Use caching for repeated, deterministic requests (low temperature), such as classification or fixed prompts.
• Keep system prompts and message structure stable so repeated requests match exactly.
• Leave caching off for requests whose answer must reflect changing data.