Rate Limiting
Understand Cencori's rate limiting system, how to handle limits, and best practices for high-volume applications.
What is Rate Limiting?
Rate limiting controls how many requests you can make in a given time period. This protects the platform from abuse and ensures fair usage for all users.
Cencori enforces a per-project rate limit (per-minute). There is no monthly request cap on any plan.
Default Rate Limits
Reads have their own allowance, so status polling never uses up the budget for creating runs. Cancellation is always admitted: the corrective action for overload must never itself be blocked by overload. X-RateLimit-Limit on each response tells you which tier applied. Need more write throughput? Add a second project.
Note: write limits are enforced project-wide across all AI endpoints (chat, vision, audio, embeddings), not per endpoint.
Polling guidance
Poll run status no faster than once every 2 seconds, always honor the Retry-After header on 429, and back off rather than retrying aggressively. For fleets of concurrent runs, prefer webhooks (run.completed delivery) or mode: 'sync' over per-run poll loops.
How the window works
Each limit counts requests in a fixed 60-second window. Rejected requests don't count against you or extend the wait: when the window ends, your full allowance is back.
Monthly Requests
There is no monthly request ceiling on any plan. What bounds usage is spend, not request count: an organization out of credits gets 402 insufficient_credits, a frozen account gets 403 billing_frozen, and spend caps apply where you've set them (project Settings → Budget).
Rate Limit Headers
429 responses include headers showing your current rate limit status:
Every gateway response (success or error) additionally carries X-Request-Id for correlation.
Handling Rate Limit Errors
When you exceed the rate limit you'll receive a 429 Too Many Requests response with a standards-compliant Retry-After header and this body:
{
"error": "Rate limit exceeded",
"message": "60 requests per minute allowed. Try again shortly.",
"code": "rate_limit_exceeded",
"retry_after_ms": 45000
}Exponential Backoff Implementation
async function makeRequestWithRetry(
messages: any[],
maxRetries = 3
): Promise<any> {
let retries = 0;
while (retries < maxRetries) {
try {
const response = await cencori.ai.chat({
model: 'gpt-4o',
messages,
});
return response;
} catch (error: any) {
if (error.status === 429) {
retries++;
if (retries >= maxRetries) {
throw error; // Max retries reached
}
const retryAfter = Math.max(
1,
Math.ceil((error.retryAfterMs ?? 60_000) / 1000)
);
await new Promise(resolve => setTimeout(resolve, retryAfter * 1000));
} else {
throw error; // Not a rate limit error
}
}
}
}Checking Rate Limits Before Requests
async function makeSmartRequest(messages: any[]) {
const response = await fetch('https://cencori.com/api/ai/chat', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'CENCORI_API_KEY': process.env.CENCORI_API_KEY!,
},
body: JSON.stringify({ model: 'gpt-4o', messages }),
});
// Check rate limit headers
const remaining = parseInt(response.headers.get('X-RateLimit-Remaining') || '0');
const resetTime = parseInt(response.headers.get('X-RateLimit-Reset') || '0');
if (remaining < 5) {
console.warn(`Only ${remaining} requests remaining!`);
console.warn(`Resets at: ${new Date(resetTime).toISOString()}`);
// Maybe slow down or queue requests
}
return response.json();
}Best Practices
- Implement Exponential Backoff: Always retry rate-limited requests with exponential backoff rather than aggressive retries.
- Monitor Headers: Track
X-RateLimit-Remainingto know when you're approaching limits. - Use Request Queues: Queue requests and process them at a controlled rate to stay under limits.
- Cache Responses: Turn on Response caching so repeated identical requests are served from cache.
- Distribute Load: If hitting limits, consider using multiple projects (each has its own 60/min window).
- Audio calls count too: transcriptions and speech each count as one request; spread heavy audio workloads across projects.
Request Queue Implementation
class RequestQueue {
private queue: Array<() => Promise<any>> = [];
private processing = false;
private requestsPerMinute: number;
private delay: number;
constructor(requestsPerMinute = 60) {
this.requestsPerMinute = requestsPerMinute;
this.delay = 60000 / requestsPerMinute; // Time between requests
}
async add<T>(fn: () => Promise<T>): Promise<T> {
return new Promise((resolve, reject) => {
this.queue.push(async () => {
try {
const result = await fn();
resolve(result);
} catch (error) {
reject(error);
}
});
if (!this.processing) {
this.process();
}
});
}
private async process() {
this.processing = true;
while (this.queue.length > 0) {
const task = this.queue.shift();
if (task) {
await task();
await new Promise(resolve => setTimeout(resolve, this.delay));
}
}
this.processing = false;
}
}
// Usage
const queue = new RequestQueue(60); // 60 requests per minuteUpgrading Rate Limits
If you consistently hit limits:
- Create additional projects — each project gets its own 60/min window.
- Contact sales for custom Enterprise limits.
- Optimize usage patterns (batching, caching, queues).
Recoverability
503 rate_limit_unavailable: transient — retry with backoff.429 rate_limit_exceeded: waitRetry-Afterseconds.