# Tune wait budgets, pacing, and cooldowns

Canonical HTML: https://anchorshell.com/guides/tune-wait-pacing-cooldowns

Control when Relay waits for a preferred model and when it evaluates a fallback.

Last updated: 2026-08-18

## Purpose

Set a wait budget that protects response time without wasting preferred capacity. Configure pacing only where Relay can release requests at useful intervals.

## Prerequisites

- A Group contains at least two ranked Models.
- The Models have known or observed limits.
- Fallback is enabled when lower-ranked Models are acceptable.

## Set the wait budget

1. Open **Groups**.

2. Edit the target Group.

3. Set **Maximum wait** to the longest queue delay that callers accept.

4. Save the Group.

Relay waits for the preferred Model when its predicted eligibility is inside this budget. Relay evaluates lower-ranked Models when the preferred wait exceeds this budget and fallback is enabled.

## Configure request pacing

1. Open **Limits**.

2. Add a request limit with a `second` or `minute` period.

3. Enable pacing on the Model.

4. Send a controlled request series.

5. Open **Realtime** and confirm that requests move at the expected interval.

Pacing applies only to request limits over second and minute periods. Hour, day, and month request limits are hard caps. Token and spend limits are not paced.

## Read cooldown state

A cooldown records the next time that a resource can be eligible. Relay can derive a cooldown from:

- Configured or observed request capacity.
- Provider `Retry-After` information.
- Bounded throttle backoff.
- Provider health after connection, timeout, or `5xx` failures.

A user-specific or API-key-specific limit does not cool the shared Provider or Model. It defers or rejects only that principal's request.

## Test two outcomes

### Wait for the preferred Model

Use a cooldown shorter than the Group maximum wait. Confirm that the request remains queued for the preferred Model.

### Select a fallback

Use a predicted preferred wait longer than the Group maximum wait. Confirm that Relay selects the next eligible ranked membership.

## Expected result

Relay waits when the preferred Model fits the wait budget. It evaluates a fallback only when the preferred wait exceeds that budget.

## Verification

Inspect **Queue**, **Realtime**, and **Logs**. The selected route, predicted eligibility, wait duration, and fallback count must describe the same decision.

## Recovery

- If requests reject too early, inspect the effective wait budget and tighter applicable limits.
- If requests wait too long, reduce the Group maximum wait.
- If capacity releases later than expected, inspect conservative long-window buckets and observed limits.

## Next

[Configure guardrails before and after a model call](https://anchorshell.com/guides/configure-guardrails).