Add a provider, model, and group

Purpose

Create the three objects that Relay needs for grouped routing: a Provider, one or more Models, and a Group with ordered memberships.

Prerequisites

Collect the upstream base URL, authentication method, secret, and exact model identifier. Confirm that the upstream implements the OpenAI-compatible API surface that you plan to call.

Add the provider

1. Open Providers.
2. Select Add provider.
3. Enter a clear provider name.
4. Enter the upstream API root in Base URL.
5. Select the authentication method.
6. Enter the secret only in the credential field.
7. Enable the provider.

Relay encrypts the stored credential. Relay does not return the plaintext credential through the admin API.

Add a model

1. Select Add model on the provider row.
2. Enter the display name.
3. Enter the exact upstream model identifier.
4. Select the supported route kind.
5. Add configured pricing if you need cost estimates or spend limits.
6. Add known request, token, spend, or concurrency limits.
7. Enable pacing only when you want Relay to spread second- or minute-level request capacity.
8. Save the model.

Create the group

1. Open Groups.
2. Select Add group.
3. Enter a stable group name, such as production.
4. Set the maximum wait that callers accept.
5. Enable fallback if Relay may examine lower-ranked memberships.
6. Add model memberships in the required order.

A group membership rank controls grouped routing. A model's direct-route rank does not replace its rank inside a group.

Send a grouped request

curl http://localhost:11730/v1/chat/completions \
  -H 'Authorization: Bearer <RELAY_API_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{"model":"production","messages":[{"role":"user","content":"Return a short status update."}]}'

Response — 200 OK

Example non-streaming response (selected fields). Content, model, and usage depend on the selected provider.

{
  "id": "<COMPLETION_ID>",
  "object": "chat.completion",
  "model": "gpt-5-mini",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Relay waits for the preferred model before considering fallback."
      },
      "finish_reason": "stop"
    }
  ]
}

Expected result

Relay resolves production to its ordered memberships and dispatches the first eligible Model. The upstream receives its exact configured model identifier.

Verification

Open Logs. Confirm that the incoming group is production and that Relay recorded the selected provider/model.

Recovery

If no candidate is available, verify the provider, model, membership, route kind, health state, limits, and group wait budget. Relay does not silently replace an explicitly missing group with a direct model.

Next

Build a free, local, and paid model order.