Configure guardrails before and after a model call

Purpose

Call an external or locally hosted HTTP policy service before provider dispatch, after a non-streaming response, or at both stages.

Relay applies the action that you configure. The policy service supplies the verdict. Relay does not include a built-in moderation classifier.

Prerequisites

Collect the policy service URL, authentication method, request template, verdict mapping, timeout, and failure policy. Decide whether the Guardrail binds to one Group, Provider, or Model.

Create the Guardrail

1. Open Guardrails.
2. Add a Guardrail.
3. Select an editable preset or a custom HTTP service.
4. Enter the service URL and authentication settings.
5. Select Pre-dispatch, Post-response, or both.
6. Map the service response to allowed and blocked verdicts.
7. Select the action for a matched rule.
8. Select fail_open, fail_closed, or return_error for service failures.
9. Bind the Guardrail to exactly one Group, Provider, or Model.
10. Save and enable the Guardrail.

Test before provider dispatch

1. Use the explicit Guardrail test action with representative bounded input.
2. Send a non-streaming request that matches the configured block rule.
3. Open Logs.

A pre-dispatch block must show that the Provider was not called. No provider usage or cost occurs.

Test after the model response

1. Enable a post-response rule.
2. Send a non-streaming request that produces a matching response.
3. Open Logs.

A post-response block withholds or replaces the response. Provider usage and cost remain because the Provider already completed the request.

Streaming boundary

If an effective Guardrail applies, Relay rejects streaming before provider dispatch. The response uses guardrails_require_non_streaming.

Expected result

Relay applies the configured service verdict at the selected stage. A pre-dispatch block prevents provider use. A post-response block retains provider use.

Verification

Confirm the stage, service result, action, failure policy, binding, and final request status in Logs. Guardrail results must not mark the Provider unhealthy or trigger model fallback.

Recovery

If the service fails, inspect its URL, timeout, authentication, template, and verdict mapping. Confirm that the chosen failure policy matches the intended security posture.

Next

Handle provider 429 responses.