Compare Relay.
Inspect the differences.
Gateways, infrastructure, and LLMOps tools solve different parts of the same problem. Compare documented capabilities across AnchorShell Relay and eleven alternatives, with definitions and sources you can check.
Capabilities, side by side.
Compare Relay editionsCompetitors 11 of 11
Scroll across for more products. Select a cell for its definition, scope, and evidence. A check may require a paid plan or integration.
Relay labels: “Hosted” includes Free Hosted, Personal, Team, and Enterprise. “All editions” also includes self-hosted Relay. Enterprise offerings require agreed scope.
| Feature | AnchorShell RelayAnchorShell RelayGateway / Observability | HeliconeGateway / Observability | LiteLLMGateway | TrueFoundryGateway / AI Platform | CloudflareGateway | VercelGateway | RequestyGateway | Kong AI GatewayAPI / AI Gateway | Envoy AI GatewayInfrastructure GatewayNow Agent Router | Apache APISIXInfrastructure Gateway | LangfuseObservability / LLMOps | TensorZeroGateway / LLMOpsArchived project |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Core Gateway | ||||||||||||
| Self-hosted | ||||||||||||
| Managed hosted option | ||||||||||||
| OpenAI-compatible API | ||||||||||||
| Multiple model providers | ||||||||||||
| Bring your own provider keys | ||||||||||||
| Connected subscription accounts | ||||||||||||
| Routing & Reliability | ||||||||||||
| Automatic retries | ||||||||||||
| Provider / model fallback | ||||||||||||
| Queue-first routing | ||||||||||||
| Rate-limit-aware request pacing | ||||||||||||
| Characterization-aware routing | ||||||||||||
| Conditional routing policies | ||||||||||||
| Provider-health-aware routing | ||||||||||||
| Request intent / complexity classification | ||||||||||||
| Cost & Usage | ||||||||||||
| Request rate limits | ||||||||||||
| Token rate limits | ||||||||||||
| Spend budgets | ||||||||||||
| Per-user spend budgets | ||||||||||||
| Per-API-key spend budgets | ||||||||||||
| Provider / model usage limits | ||||||||||||
| Custom model pricing | ||||||||||||
| Identity & Governance | ||||||||||||
| Managed, revocable API keys | ||||||||||||
| Granular permissions / RBAC | ||||||||||||
| Security | ||||||||||||
| Prompt / response guardrails | ||||||||||||
| PII detection / DLP | ||||||||||||
| Administrative audit history | ||||||||||||
| Encrypted provider credentials | ||||||||||||
| Observability | ||||||||||||
| Request logs | ||||||||||||
| Usage & cost analytics | ||||||||||||
| Per-user usage & cost analytics | ||||||||||||
| Per-API-key usage & cost analytics | ||||||||||||
| Per-agent usage & cost analytics | ||||||||||||
| Per-team usage & cost analytics | ||||||||||||
| Live routing-flow visualization | ||||||||||||
| Distributed tracing | ||||||||||||
| Developer / LLMOps | ||||||||||||
| Playground | ||||||||||||
| Prompt management | ||||||||||||
| Evaluations | ||||||||||||
| Human review & feedback | ||||||||||||
| Agents | ||||||||||||
| MCP Gateway | ||||||||||||
| Enterprise | ||||||||||||
| Enterprise SSO | ||||||||||||
| SOC 2 offering | ||||||||||||
| HIPAA offering | ||||||||||||
| Air-gapped deployment |
Based on documented capabilities as of September 16, 2026. A dash is not a claim that a product lacks a capability. Availability, plans, integrations, and product scope vary; inspect the evidence before deciding.
Sources & methodology
Published by AnchorShell. We use the same definitions for Relay and every alternative. This is a documentation review, not a performance benchmark, security audit, or ranked recommendation.
We reviewed first-party pricing, product documentation, and official repositories. Relay checks also use its implementation and approved Enterprise offering. A check means documented or advertised support somewhere in the reviewed product range—not every plan.
Retries, fallback, and queue-first routing are separate. So are monetary budgets and token quotas, and administrative audit and request logs. Langfuse’s gateway rows are marked not applicable, not missing.
AnchorShell MCP Gateway is coming soon. This is planned functionality, not current support; release timing and edition availability have not been announced.
User, API-key, agent, and team analytics are compared separately. For Relay, assign one User API key per agent or workload; shared keys combine their usage. Agent analytics means attributed inference usage and cost—not automatic agent detection or a trace of every tool call. The key-level analytics view requires Team or Enterprise organization visibility.
Compliance rows describe scoped commercial offerings, not independent certification of every deployment. Confirm contracts, reports, BAAs, configuration, and upstream-provider obligations with each vendor.
AnchorShell RelayGateway / Observability
Self-hosted plus hosted Free, Personal, Team, and Enterprise. Labels identify edition boundaries. Enterprise security and deployment rows reflect the approved commercial offering, not a claim that every installation is certified.
- Public core and architecturehttps://github.com/anchorshell/bouncer (opens a new tab)
- Queueing and pacinghttps://anchorshell.com/docs/queue (opens a new tab)
- Limits and usagehttps://anchorshell.com/docs/limits (opens a new tab)
- Guardrail integrationshttps://anchorshell.com/docs/guardrails (opens a new tab)
- Hosted smart routinghttps://anchorshell.com/docs/groups (opens a new tab)
- Hosted identity and permissionshttps://anchorshell.com/docs/team (opens a new tab)
- Usage attribution and visibilityhttps://anchorshell.com/docs/usage (opens a new tab)
- Core request characterizationhttps://anchorshell.com/docs/logs (opens a new tab)
- User and API-key limitshttps://anchorshell.com/docs/limits (opens a new tab)
- Connected provider accountshttps://anchorshell.com/docs/providers (opens a new tab)
- Editions and Enterprise offeringhttps://anchorshell.com/pricing (opens a new tab)
- AnchorShell MCP Gateway roadmap statementhttps://anchorshell.com/compare#relay-roadmap (opens a new tab)
Last verified: 2026-09-16
HeliconeGateway / Observability
Public core, cloud service, and higher-tier offerings. LLM Security documents OpenAI-only coverage; token-based custom rate limiting is listed as coming soon.
- Plans and featureshttps://www.helicone.ai/pricing (opens a new tab)
- Official repositoryhttps://github.com/Helicone/helicone (opens a new tab)
- AI Gateway overviewhttps://docs.helicone.ai/gateway/overview (opens a new tab)
- Provider routinghttps://docs.helicone.ai/gateway/provider-routing (opens a new tab)
- Custom rate limitshttps://docs.helicone.ai/features/advanced-usage/custom-rate-limits (opens a new tab)
- LLM Securityhttps://docs.helicone.ai/features/advanced-usage/llm-security (opens a new tab)
- Response cachinghttps://docs.helicone.ai/features/advanced-usage/caching (opens a new tab)
- Prompt managementhttps://docs.helicone.ai/features/advanced-usage/prompts/overview (opens a new tab)
- Evaluation scores and manual reviewhttps://docs.helicone.ai/features/advanced-usage/scores (opens a new tab)
- User and workload cost segmentationhttps://docs.helicone.ai/features/advanced-usage/custom-properties (opens a new tab)
Last verified: 2026-09-16
LiteLLMGateway
Self-hosted open-source and Enterprise gateway. Auto Routing is documented as beta. Rate-aware target selection alone is not counted as queue-first pacing.
- Plans and deploymenthttps://www.litellm.ai/pricing (opens a new tab)
- Feature inventoryhttps://www.litellm.ai/features (opens a new tab)
- Routing and load balancinghttps://docs.litellm.ai/docs/routing (opens a new tab)
- Auto Routing (beta)https://docs.litellm.ai/docs/proxy/auto_routing (opens a new tab)
- Exact and semantic cachinghttps://docs.litellm.ai/docs/proxy/caching (opens a new tab)
- OpenTelemetry v2 tracinghttps://docs.litellm.ai/docs/observability/opentelemetry_v2 (opens a new tab)
- Guardrails and PII maskinghttps://docs.litellm.ai/docs/proxy/guardrails/quick_start (opens a new tab)
- Official Playground implementationhttps://raw.githubusercontent.com/BerriAI/litellm/main/ui/litellm-dashboard/src/app/%28dashboard%29/playground/page.tsx (opens a new tab)
- MCP Gatewayhttps://docs.litellm.ai/docs/mcp (opens a new tab)
- User, key, team, and workload usagehttps://docs.litellm.ai/docs/proxy/cost_tracking (opens a new tab)
- Custom model pricinghttps://docs.litellm.ai/docs/proxy/custom_pricing (opens a new tab)
Last verified: 2026-09-16
TrueFoundryGateway / AI Platform
AI Gateway and its documented platform integrations, including SaaS, VPC, and Enterprise deployment. Virtual-account budgets are not assumed to be independent budgets for every credential issued to that account.
- AI Gateway capabilitieshttps://www.truefoundry.com/docs/ai-gateway/intro-to-llm-gateway (opens a new tab)
- Rate limitinghttps://www.truefoundry.com/docs/ai-gateway/ratelimiting (opens a new tab)
- Budget limiting V2https://www.truefoundry.com/docs/ai-gateway/budget-limiting-v2 (opens a new tab)
- Gateway access controlhttps://www.truefoundry.com/docs/ai-gateway/gateway-access-control (opens a new tab)
- Content-based Auto Routinghttps://www.truefoundry.com/docs/ai-gateway/auto-routing (opens a new tab)
- Exact and semantic response cachinghttps://www.truefoundry.com/docs/ai-gateway/caching (opens a new tab)
- API-key creation and revocationhttps://www.truefoundry.com/docs/generating-truefoundry-api-keys (opens a new tab)
- Enterprise product and deploymenthttps://www.truefoundry.com/ (opens a new tab)
- User/team analytics and custom model priceshttps://www.truefoundry.com/docs/ai-gateway/cost-tracking (opens a new tab)
- User and virtual-account usage metricshttps://www.truefoundry.com/docs/ai-gateway/fetch-model-metrics (opens a new tab)
Last verified: 2026-09-16
CloudflareGateway
Cloudflare AI Gateway, not every Workers or Zero Trust capability. Several controls are beta. Metadata-based user budgets require trustworthy application-supplied identity.
- AI Gateway featureshttps://developers.cloudflare.com/ai-gateway/features/ (opens a new tab)
- Pricing and availabilityhttps://developers.cloudflare.com/ai-gateway/reference/pricing/ (opens a new tab)
- Dynamic routinghttps://developers.cloudflare.com/ai-gateway/features/dynamic-routing/ (opens a new tab)
- Request handlinghttps://developers.cloudflare.com/ai-gateway/configuration/request-handling/ (opens a new tab)
- AI Gateway administrative audit logshttps://developers.cloudflare.com/ai-gateway/reference/audit-logs/ (opens a new tab)
- OpenTelemetry integrationhttps://developers.cloudflare.com/ai-gateway/observability/otel-integration/ (opens a new tab)
- Identity-scoped usage and cost analyticshttps://developers.cloudflare.com/ai-gateway/observability/user-insights/ (opens a new tab)
- Application metadata attributionhttps://developers.cloudflare.com/ai-gateway/observability/custom-metadata/ (opens a new tab)
Last verified: 2026-09-16
VercelGateway
Vercel AI Gateway plus explicitly identified platform security offerings. Budgets exclude BYOK spend and are soft caps checked before requests. The open-source AI SDK is not the hosted gateway.
- AI Gateway overviewhttps://vercel.com/docs/ai-gateway (opens a new tab)
- Gateway pricing and capabilitieshttps://vercel.com/docs/ai-gateway/pricing (opens a new tab)
- Budgets and permissionshttps://vercel.com/docs/ai-gateway/observability-and-spend/budgets (opens a new tab)
- Managed gateway API keyshttps://vercel.com/docs/ai-gateway/authentication-and-byok/api-keys (opens a new tab)
- Routing ruleshttps://vercel.com/docs/ai-gateway/models-and-providers/routing-rules (opens a new tab)
- Gateway security controlshttps://vercel.com/docs/ai-gateway/security-and-compliance (opens a new tab)
- Model access and Playgroundhttps://vercel.com/changelog/claude-sonnet-5-ai-gateway (opens a new tab)
- Platform security offeringhttps://vercel.com/security (opens a new tab)
Last verified: 2026-09-16
RequestyGateway
Hosted Free, pay-as-you-go, and Enterprise offerings. Auto Cache is provider prompt-prefix caching, not completed-response caching. Concurrent-request caps are not request-per-minute policies.
- Plans and feature inventoryhttps://www.requesty.ai/pricing (opens a new tab)
- Gateway and management overviewhttps://docs.requesty.ai/quickstart (opens a new tab)
- Spend limits and provider rate limitshttps://docs.requesty.ai/features/api-limits (opens a new tab)
- API-key lifecycle managementhttps://docs.requesty.ai/features/key-management-api (opens a new tab)
- User spending controlshttps://docs.requesty.ai/features/users (opens a new tab)
- Auto Cache semanticshttps://docs.requesty.ai/features/auto-caching (opens a new tab)
- Official documentation inventoryhttps://docs.requesty.ai/llms.txt (opens a new tab)
- Usage analytics and identity filtershttps://docs.requesty.ai/features/usage-analytics (opens a new tab)
- Cost attribution by member, user, and keyhttps://docs.requesty.ai/features/cost-tracking (opens a new tab)
Last verified: 2026-09-16
Kong AI GatewayAPI / AI Gateway
Kong Gateway core plus commercial AI Gateway policies. The current AI Gateway 2.0 architecture uses a hosted control plane and self-managed data plane; older plugin deployments have different topology support. A core open-source check does not make commercial AI policies open source.
- Open-source Kong Gatewayhttps://github.com/Kong/kong (opens a new tab)
- AI Gateway overviewhttps://developer.konghq.com/ai-gateway/ (opens a new tab)
- AI Gateway architecture and routinghttps://developer.konghq.com/ai-gateway/architecture/ (opens a new tab)
- AI policies and integrationshttps://developer.konghq.com/ai-gateway/policies/ (opens a new tab)
- AI Consumers and credentialshttps://developer.konghq.com/ai-gateway/entities/ai-consumer/ (opens a new tab)
- Per-application API keys and revocationhttps://developer.konghq.com/cookbooks/mistral-ai-with-kong-ai-gateway/ (opens a new tab)
- Token and monetary rate-limit policieshttps://developer.konghq.com/ai-gateway/policies/ai-rate-limiting-advanced/ (opens a new tab)
- Consumer-scoped cost analyticshttps://developer.konghq.com/cookbooks/llm-cost-optimization/ (opens a new tab)
Last verified: 2026-09-16
Envoy AI GatewayInfrastructure Gateway
The project is now named Agent Router; original Envoy AI Gateway links redirect to it. This column reviews the same open-source project and its 1.1 documentation, not a separate vendor’s managed offering.
- Project and rename (Agent Router)https://github.com/theagentrouter/agent-router (opens a new tab)
- Gateway capabilitieshttps://theagentrouter.ai/docs/capabilities/ (opens a new tab)
- Provider fallback and retrieshttps://theagentrouter.ai/docs/capabilities/traffic/provider-fallback/ (opens a new tab)
- Usage-based rate limitinghttps://theagentrouter.ai/docs/capabilities/traffic/usage-based-ratelimiting (opens a new tab)
- Token quota policieshttps://theagentrouter.ai/docs/capabilities/traffic/quota-policy/ (opens a new tab)
- Logs, metrics, and distributed traceshttps://theagentrouter.ai/docs/capabilities/observability/ (opens a new tab)
Last verified: 2026-09-16
Apache APISIXInfrastructure Gateway
Apache APISIX and its AI gateway plugins. API7 commercial services are not attributed to the Apache project. Plugin configuration is required; an infrastructure capability does not imply a turnkey hosted workflow.
- Apache APISIX AI Gatewayhttps://apisix.apache.org/ai-gateway/ (opens a new tab)
- Leaky-bucket request pacinghttps://apisix.apache.org/docs/apisix/plugins/limit-req/ (opens a new tab)
- Multiple consumer credentialshttps://apisix.apache.org/docs/apisix/terminology/credential/ (opens a new tab)
- Credential lifecycle APIhttps://apisix.apache.org/docs/apisix/admin-api/#credential (opens a new tab)
Last verified: 2026-09-16
LangfuseObservability / LLMOps
An observability, evaluation, and prompt-management platform, not primarily an inference gateway. Gateway-enforcement rows are not applicable to the product scope reviewed. Cloud and self-hosted plans have different collaboration and compliance entitlements.
- Cloud and self-hosted planshttps://langfuse.com/pricing (opens a new tab)
- LLM observability and traceshttps://langfuse.com/docs/observability/overview (opens a new tab)
- Evaluation workflowshttps://langfuse.com/docs/evaluation/overview (opens a new tab)
- Token costs, user metrics, and custom priceshttps://langfuse.com/docs/observability/features/token-and-cost-tracking (opens a new tab)
- Workload and agent telemetry segmentationhttps://langfuse.com/docs/observability/features/tags (opens a new tab)
Last verified: 2026-09-16
TensorZeroGateway / LLMOps
Self-hosted open-source LLMOps stack. The official repository was archived on June 12, 2026; checks describe its published code/documentation, not a promise of ongoing maintenance or a hosted service.
- Official repository and archived statushttps://github.com/tensorzero/tensorzero (opens a new tab)
- Gateway API-key lifecyclehttps://raw.githubusercontent.com/tensorzero/tensorzero/main/docs/operations/set-up-auth-for-tensorzero.mdx (opens a new tab)
- Request, token, and monetary budgetshttps://raw.githubusercontent.com/tensorzero/tensorzero/main/docs/operations/enforce-custom-rate-limits.mdx (opens a new tab)
- Inference response cachinghttps://raw.githubusercontent.com/tensorzero/tensorzero/main/docs/gateway/guides/inference-caching.mdx (opens a new tab)
Last verified: 2026-09-16
Choose how you run Relay.
Self-host the Relay core or start with Free Hosted. Compare the operating model, capacity, and controls.