Open source · Written in Rust

The fastest programmable gateway for models, tools and services.

Arion is an ultra-high-performance L7 data plane written in Rust. One data plane, multiple gateways: an LLM gateway for model traffic, an MCP gateway for agent tools, an API gateway and L7 proxy for everything else — speaking Envoy configuration over xDS.

Get started See the benchmarks

docker run --rm -p 8080:8080 ghcr.io/arion-gateway/arion:latest

01 · Gateways

One data plane, multiple gateways.

LLM LLM gateway One endpoint for several providers, with failover, token budgets and guardrails. Guides: LLM gateway →

MCP MCP gateway Turn REST and OpenAPI services into tools agents can call, with per-tool authorization. Guide: Build an MCP gateway → code mode ↓

L7 L7 proxy and API gateway Load balancing, routing, TLS, JWT, rate limits. Guide: Build an API gateway →

02 · Benchmarks

The fastest in the ecosystem.

Throughput 2.1M requests per second on 24 cores Versus Envoy 7x the throughput of Envoy and agentgateway

Only nginx comes close on speed. Only Arion is this programmable.

Program the data path using

1xDS
2Rust HttpFilters
3WASM filters
4External Processing (ext_proc)

See the benchmarks

Throughput at 24 cores, req/s · basic forwarding

03 · Extend

Programmable at every layer.

WASM filters

Write filters in Rust with the Arion SDK: async HTTP and gRPC callouts, shared state across instances.

auth_filter.rsrust

#[arion_plugin]
impl Plugin for AuthFilter {
    fn on_request_headers(&mut self, ctx: &RequestHandle<HttpHeaders>) -> FilterAction {
        match ctx.get_header("authorization") {
            Ok(Some(v)) if is_valid(&v) => FilterAction::Continue,
            _ => ctx.direct_response(unauthorized()),
        }
    }
}

External Processing (ext_proc)

Hand requests to an out-of-process server over gRPC, with per-route overrides.

arion.yamlyaml

- name: envoy.filters.http.ext_proc
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_proc.v3.ExternalProcessor
    failure_mode_allow: false
    processing_mode:
      request_header_mode: SEND
      response_header_mode: SKIP
    message_timeout: 5s
    grpc_service:
      envoy_grpc: { cluster_name: ext_proc_server }

Rust HttpFilters

Compile a native filter into Arion in Rust: async, with full access to the request and nothing between your code and the data path.

tenant_filter.rsrust

// A native filter, compiled into Arion
pub async fn apply_request(
    &mut self,
    request: &mut Request<ArionRequestBody>,
    ctx: &RequestCtx,
) -> FilterDecision {
    let tenant = self.tenants.lookup(request.headers());
    request.headers_mut().insert("x-tenant", tenant);
    FilterDecision::Continue
}

04 · MCP gateway · Code mode

MCP Code Mode: sandboxed code execution -- in the gateway.

Arion's MCP gateway supports REST APIs, OpenAPI documents, upstream MCP servers and serves two distinct modes.
Tools mode is conventional MCP: one call per model turn.
Code mode provides all tools as an SDK available to agent generated code which Arion executes in a sandbox.

Fewer turns. Less Tokens.

In code mode, Arion publishes just one tool called 'execute'. Arion executes the agent's code in a sandbox, in the context of the gateway, with full policy enforcement applied to every upstream call.

Measure	tools mode	code mode
Model turns	4	1
Model ↔ gateway crossings	8	2
Raw results in context	4	0

Same tools, same guardrails: every call a script makes passes the same authorization, schema checks and limits as a direct tool call.

How code mode works Try the guide

Agent Programmability

Work involving multiple tool calls can be batched programatically: look up a customer, fetch history and plan in parallel, branch, open a ticket.

execute.tstypescript

// One execute call: look up, fan out, decide, act.
const customer = await tools["crm.find_customer"]({ email: "ada@acme.io" });
if (!customer.ok) return { error: "unknown customer" };

const [history, plan] = await Promise.all([
  tools["support.recent_tickets"]({ customer_id: customer.data.id }),
  tools["billing.current_plan"]({ customer_id: customer.data.id }),
]);

const ticket = await tools["support.create_ticket"]({ ticket: {
  customer_id: customer.data.id,
  priority: plan.data.tier === "enterprise" ? "high" : "normal",
  summary: "Refund not received",
} });

return { ticket: ticket.data.id, seen: history.data.items.length };

05 · Builder

Compose a config.

FeaturesAPI gateway

jwt issuer https://idp.example.comJWT authentication

Listener: HTTP listener
HTTP filters: Rate limit; JWT authentication; Router
Routes: Route to cluster; Retries
Clusters: Cluster; HTTP health check

25- name: envoy.filters.http.jwt_authn
26  typed_config:
27    "@type": type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.JwtAuthentication
28    providers:
29      idp:
30        issuer: https://idp.example.com
31        from_headers:
32          - name: Authorization
33            value_prefix: "Bearer "
34        payload_in_metadata: jwt_payload
35        remote_jwks:
36          http_uri:
37            uri: https://idp.example.com/.well-known/jwks.json
38            cluster: idp_jwks

Describe a feature in a few words. The Builder puts it where Arion applies it, adds what the config needs, and writes the complete Envoy-compatible file for you to edit and copy.

Open in Builder

07 · Blog

From the blog.

All posts

Explore Arion’s gateway model.

Browse docs GitHub