Request compression
Long conversations resend system instructions, tool schemas, source files, and tool results on each turn. Rho gzip-compresses large JSON inference requests where the receiving endpoint is known to accept Content-Encoding: gzip. Custom hosts can opt in to zstd or gzip with request_compression.
This is request-body compression, not Accept-Encoding response negotiation. Streaming responses remain SSE. Codex WebSocket messages are not HTTP request bodies and are unchanged.
Behavior
- Automatic gzip on verified first-party Anthropic Messages, Google Gemini Generate Content, xAI REST, and TypeSafe System One endpoints, including streaming turns where applicable.
- Requests smaller than 16 KiB (16,384 serialized bytes) stay JSON. Compression uses the fast gzip level, and the compressed body is sent only if it is smaller.
Content-Type: application/json, credentials, provider headers, and SSE negotiation are preserved. The HTTP client computes the compressedContent-Lengthrather than retaining the original length.- Existing content encodings and streaming request bodies are not recompressed.
- For built-in providers, support is checked against the actual HTTPS host, port, and inference path. A custom endpoint does not inherit support from its provider name, models.dev catalog, or OpenAI-compatible dialect.
- Unsupported and unverified endpoints keep ordinary JSON unless a custom host sets
request_compression. Rho does not replay a generation after a speculative compressed request: an ambiguous failure could otherwise duplicate a billed turn.
Custom hosts
A custom OpenAI-compatible host can opt in with request_compression = "zstd" or "gzip" in its [providers.custom.<name>] table. This tells Rho that the deployment decodes that encoding. It replaces the allowlist for that host on Chat Completions and Responses (and Messages for SDK-declared hosts), over http or https and on any port. The same 16 KiB threshold and the "send only if smaller" rule apply. zstd uses level 3: on a 1 MB Rho session body it produced 153,718 bytes in 1.9 ms, where gzip's fast level produced 211,484 bytes.
Compression pays off only when upload bandwidth is the bottleneck, as on a WAN or VPN link to a remote proxy. On loopback, compressing a 1.4 MB body to CLIProxyAPI took longer (about 11–13 ms) than sending it plain (about 7–10 ms).
Built-in providers cannot opt in, so the allowlist still governs them. ProviderBuildOptions::request_compression returns an error for a built-in provider rather than ignoring the setting.
The threshold rationale and performance measurements are recorded in PR #1524.
Provider support audit
Gzip support must be established for requests. A server emitting gzipped responses, advertising Accept-Encoding, or returning the same early authentication failure for both payloads is not proof that it decodes gzip requests.
T3 Code #4788 compresses negotiated JSON responses carrying thread snapshots. Outbound model requests need their own endpoint compatibility check; there is no equivalent universal Accept-Encoding negotiation for them.
| Provider / endpoint | Finding | Rho behavior |
|---|---|---|
Anthropic api.anthropic.com/v1/messages | Confirmed: Claude Code's documented gzip request behavior, restricted to api.anthropic.com. | Automatic gzip at/above the threshold. |
Google Gemini generativelanguage.googleapis.com/v1beta/models/*:generateContent and :streamGenerateContent | Confirmed decoder with malformed-JSON controls (below), including the SSE endpoint. Generate Content API. | Automatic gzip at/above the threshold. |
xAI api.x.ai/v1/responses, /responses/compact, /chat/completions | Confirmed decoder: actual gzip reaches matching plaintext JSON/schema validation; corrupt gzip fails body buffering. API specification. | Automatic gzip at/above the threshold, including native HTTP compaction. |
TypeSafe api.typesafe.ai/v1/systemone | Explicit request compression contract, including gzip/x-gzip and zstd. | Automatic gzip at/above the threshold. Existing request-size budgets still apply to the serialized JSON before compression. |
OpenAI api.openai.com/v1/responses, /responses/compact, /chat/completions | Unverified: probes stop at authentication; no affirmative gzip contract in the Responses or Chat references. | Ordinary JSON. |
Codex HTTP chatgpt.com/backend-api/codex/responses | Gzip unverified. The official client uses zstd request encoding, selected for eligible Codex backend authentication. | Ordinary JSON for HTTP fallback/remote compaction; gzip is not substituted for zstd. |
Codex WebSocket wss://chatgpt.com/backend-api/codex/responses | HTTP body gzip does not apply. The official client negotiates WebSocket permessage-deflate. | Unchanged JSON frames. WebSocket extension negotiation and HTTP zstd are separate work. |
GitHub Copilot api.githubcopilot.com/chat/completions and token-selected hosts | Unverified: probes fail the Authorization check before decoding can be observed. First-party transport. | Ordinary JSON. |
OpenRouter openrouter.ai/api/v1/chat/completions | Unverified: all controls stop at authentication; the API contract does not establish request gzip. | Ordinary JSON. |
Ollama local /v1/chat/completions | Not decoded by inspected stock server: middleware handles zstd only, and gzip reaches JSON binding undecoded. A configured LAN Ollama instance also rejected actual gzip at the gzip magic byte. | Ordinary JSON. |
Ollama Cloud ollama.com/v1/chat/completions | Unverified: ordinary JSON, gzip, and corrupt gzip all return early 401s. Local-server behavior does not prove cloud ingress behavior. Compatibility docs. | Ordinary JSON. |
Moonshot api.moonshot.ai/v1/chat/completions | Unverified end-to-end: valid gzip reaches authentication while corrupt gzip fails earlier, suggesting ingress recognition but not proving API decoding. Endpoint. | Ordinary JSON. |
Kimi Code api.kimi.com/coding/v1/chat/completions | Unverified: all controls stop at authentication. Endpoint documentation. Do not inherit Moonshot's partial evidence. | Ordinary JSON. |
Poolside inference.poolside.ai/v1/chat/completions | Rejects actual gzip in controlled probes: plain invalid-model JSON reaches authentication, while the compressed equivalent gets 400 Invalid request body. API docs. | Ordinary JSON. |
Alibaba/Qwen Token Plan token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions | Unverified: Rho's exact endpoint returns 401 for every control. Token Plan docs. Other Alibaba/Qwen hostnames are not interchangeable capability evidence. | Ordinary JSON. |
Meta api.meta.ai/v1/chat/completions | Unverified: all controls return 401 invalid_api_key. API docs. | Ordinary JSON for both API-key and Muse authentication. |
OpenCode Go opencode.ai/inference/go/v1/{chat/completions,responses,messages}; Zen opencode.ai/zen/v1/{chat/completions,responses,messages} | Likely rejects (moderate confidence): public proxy preserves compressed bytes and encoding, with no decompressor. Probes produce different 400/500 errors, but the downstream inference implementation is unavailable. | Ordinary JSON on every adapter; no inherited Anthropic/Responses capability. |
MiniMax api.minimax.io/anthropic/v1/messages | Unverified: controls stop at authentication; the Messages specification does not promise gzip. | Ordinary JSON. |
Mistral api.mistral.ai/v1/chat/completions | Not audited: added after the audit snapshot below; the Chat Completions reference does not promise gzip. | Ordinary JSON. |
Stock CLIProxyAPI, custom /responses, /chat/completions, /messages | zstd on Responses/Chat, no gzip: the Chat/Responses decoder handles only zstd and rejects gzip with unsupported request content encoding; Messages reads raw bytes. A live deployment (2026-10-18) decoded zstd that contained a bogus model name and reported that name, while gzip and corrupt zstd failed at the decoder. A real Rho Responses turn with request_compression = "zstd" succeeded, shrinking the request from 50,994 to 16,725 bytes. | Ordinary JSON by default. Set request_compression = "zstd" on the custom host to opt in. |
| Other custom OpenAI-compatible endpoints | Deployment-specific: a shared JSON schema is not a transport contract. | Ordinary JSON by default; opt in with request_compression when the deployment decodes it. |
Audit snapshot: 2026-10-17 UTC, covering the 17 built-in providers shipped then plus CLIProxyAPI, OpenCode Zen, and arbitrary custom hosts. Mistral was added later and has not been probed. No credentials were read and no authenticated generation was requested for the audit. Positive Google/xAI controls establish body decoding, not a completed authenticated generation or SSE stream. The body decoder acts before authentication/schema handling; streaming flags do not change the compressed JSON representation. Unknown results are deliberately described as unverified, not unsupported. Vertex AI and differently hosted proxies are not covered by the Gemini finding.
Header-only tests are insufficient: CLIProxyAPI can accept plain JSON incorrectly labelled Content-Encoding: gzip via its raw-JSON fallback while rejecting actual gzip bytes. The probes use actual compressed bodies and corrupt-gzip controls, not just the header.
Verifying a decoder without a paid generation
For Gemini's unary and streaming endpoints, malformed JSON provides a useful control. With an invalid probe key, gzipped {not-json returns a JSON parse error quoting the decompressed text; gzipped {} advances to API_KEY_INVALID. A corrupt gzip stream instead fails payload parsing. This establishes request decoding, not successful authenticated inference.
Larger controls used a 1,048,605-byte malformed JSON object compressed to 1,084 bytes. Gemini's unary/streaming routes reached schema validation on the decompressed padding field. xAI create/compact reported the malformed gzip_probe value at plaintext column 1,048,604, demonstrating decoding beyond the production size threshold without requesting generation.
For example (repeat with streamGenerateContent?alt=sse):
printf '{not-json' | gzip -c | curl -sS \
'https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent' \
-H 'Content-Type: application/json' \
-H 'Content-Encoding: gzip' \
-H 'x-goog-api-key: invalid-probe' --data-binary @-