Skip to content

Custom OpenAI-compatible hosts ​

You can point Rho at any local or remote host that speaks OpenAI Chat Completions or the OpenAI Responses API. Add a name and a base URL in config, or create a host from /login with Custom · Chat Completions or Custom · Responses. Requests send an Authorization header only when you store an API key.

This is not the first-party OpenAI provider. First-party OpenAI API-key requests currently use Chat Completions. Custom hosts that set api = "responses" use the same {base}/responses HTTP contract as OpenAI's Responses API (not Codex WebSocket). If you know Codex's wire_api setting, that is a different surface; here the key is api.

Define a host ​

The fastest path is /login in the interactive TUI. Choose Custom · Chat Completions or Custom · Responses, name the provider, enter its base URL, then enter an API key or leave that field blank.

You can also edit ~/.rho/config.toml. The table key is the provider name used in /model.

toml
[providers.custom.vllm]
base_url = "http://127.0.0.1:8000/v1"
edit_tool = "apply_patch"

edit_tool is optional. It sets the format that [behavior].edit_tool = "auto" uses for this provider. Choose str_replace or apply_patch. Without it, auto uses str_replace. A pinned global edit tool still wins.

A mixed proxy that is not itself in models.dev can borrow another catalog for context windows, prices, and reasoning lists. Set catalog to that models.dev provider slug. Model ids must match the borrowed catalog (gpt-5.6-sol, not openai/gpt-5.6-sol):

toml
[providers.custom.cliproxyapi]
base_url = "http://127.0.0.1:8317/v1"
catalog = "llmgateway"

llmgateway is a mixed models.dev catalog with bare model ids. openrouter only matches if the host uses OpenRouter-style owner/model ids. openai-codex borrows Rho's Codex catalog, including built-in window overrides. Requests still go to the custom host; only metadata is borrowed. For one model that should use a different slug, set catalog on that row in ~/.rho/models.toml. See local model metadata.

If the host already pushes provider/model ids (anthropic/claude-sonnet-4-5), set catalog_mode = "model-id" instead of a borrowed slug. Rho looks the unsplit id up in models.dev (foo/bar/baz stays one catalog id). When no models.dev provider serves that id under its slug, Rho uses the canonical model entry instead, which has limits and modalities but no prices; see Local model metadata. A host cannot set both catalog and catalog_mode = "model-id". A bare id with no slash misses catalog metadata and inserts a transcript notice. Per-model catalog in models.toml still wins. Run /refresh-models to redownload that snapshot on demand.

Keep the /v1 suffix. Rho appends /models for discovery. Agent turns append /chat/completions by default, or /responses when api = "responses". The URL must use http or https and cannot contain credentials, a query, or a fragment.

Chat Completions is the default so existing hosts keep working. To speak Responses instead, pick Custom · Responses in /login, or set api in config:

toml
[providers.custom.litellm]
base_url = "http://127.0.0.1:4000/v1"
api = "responses"

api accepts "chat-completions" (default, omitted on save) or "responses". Restart Rho after you hand-edit this table, including api, base_url, or edit_tool. Creating or re-creating a host through /login applies immediately and writes api to the Custom row you pick. Hand-editing only base_url keeps previously saved api and edit_tool values.

Names must be lowercase letters, digits, and hyphens, start with a letter, and must not match a built-in provider.

Request compression ​

Long conversations resend the whole transcript every turn. If the host decodes compressed request bodies, set request_compression so JSON requests of 16 KiB or more go out as "zstd" or "gzip" (sent only when that makes them smaller):

toml
[providers.custom.cliproxyapi]
base_url = "http://node0:8317/v1"
api = "responses"
request_compression = "zstd"

This is your claim about the host, and Rho does not check it: a host that cannot decode the body rejects every large turn. Rho also does not retry a rejected turn as plain JSON. Stock CLIProxyAPI decodes zstd (not gzip) on /responses and /chat/completions. Leave the setting unset for hosts on the same machine or LAN, where compressing costs more time than it saves. See request compression.

Authentication ​

Custom hosts default to auth = "none" and send no Authorization header. If the host requires a key, store one with /login <name> or during onboarding. That selects {name}-api-key and sends Authorization: Bearer <key>. Restart keeps a keyed profile. Startup only promotes leftover none when a key is stored; it does not write none over a keyed profile. Secrets stay in the credential store, not in config.

toml
[model]
provider = "vllm"
model = "qwen2.5-coder"
auth = "none"

With a stored key:

toml
[model]
provider = "vllm"
model = "qwen2.5-coder"
auth = "vllm-api-key"

Leave the /login key field blank to keep a stored or env-supplied key, or to run keyless when none is set. /logout <name> deletes a stored key. The CI/development override is RHO_<NAME>_API_KEY, with the provider name uppercased and hyphens turned into underscores (RHO_VLLM_API_KEY). The override applies to hosts defined in config.toml; Rho also strips those names from agent command environments so a tool cannot read them.

Use it ​

After a restart, or immediately after /login onboarding, Rho fetches /v1/models in the background for each custom host so the picker can fill in. You can also refresh by hand with /refresh-models if the host was down at startup.

text
/model vllm/qwen2.5-coder

Models ​

Rho fetches /v1/models in the background at startup for every custom host so the picker can fill in. A down host is skipped so startup still succeeds; refresh later with /refresh-models once it is up. rho run and rho acp wait only for the host the session uses, and only when its cached list is over a day old or lacks the selected model; in ACP, a host that answers late shows up in the model picker of the next session. Opening /model before the fetch lands can show a stale or empty custom list. The host must support tool calls if you want a coding agent. Proxies such as CLIProxyAPI, and many local servers, omit tool-call ids, use sparse indexes, or skip {} for zero-argument tools. Rho fills those in so the tool loop can continue. First-party OpenAI-compatible providers stay strict.

Rho sends reasoning_effort on each turn, including "none" when reasoning is off. Shift+Tab cycles the level, and /config → Models → Reasoning sets it. Hosts that do not accept that field may reject the request; pin levels in ~/.rho/models.toml if you need a smaller set.

When the session has a prompt cache key, Rho also sends prompt_cache_key on Chat Completions and Responses so compatible hosts and proxies can pin prompt cache across turns. The field is omitted when there is no key.

Responses hosts reuse OpenAI's Responses request shape, including reasoning when catalog metadata supplies it. Borrow a catalog with catalog or catalog_mode if you want models.dev reasoning lists and windows; unknown models may omit reasoning.effort. Rho does not inject OpenAI hosted web_search on custom hosts. Native /responses/compact is attempted when the session compact path asks for it; hosts that 404 fall back to summary compaction.

Automation ​

bash
rho --provider vllm --model vllm/qwen2.5-coder run "review this project"