Custom OpenAI-compatible hosts

You can point Rho at any local or remote host that speaks OpenAI Chat Completions or the OpenAI Responses API. Add a name and a base URL in config, or create a host from /login with Custom · Chat Completions or Custom · Responses. Requests send an Authorization header only when you store an API key.

This is not the first-party OpenAI provider. First-party OpenAI API-key requests currently use Chat Completions. Custom hosts that set api = "responses" use the same {base}/responses HTTP contract as OpenAI's Responses API (not Codex WebSocket). If you know Codex's wire_api setting, that is a different surface; here the key is api.

Define a host

The fastest path is /login in the interactive TUI. Choose Custom · Chat Completions or Custom · Responses, name the provider, enter its base URL, then enter an API key or leave that field blank.

You can also edit ~/.rho/config.toml. The table key is the provider name used in /model.

[providers.custom.vllm]
base_url = "http://127.0.0.1:8000/v1"
edit_tool = "apply_patch"

edit_tool is optional. It sets the format that [behavior].edit_tool = "auto" uses for this provider. Choose hashline, apply_patch, or str_replace. A pinned global edit tool still wins.

A mixed proxy that is not itself in models.dev can borrow another catalog for context windows, prices, and reasoning lists. Set catalog to that models.dev provider slug. Model ids must match the borrowed catalog (gpt-5.6-sol, not openai/gpt-5.6-sol):

[providers.custom.cliproxyapi]
base_url = "http://127.0.0.1:8317/v1"
catalog = "llmgateway"

llmgateway is a mixed models.dev catalog with bare model ids. openrouter only matches if the host uses OpenRouter-style owner/model ids. openai-codex borrows Rho's Codex catalog, including built-in window overrides. Requests still go to the custom host; only metadata is borrowed. For one model that should use a different slug, set catalog on that row in ~/.rho/models.toml. See local model metadata.

If the host already pushes provider/model ids (anthropic/claude-sonnet-4-5), set catalog_mode = "model-id" instead of a borrowed slug. Rho looks the unsplit id up in models.dev (foo/bar/baz stays one catalog id). A host cannot set both catalog and catalog_mode = "model-id". A bare id with no slash misses catalog metadata and inserts a transcript notice. Per-model catalog in models.toml still wins. Open /config, choose Providers, then Refresh models.dev catalog to redownload that snapshot on demand.

Keep the /v1 suffix. Rho appends /models for discovery. Agent turns append /chat/completions by default, or /responses when api = "responses". The URL must use http or https and cannot contain credentials, a query, or a fragment.

Chat Completions is the default so existing hosts keep working. To speak Responses instead, pick Custom · Responses in /login, or set api in config:

[providers.custom.litellm]
base_url = "http://127.0.0.1:4000/v1"
api = "responses"

api accepts "chat-completions" (default, omitted on save) or "responses". Restart Rho after you hand-edit this table, including api, base_url, or edit_tool. Creating or re-creating a host through /login applies immediately and writes api to the Custom row you pick. Hand-editing only base_url keeps previously saved api and edit_tool values.

Names must be lowercase letters, digits, and hyphens, start with a letter, and must not match a built-in provider.

Authentication

Custom hosts default to auth = "none" and send no Authorization header. If the host requires a key, store one with /login <name> or during onboarding. That selects {name}-api-key and sends Authorization: Bearer <key>. Restart keeps a keyed profile. Startup only promotes leftover none when a key is stored; it does not write none over a keyed profile. Secrets stay in the credential store, not in config.

[model]
provider = "vllm"
model = "qwen2.5-coder"
auth = "none"

With a stored key:

[model]
provider = "vllm"
model = "qwen2.5-coder"
auth = "vllm-api-key"

Leave the /login key field blank to keep a stored or env-supplied key, or to run keyless when none is set. /logout <name> deletes a stored key. The CI/development override is RHO_<NAME>_API_KEY, with the provider name uppercased and hyphens turned into underscores (RHO_VLLM_API_KEY). The override applies to hosts defined in config.toml; Rho also strips those names from agent command environments so a tool cannot read them.

Use it

After a restart, or immediately after /login onboarding, Rho fetches /v1/models in the background for each custom host so the picker can fill in. You can also refresh by hand in /config if the host was down at startup.

/model vllm/qwen2.5-coder

Models

Rho fetches /v1/models in the background at startup for every custom host so the picker can fill in. A down host is skipped so startup still succeeds; refresh later in /config once it is up. Opening /model before the fetch lands can show a stale or empty custom list. The host must support tool calls if you want a coding agent. Proxies such as CLIProxyAPI, and many local servers, omit tool-call ids, use sparse indexes, or skip {} for zero-argument tools. Rho fills those in so the tool loop can continue. First-party OpenAI-compatible providers stay strict.

Rho sends reasoning_effort on each turn, including "none" when reasoning is off. Shift+Tab and /config cycle the level. Hosts that do not accept that field may reject the request; pin levels in ~/.rho/models.toml if you need a smaller set.

When the session has a prompt cache key, Rho also sends prompt_cache_key on Chat Completions and Responses so compatible hosts and proxies can pin prompt cache across turns. The field is omitted when there is no key.

Responses hosts reuse OpenAI's Responses request shape, including reasoning when catalog metadata supplies it. Borrow a catalog with catalog or catalog_mode if you want models.dev reasoning lists and windows; unknown models may omit reasoning.effort. Rho does not inject OpenAI hosted web_search on custom hosts. Native /responses/compact is attempted when the session compact path asks for it; hosts that 404 fall back to summary compaction.

Automation

rho --provider vllm --model vllm/qwen2.5-coder run "review this project"