SDK providers
Current provider surface
ModelProvider is the provider extension point. It is object-safe, Send + Sync, and returns explicit Send futures. A provider reports an exact ModelIdentity containing provider, API, and model names, and receives a borrowed ModelRequest for each turn.
The current core crate includes ScriptedProvider for deterministic examples and tests. rho-sdk does not automatically expose or configure production provider implementations used by the Rho application, and the SDK has no provider Cargo features today. Do not infer SDK availability from the application's provider list.
Implementor contract
A provider implementation must:
- return a stable, exact identity for replay decisions
- accept provider-neutral history and tool schemas without mutating session history
- normalize output into
ModelResponse,ModelEvent,ModelUsage, and sanitizedProviderErrorvalues - cooperate with the request's
CancellationToken - avoid blocking the async runtime thread
- apply bounded buffering and backpressure when bridging another streaming API
- keep credentials, authorization headers, signed URLs, raw secret-bearing payloads, and transport-specific errors out of public
Debug, events, diagnostics, and error messages - tag opaque provider-native replay data with its exact producing provider/API/model identity and a
ReplayScope, and filter replay throughProviderContextBlock::is_replayable_to - lower history through
Message::wire(), which renders SDK entries (compaction summaries, model context, boundary input) as the same user-role content every provider sends, and decide whatWireMessage::ReasoningShiftsends (usually nothing) - replay
WireMessage::AbortedAssistantwithout itsreasoning_summary, so a turn's partial reasoning never reaches a model as plain text; aborted reasoning travels only as opaque provider context
send_turn is the non-streaming primitive. send_turn_stream is the streaming method; it receives the request snapshot and a ProviderTurnContext carrying the bounded event sender (events), per-request ModelRequestOptions (options), and an optional mid-turn steering port (steering). Its default implementation ignores options, drops the steering port (which releases every steering request to the turn boundary), and invokes send_turn while observing cancellation. Providers that support any option or mid-turn steering override it. native_compact takes the same ModelRequestOptions, so native compaction sees the session service tier. Streaming providers send semantic ModelEvent values through the supplied bounded sender and still return a complete normalized response. The final response, not accumulated deltas, is the authoritative completed turn.
A provider whose turn ended at its output-token limit, or that refused or filtered the output, sends ModelEvent::Stopped(ModelStopReason::MaxTokens) or ModelEvent::Stopped(ModelStopReason::Refusal) before returning the response. A turn without one finished normally. The runtime consumes the event, so it never reaches history or RunEvent::ProviderContextUpdated, and reports the final turn's reason as the run's StopReason.
A provider that accepted a tool call as async sends ModelEvent::ToolCallAccepted { id, execution: ToolExecutionMode::Async }. The runtime sets ToolCall::execution on the stored call; adapters that replay the flag read it from history for calls their own model accepted.
Requests can include live host context appended by a RequestContext source as Message::ModelContext. WireMessage sends it as labelled user-role content; never classify it as a new human instruction. Recognition is attribution only and must never authorize an action. These request-only messages do not enter committed conversation history.
Session::set_reasoning_level appends Message::ReasoningShift(ReasoningShift { from, to }) once the conversation has started (consecutive changes collapse into one). Providers that pin effort mid-conversation without busting the cached prefix rebuild it from these entries on every request, including after resume; the Anthropic adapter sends an empty role: system message carrying the new effort before the next user turn on models with per-message effort. Other providers send nothing for it.
Credentials and endpoints
The core SDK does not acquire credentials. It does not read provider environment variables, Rho config files, or an operating-system credential store. An embedding host should construct its provider with an injected secret value or a narrow credential adapter.
Credential-bearing provider structs need custom Debug implementations that omit or redact values. Errors should classify authentication, rate limiting, transport, invalid responses, interruption, and other failures without embedding raw response bodies. Endpoint overrides, proxy behavior, TLS roots, timeouts, and retry policy belong to the adapter and must be explicit.
The Rho CLI has separate application-level authentication behavior documented in authentication and models. That behavior is not an ambient SDK default.
Streaming and cancellation
The provider receives the run's shared cancellation token. The runtime also races the provider future against cancellation, so it stops polling the future when cancelled. Providers that create child tasks, network streams, or subprocesses remain responsible for propagating cancellation and cleaning up those resources when their future is dropped.
Provider events are forwarded in arrival order into the run event stream. Usage events are accumulated across steps. Raw reasoning is emitted as ephemeral ReasoningDelta events but is not copied into completed assistant history or snapshots. Provider-produced reasoning summaries may be persisted.
Provider-native replay and handoff
ProviderContextBlock can contain opaque provider-native data. Each block is tagged with the exact ModelIdentity that produced it. Canonical ModelRequest history still contains these tagged blocks so an adapter can lower enriched messages. The adapter must use the handoff helpers or equivalent ProviderContextBlock::is_replayable_to filtering before constructing an upstream wire request:
ReplayScope::ExactModel(the default) permits replay only to the exact provider/API/model identityReplayScope::ProviderApialso replays across model names of the same provider and API; the Codex adapter sets it onopenai_response_output_itemcompaction items and on reasoning items with a nonempty stringencrypted_contentReplayScope::Nevermarks a provider note, never replayed- a changed provider or API always requires omission
- portable assistant content remains available
Session::replace_providerreturns aHandoffReportso the host can surface omissionshandoff::report_message_omissionscounts the blocks a target cannot replay;handoff::omitted_message_contextyields those same blocks, for hosts that track which omissions a user already acknowledged
Opaque blocks may still exist in session history and snapshots. Treat their data as sensitive provider content, do not render it by default, and apply retention and encryption policy. Identity checks protect compatibility, not confidentiality.
The Codex cross-model scope does not extend to unencrypted reasoning, OpenAI API-key requests, or Responses-compatible gateways. Sharing a Responses serializer does not establish backend replay compatibility.
A live Codex tool round trip verified that gpt-5.6-sol accepts an unchanged encrypted reasoning item produced by gpt-6-astra. Rho preserves those items when switching models instead of dropping them or requiring compaction. This verifies that path, not every model pair or replay longevity. A rejected native item fails the turn; the adapter does not automatically retry without it. Acceptance remains upstream-dependent.
GPT-6 models also record their reasoning effort as a provider note (a block with ReplayScope::Never). Notes are SDK metadata: the recording provider reads them back for the exact model, they never reach the wire, and switching models never reports them as omissions.
SDK metadata is never native replay data. Unencrypted reasoning and unknown formats retain exact-model filtering.
Stable versus upstream-dependent behavior
The 1.0 SDK contract can stabilize provider-neutral types, trait signatures, event ordering, cancellation propagation, error categories, identity matching, and handoff rules. It cannot guarantee behavior controlled by an upstream service, including:
- model availability, aliases, context limits, prices, or supported reasoning levels
- token accounting precision or when usage arrives
- streaming chunk boundaries and latency
- provider-native context format or replay longevity
- rate limits, quotas, authentication flows, and error wording
- tool-call quality, schema adherence, or image support
- server-side safety filtering, retention, or data residency
Provider-private wire formats are not part of the SDK compatibility promise. Adapters must normalize upstream changes without leaking private response types into the public API. Applications that depend on a specific service behavior should pin and test the adapter and surface service degradation independently of SDK SemVer.
Test expectations
Use ScriptedProvider for downstream tests that must not require network credentials. A production adapter should have contract coverage for final text, streaming, tools, usage, malformed responses, cancellation, sanitized errors, and provider-native replay. Live credentialed tests may supplement those checks, but should not be required for deterministic SDK consumers.