Model Providers¶

Model providers connect your agents to large language models (LLMs). RunAgents supports OpenAI, Anthropic, AWS Bedrock, and self-hosted Ollama out of the box. Register a provider once, and any agent on the platform can use it through a single, unified API.

Navigate to Models in the sidebar, then click + New Model Provider.

Model Provider Configuration¶

Authentication¶

Choose the authentication method for the provider.

API Key¶

For OpenAI and Anthropic providers. Provide the API key secret; the platform injects it into every request.

AWS Signature¶

For AWS Bedrock providers. The platform signs requests with SigV4 automatically.

OAuth2¶

For custom providers that require OAuth2 authentication. Configure authorization URL, token URL, scopes, and client credentials.

None¶

For Ollama and other self-hosted models that run without authentication.

How the LLM Gateway Works¶

From your agent's perspective, using language models is simple:

1. Single endpoint -- Your agent calls /v1/chat/completions (an OpenAI-compatible API). This is the only URL your agent needs.

2. Model-based routing -- Include the model name in your request (e.g., gpt-4o, claude-sonnet-4-20250514, us.anthropic.claude-opus-4-6-v1). The gateway looks up which registered model provider supports that model and routes the request there.

3. Automatic format translation -- If the upstream provider uses a different API format (e.g., Anthropic's Messages API or AWS Bedrock's InvokeModel), the gateway translates the request and response transparently. Your code always uses the OpenAI chat completions format.

4. Credentials managed by the platform -- API keys, AWS credentials, and OAuth tokens are injected by the gateway. Your agent code never handles or stores provider credentials.

# Your agent code -- same format regardless of provider
import urllib.request, json

body = {
    "model": "gpt-4o-mini",  # or "claude-sonnet-4-20250514", "us.anthropic.claude-opus-4-6-v1", etc.
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is RunAgents?"},
    ],
}

req = urllib.request.Request(
    f"{LLM_GATEWAY_URL}/v1/chat/completions",
    data=json.dumps(body).encode(),
    headers={"Content-Type": "application/json"},
)
with urllib.request.urlopen(req) as resp:
    result = json.loads(resp.read())
    print(result["choices"][0]["message"]["content"])

Supported Providers¶

Rate Limiting¶

You can optionally configure a requests-per-minute cap on a model provider to prevent runaway costs or quota exhaustion.

Model Provider Status¶

Built-in Playground LLM¶

The playground-llm is a built-in model provider included in the RunAgents starter kit. It is created when you click Deploy Hello World Agent on the Dashboard or Add starter Playground LLM on the Models page.

• Provider: OpenAI
• Models: gpt-4o-mini
• Authentication: API Key (uses the platform's configured key)

The playground LLM allows you to test the full deploy and wiring flow without configuring your own provider credentials.

What's Next¶