AI Gateway

Set one line in your spec and your agent gets a managed API key for calling supported models — no provider account, no API keys to store, no key rotation to manage. The gateway is OpenAI-API-compatible, so it works with the SDKs and frameworks you already use.

Quick start

Scaffold a new agent wired to the gateway in one step:

$
ast create my-agent --model gateway

This generates astro_ai_gateway: true in the spec and agent code that reads the injected env vars — no provider key to configure. To enable the gateway on an existing agent, add the flag to astropods.yml by hand:

1
agent:
2
  image: my-agent:latest
3
  astro_ai_gateway: true

On ast deploy (and ast dev), your agent container receives two environment variables:

Calling the gateway

The OpenAI-compatible API lives under the /v1 path, so set your client’s base URL to ${ASTRO_GATEWAY_URL}/v1. Everything else is a standard OpenAI-style request — pass one of the supported models as model and authenticate with ASTRO_GATEWAY_API_KEY as a bearer token.

1
from openai import OpenAI
2
import os
3
4
client = OpenAI(
5
    api_key=os.environ["ASTRO_GATEWAY_API_KEY"],
6
    base_url=f"{os.environ['ASTRO_GATEWAY_URL']}/v1",
7
)
8
9
response = client.chat.completions.create(
10
    model="claude-sonnet-4-6",
11
    messages=[{"role": "user", "content": "Hello"}],
12
)
13
print(response.choices[0].message.content)

For Mastra: the AI-SDK provider returned by createOpenAI is what Agent.model: accepts, so the rest of your Agent / Workflow / Tool definitions stay the same.

Supported models

Pass any of these strings as model on the SDK call:

All chat models are called the same way — swap the model string. titan-embed-text-v2 is an embeddings model (use the embeddings endpoint, below).

Streaming

Set stream: true to receive tokens incrementally as they’re generated.

1
stream = client.chat.completions.create(
2
    model="claude-sonnet-4-6",
3
    messages=[{"role": "user", "content": "Write a haiku about the sea."}],
4
    stream=True,
5
)
6
for chunk in stream:
7
    delta = chunk.choices[0].delta.content
8
    if delta:
9
        print(delta, end="", flush=True)

Embeddings

Use titan-embed-text-v2 on the embeddings endpoint for RAG and vector search. It returns a 1024-dimension vector per input.

1
resp = client.embeddings.create(
2
    model="titan-embed-text-v2",
3
    input="Astro deploys agents as containers.",
4
)
5
vector = resp.data[0].embedding   # list[float], length 1024

Images (vision)

pixtral-large accepts image input alongside text. Pass the image as a base64 data URI in an image_url content block — the gateway forwards the image bytes to the model, so remote URLs are not fetched server-side.

1
import base64
2
3
with open("chart.png", "rb") as f:
4
    data_uri = "data:image/png;base64," + base64.b64encode(f.read()).decode()
5
6
response = client.chat.completions.create(
7
    model="pixtral-large",
8
    messages=[{
9
        "role": "user",
10
        "content": [
11
            {"type": "text", "text": "What does this chart show?"},
12
            {"type": "image_url", "image_url": {"url": data_uri}},
13
        ],
14
    }],
15
)
16
print(response.choices[0].message.content)

Structured output

1
import json
2
3
response = client.chat.completions.create(
4
    model="claude-sonnet-4-6",
5
    messages=[
6
        {"role": "system", "content": "Reply with only a JSON object: {\"sentiment\": \"positive|negative|neutral\"}."},
7
        {"role": "user", "content": "I love this product."},
8
    ],
9
)
10
raw = response.choices[0].message.content
11
try:
12
    result = json.loads(raw)
13
except json.JSONDecodeError:
14
    result = {"sentiment": "unknown"}

Local development

ast dev handles the gateway automatically. Run:

$
ast login
$
ast dev project start

Your local agent container receives the same ASTRO_GATEWAY_* env vars it would have in production. Code written against those env vars works identically in dev and prod.

Mixing with your own keys

astro_ai_gateway: true is independent of any models you declare. You can use the gateway for some calls and your own provider key for others:

1
agent:
2
  image: my-agent:latest
3
  astro_ai_gateway: true        # gateway-managed models
4
models:
5
  custom-fine-tune:
6
    provider: openai      # your own OpenAI key for a specific use case

Your agent code reads ASTRO_GATEWAY_API_KEY for gateway calls and OPENAI_API_KEY for the BYOK provider.

Errors and limits

The gateway returns standard OpenAI-style error responses. Common cases:

What it doesn’t cover

• Provider-native SDKs and prefixes. The gateway serves models from several providers (Anthropic Claude, Amazon Nova/Titan, Mistral), but you call them all through the OpenAI-compatible API by their gateway id (for example claude-sonnet-4-6), not with a provider prefix such as anthropic/.
• Image generation. No image-generation models are offered. Image input (vision) is supported by pixtral-large.
• Bring-your-own-model. Adding new models or fine-tunes to the gateway isn’t self-serve; reach out to support.