Support agents
Stream answers with structured tool calls. Keep conversation context inside a 32K token window.
POST /v1/chat/completionsBUILDING THE INFERENCE LAYER
Run language models through a single inference API. Stream tokens to your app, generate embeddings, or send a million prompts in a batch.
OpenAI-compatible request format · JSON over HTTPS
Responses arrive one token at a time.▌
FROM REQUEST TO RESPONSE
Keep your application code. Route requests by model ID, set token limits, and separate production keys from experiments.
Explore the batch endpoint →HTTP / Python / JSkey scope · model ID · limitsStream answers with structured tool calls. Keep conversation context inside a 32K token window.
POST /v1/chat/completionsClassify and summarize a large corpus with JSONL inputs and a downloadable result file.
POST /v1/batchesTurn passages into vectors for a search index. Choose output dimensions for your storage budget.
POST /v1/embeddingsEARLY ACCESS, THOUGHTFULLY SIZED