Chat completions

POST /v1/chat/completions — primary inference endpoint (stream + non-stream).

Method: POST · Path: /v1/chat/completions · Scope: inference:write

Request body

Fields validated by the gateway include model, messages[] (role, content, optional tool fields), optional temperature (0–2), max_tokens (1–32000), top_p, n, stream, stream_options.include_usage, stop, presence_penalty, frequency_penalty.

curl -s -X POST https://api.kielo.ai/v1/chat/completions \
  -H "Authorization: Bearer $KIELO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/llama-3.1-8b-instruct",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Confirm connectivity."}
    ],
    "temperature": 0.7,
    "max_tokens": 256
  }'

Streaming

Set stream: true. Response is text/event-stream with heartbeats (default interval 15s) and max duration 5m. Also sets Cache-Control: no-cache, Connection: keep-alive, X-Accel-Buffering: no.

curl -N -X POST https://api.kielo.ai/v1/chat/completions \
  -H "Authorization: Bearer $KIELO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/llama-3.1-8b-instruct",
    "messages": [{"role": "user", "content": "hello"}],
    "stream": true
  }'

Response

OpenAI-style JSON: id, object, created, model, choices[], usage. Look for Kielo response headers such as X-Kielo-Request-ID, X-Kielo-Model-Provider, X-Kielo-Routing-Policy, X-Kielo-Cache-Status.

Availability

Live when the gateway has routing + provider dependencies. Without them the handler may return 501 not_implemented.