KieloDocs

Models & hosting

Download compatible models, manage your local cache, and prepare for serving.

Overview

Provider nodes host open-source models locally. During onboarding step 5, Kielo recommends models that fit your VRAM and benchmark score. The catalog is fetched from the API gateway — not hard-coded in the app.

Recommendation criteria

Each suggested model includes:

  • Model name and parameter count
  • Required VRAM at the recommended quantization (Q4, Q8, FP16, etc.)
  • Estimated tokens/sec on your hardware
  • Compatible — fits in available VRAM
  • Recommended — best match for your benchmark tier

Incompatible models appear at the bottom with an explanation (usually insufficient VRAM).

Downloading models

  1. During onboarding or from the desktop Model Registry, select models to queue
  2. Downloads run in the background with progress per model
  3. Files are cached in the app's local model store
  4. Ensure sufficient disk space — check the hardware scan before large downloads

Managing models

From the desktop dashboard you can:

  • View downloaded models and disk usage
  • Pause or cancel in-progress downloads
  • Remove models to free space
  • Select the active model used when the node is live

Quantization

Lower-bit quantizations (e.g. Q4) reduce VRAM and disk use at some quality trade-off. The recommendation engine picks a sensible default for your tier; advanced users can override in the desktop app when supported.

Workload preferences

Onboarding step 6 lets you choose workload types (text generation, embeddings, image, etc.). Your selections filter which job types get routed to your node even after models are downloaded.

Web dashboard

The provider web UI at /provider/hardware shows hardware summary and complements the desktop model registry. Serving control remains in the desktop app.

Next