Local models on demand.

drove wakes your models when a request comes in, and puts them back to sleep when they go idle. One OpenAI-compatible endpoint in front of llama.cpp (text + vision), a built-in ONNX worker (speech-to-text), and local text-to-speech engines.

Quickstart · Documentation · Configuration · CLI

Why drove?

Point an OpenAI client at http://127.0.0.1:8080/v1. drove starts the requested model, forwards the request to its backend, and stops the process after it goes idle. Count and memory budgets let you trade off keeping models ready against freeing resources.

Workload	Backend	Guide
Text and vision	llama-server from llama.cpp	Text and vision
Speech-to-text	Built-in ONNX worker	Transcribe audio
Text-to-speech	Kokoro, Qwen3-TTS, Chatterbox, Fish, KittenTTS 2	Generate speech

Quickstart

You need Git, Make, and, for this text example, llama-server on your PATH. make install bootstraps uv if needed and uses it to install Python 3.14 and drove, including speech-to-text dependencies. Downloads and first model loads can take several minutes.

git clone https://github.com/cleanunicorn/drove.git
cd drove
make install
drove init
drove models download unsloth/Qwen3-8B-GGUF:Q4_K_M --name qwen

Start the proxy in one terminal:

drove serve --host 127.0.0.1

In another terminal, send a request using the local name you chose:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen","messages":[{"role":"user","content":"Write a haiku about lazy servers."}]}'

Or run drove chat qwen for terminal chat. The first request loads the model; it unloads after 30 minutes idle by default. Stop the foreground server with Ctrl+C. The example binds to loopback; the configured default is 0.0.0.0 (all interfaces). See Running drove for network and service setup.

Run drove chat without a model name for a searchable keyboard picker showing loaded state, size and capabilities. Inside chat, /model saves the current conversation and starts a new one with another model. See Terminal chat for navigation and remote endpoints.

For installation details, SDK setup, and checks at each step, follow Getting started.

Set a preferred model with drove config default_model "qwen" after initialization. It is used by drove chat and API requests that omit model; explicit model choices still win. See default-model configuration.

What you can do

• Manage resources: load models on demand, evict least-recently-used models, and persist llama.cpp prompt caches across idle shutdowns.
• Bring your own models: download from Hugging Face, choose quantizations with keyboard controls and memory guidance, resume downloads, and set llama-server arguments per model.
• Use familiar clients: connect an OpenAI SDK, the built-in chat TUI, or pi, opencode, and Kilo Code via drove integrate.
• Inspect requests: watch server status, browse optional request logs in the terminal or web UI, and apply supported config changes live.

OpenAI client / curl / drove chat
              │
              ▼
      drove proxy :8080
              │  start the requested model if needed
              ▼
   llama-server / ONNX worker / TTS worker
              │
              ▼
     response → idle timer → shutdown

See Architecture for lifecycle and backend details.

Documentation

Start with the documentation hub, or pick a task:

I want to…	Read
Install drove and make my first request	Getting started
Download, name, inspect, or tune a model	Managing models
Send text, images, or connect a coding agent	Text and vision
Run a service or inspect traffic	Running drove
Look up commands, settings, or endpoints	CLI · Configuration · HTTP API
Diagnose a failed request	Troubleshooting

The Markdown files in this repository are the canonical docs. The hosted drove.dev/docs site is planned.

Development

From a checkout with uv installed:

make lint
make typecheck
make test

The targets sync the development environment automatically. See Contributing for the branch and PR workflow, and Release process for publishing.

Report a bug · Request a feature