What’s new here
Vui Nano writes each turn into its KV cache as text plus codec codes, so your words and the audio of your voice condition the reply. It was trained on two-speaker dialogue with a speaker-change token, so it can produce breaths, laughter, hesitations and overlap. The same model also has a C build that runs on a CPU.
What it does
Vui is a voice assistant server built around its own TTS model, Vui Nano (219M active parameters, 305M total). You speak into a browser mic over WebRTC. The server runs VAD and ASR (faster-whisper or Moonshine), sends the text to a local LLM (Ollama, vLLM or any OpenAI-compatible endpoint), and streams the spoken reply back. Barge-in cancels the reply when you start talking.
It also exposes an OpenAI Realtime-compatible WebSocket at /v1/realtime and a one-shot POST /v1/voice-note endpoint. A parallel “thoughts” LLM routes voice intent to about 15 tools: memories, timers, task control and web search. An optional Claude task server handles slower agentic work.
You can also use just the model. demo.py is a Gradio playground, pip install vui-tts gives you the Python engine, and cpu/ holds the single-binary C engine. Voice cloning works from a sample, though the README says clones sound worse than the four fine-tuned presets (maeve, abraham, rhian, harry).
Who it’s for
People who want a local voice loop they can read and change, and developers who want a small TTS model for a Pipecat or LiveKit agent, a batch job or a notebook. The streaming server targets Linux with an NVIDIA GPU. On Apple Silicon the TTS works, but the README calls the rest of the MLX stack work in progress. Check the model card on Hugging Face for the weight terms before shipping it.
Try it
The README’s one-liner clones into ~/vui, picks Docker or native, installs dependencies and launches the stack on http://localhost:8080. TTS weights download from Hugging Face on first render.
curl -fsSL https://install.fluxions.ai | bashFor the Docker route, you need Docker with the Compose plugin, the NVIDIA Container Toolkit and Ollama on the host:
ollama pull qwen3.5:4b # on the host
docker compose up -dFor just the model in your own Python (Python 3.12):
pip install vui-ttsfrom vui.engine import Engine, GenConfig
engine = Engine() # vui-nano-1.1 downloads from Hugging Face on first use
with engine.new_row() as row:
codes, audio = row.render("Hello from Vui.", GenConfig(temperature=0.7))If something looks wrong, run python -m vui.doctor. The CPU build is documented in cpu/README.md.
How mature is it
782 stars, 85 forks and 4 contributors. The repo was created on 2025-06-05 and last pushed on 2026-10-10. There are 11 releases, the latest being v1.2.3 on 2026-10-10, and 61 commits in the last 90 days. It has 0 open issues and 0 open PRs. The code is Apache 2.0. The README says the model weights carry their own terms on the model card, though its model section also calls the weights Apache 2.0, so check before commercial use. The README says the Apple Silicon streaming path is still in progress.