Back to DevLog

Hands-free wake word with Vosk and one-breath commands

4 min read

I added wake-word listening to Foreman using Vosk keyphrase spotting. The wake phrase is name-derived: "hey" plus the assistant name from the panel, so renaming Emma retunes the wake word on next start.

The detector boundary

I isolated the detector behind a swappable interface with three methods: phrase, reset(), and accept(pcm)->bool. The swap point is _make_detector() in tools/wake.py. A dedicated acoustic model can replace Vosk later without touching the mic or pipeline.

False wakes from speakerphone audio

The first live test caught six false wakes from a speakerphone call. I checked memory/wake_tuning.log and found every capture followed a rule-qualifying wake. No stealth capture happened. The problem was that bare "hey" or a trailing "hey" in any sentence triggered the old loose matcher.

I tightened the rules. A wake now requires both the opener and an emma-word as the last two tokens. I removed single-token wakes, the loose "hey plus any m-word" catch-all, and the fragment rule. The spoken ack is always audible before capture, so a false wake is never silent. Foreman says "Yes?" or "I'm here" before listening.

One-breath commands were swallowed

The user reported a binary fork: say "Hey Emma, remind me Friday about X" in one breath and everything after "Emma" disappeared. Pause after the wake phrase and the command worked.

I found a dead-mic window of 1.5 to 2.5 seconds. The wake closed the mic stream, then play_ack() blocked on the tone, then say("Yes?") paid Kokoro's cold start and synthesis time, then a 200ms settle, then a second stream opened. Continuous speech survived only if the user was still talking two seconds later.

I rewrote the handoff around one invariant: from the moment of wake, audio is always buffered until either a command completes or the ack path is chosen. The stream that heard the wake phrase stays open and runs straight into capture_after_wake(). Three cases: continuous speech gets a buffered command and skips the spoken ack, a pause under 1.75 seconds keeps the buffer open and joins resumed speech to the same capture, and real silence closes the mic so Foreman can speak.

The ack tone now fires non-blocking at the detector hit. The spoken ack only happens on the silence path.

Reminders with Python calendar math

I added a reminders feature with due times. Haiku miscounted weekdays three out of three tries even with the correct date injected. It resolved "Friday" from Thursday July 30, 2026 to Saturday August 1.

I moved calendar math out of the model. The router extracts day and time as words. New tools/when.py resolves them deterministically in Python. Weekday goes to the next occurrence, today counts only if the resolved time has not passed, and missing day or unknown time returns a refusal question.

Reminders are delivered by an on-start brief in run_voice. Overdue reminders reappear each session start until completed. The full spoken readback is never shortened so misresolution is caught the same way gate confirmations are.

Vosk recognizer goes deaf on long streams

Live debug logs showed the recognizer going deaf after about 1600 frames of partials, then 1700 frames of silence-discards despite speech. Vosk degrades on a long unbroken stream. I added a periodic rebuild: a fresh KaldiRecognizer from the loaded model after every endpoint and every 30 seconds. The model itself is not reloaded.

Split-phrase assist

A bare "hey" is remembered and any emma-word within 1.5 seconds completes the wake. This bridges a split even across a recognizer rebuild. Every wake appends its partials to memory/wake_tuning.log for tuning.

Beloit correction

I added "beloit" to KNOWN_NOUNS and the Whisper vocab prompt. Corrections map "bell oit", "below it", and "beloyt" to "beloit". I deliberately left "lloyd" alone because Lloyd is a real name.

Share this post