Back to DevLog

Voice-mode barge-in and short spoken replies for Foreman

2 min read

Voice mode blocked on every word until the reply finished. I needed to cut speech short and get back to recording.

Interruptible speech

I added a poll loop to tools/speak.py. The new _wait_or_interrupt() checks for ctrl+space or Esc every 50ms while audio plays. Ctrl+space stops playback and returns "interrupted". Esc stops playback and returns "exit". Normal completion returns None.

The keyboard import is wrapped in try/except. If the library is unavailable, speech falls back to the blocking wait. Barge-in is additive. It can never be the reason TTS breaks.

Keys are checked separately, not as a combo string. That matches tools/listen.py and proved more reliable.

Short spoken replies

The router was reading markdown bullet lists verbatim. A weather denial took 15 seconds.

I added strip_for_speech() to tools/router.py. It removes asterisks, backticks, and bullet markers. Each source line becomes its own sentence. Trailing colons and semicolons are stripped. The function caps output at two sentences.

Foreman prints the full reply but speaks only the stripped version. This applies to both router denials and capability descriptions.

I also appended an instruction to the router system prompt: "Replies will be read aloud. Answer in at most two short plain sentences. No markdown, no bullet points, no offers to help further." The live weather denial arrived as two plain sentences.

Wiring

I wrapped say() calls in foreman.py so an "exit" return prints "Leaving voice mode." and unwinds the loop. Both execute() and run_command() now return the exit signal. An Esc press mid-speech exits the whole voice loop.

The "interrupted" return needs no special handling. After playback stops, the loop falls to record_and_transcribe(). The user is still holding the keys, so stopping is the whole job.

Verification

I ran headless checks before the live test. py_compile passed on all three changed files. strip_for_speech() removed markdown from the weather denial and capped at two sentences. Clean input passed through untouched. The say() fallback called sd.wait() when keyboard import failed. Esc and ctrl+space branches returned the right signals with sd.stop().

The live test passed. Barge-in cut speech and recorded immediately. The weather denial spoke as two plain sentences. Esc mid-speech exited clean.

I changed the active TTS voice to af_bella in memory/voice_settings.json. That file is gitignored.

Commit 563bca3 is on main and dev, pushed to origin.

Share this post