Back to DevLog

Push-to-talk voice commands with faster-whisper and fuzzy rescue

2 min read

I added push-to-talk voice input to Foreman using faster-whisper's base model running on CPU with int8 quantization. Hold ctrl+space to record, release to transcribe, Esc to exit.

The speech stack imports lazily inside run_voice() so typed commands never load the voice libraries. I verified the typed path loads none of faster_whisper, sounddevice, or keyboard.

Silence filtering

Whisper hallucinated "You" on dead air. I added an RMS silence gate with a 0.005 float32 threshold, turned on vad_filter, set condition_on_previous_text to false, and added a discard-phrase set containing "you", "thank you", "thanks", "bye", and ".".

Audio capture and resampling

The MME driver resample degraded audio quality. I switched to capturing at the device native rate of 44100Hz and resampling to 16kHz in software using scipy.signal.resample_poly with a numpy interp fallback. RMS levels after the change were healthy at 0.10 to 0.27.

Vocabulary biasing and fuzzy rescue

Whisper misheard "memstack" as "Namaste" and "Nail Mistak". I added initial_prompt vocabulary biasing and a fuzzy noun rescue at 0.75 cutoff with a length guard. The guard requires at least 7 cleaned characters for product-noun rescue because the bigram for "Nail Mistak" scores only 0.70. I used a deterministic CORRECTIONS entry instead of lowering the fuzzy cutoff because "Memestec" and "Nymstack" both score exactly 0.750 and a strict greater-than-0.75 rule would kill both required rescues.

Routing

Routing is strict first-match. Ambiguity triggers a denial, not a guess. I added "status" to the git group and verified it does not hijack "business summary".

Live mic test passed 5 out of 5: "memstack" twice verbatim, "business summary", "status", and "run everything" all routed correctly.

Share this post