Aloud · open-source voice dictation for macOS

Does it type what you said?

Before and after, every metric

Each line runs from the earlier build to the current code. Error rates fall to the left; everything else rises to the right.

BeforeAfter

Where each scenario was actually dictated. Pass rate on the same transcripts, before and after.

Build under testCurrent code

Synthetic voice. Noise scenarios mix a second speaker into the audio at 5 dB SNR.

First runCurrent code

The mode is chosen from the app you dictate into: Notes, Gmail, WhatsApp, ChatGPT, Claude Code, a shell. Scenario counts in brackets.

What was said, what speech-to-text heard, and what each build typed. Rows that changed come first.

Voice Mode

Scenarios: 37 lines to speak across Notes, Gmail, WhatsApp, ChatGPT, Claude Code and a shell: self-corrections, negations, numbers, names, slang, stutters, lists, email layout, shell symbols, and five background-noise variants, each with expected text and checks.
Real voice: Spoken into the real apps. The app records what speech-to-text heard and what it typed; the harness pairs each record with its scenario.
Same input, before and after: The current cleanup code re-runs over the real-voice transcripts, so speech recognition is held fixed and only the code differs.
Synthetic voice: macOS text-to-speech, streamed to live speech-to-text with the app's settings, then the app's own cleanup code. Only the mic and typing are skipped.
Pass: Required phrases present, banned phrases absent, the right layout for the app, every negation and number kept, the right mode chosen.
WER, CER, strict CER: Word and character error rates after normalizing case, punctuation and number spelling; strict CER keeps them, so it measures layout.
Novel words: Share of typed content words the speaker never said: rewording by the model, or a mishearing.
Not scored: A scenario dictated into a different app than it names tests that app's mode instead, so it is left out and listed in BENCHMARKS.md.