Every number, and where it came from

Measured 17 and 18 September 2026 on synthetic speech with an exactly known pause, so the ground truth is certain. Real disordered speech will be harder for recognition, which is why the design never relies on recognition being right.

The same model, the same input, two ways of using it. The baseline is one model asked to say what the person meant, the way most assistants work. One Moment is the product's own decision engine, exactly as a live call runs it. A sentence counts as having invented words if it contains a content word the speaker never said.

• Sample: 16 sentences from the Harvard Sentences, IEEE Std 297-1969, public domain, run 2026-09-18 with qwen3.5-4b-32k-fast, seed 11. Small. Read it as a direction, not a rate.
• What we wrote: the degradation, in eval/negbench.mjs and eval/spike/lib/degrade.js, written by us. We did not write the sentences or their meaning.
• Text level: this measures the decision layer, not recognition. Clear-speech confidences are 0.90 to 0.99; degraded ones are drawn low, as disordered speech would be.
• The honest limit: when both listening streams lose the same “not”, the text no longer contains it, and no check on the text can recover it. That row is published so you can see the size of it.
• “Not needed” counts questions asked where the baseline's plain reading was in fact correct: the cost of caution.

npm install
node --env-file=.env eval/negbench.mjs --n 16 --seed 11

Every number above this section comes from clean synthetic speech or text. This one does not. We streamed 120 read sentences from 8 speakers with dysarthria, from the TORGO database TORGO, through both live AssemblyAI ears exactly as a call would, and then through the product's own engine. Each result is scored against the sentence the speaker was asked to read.

• The cost: One Moment asked the caller instead of speaking on 63 of 120 sentences. On 52 of those, the ordinary agent's relay would have been wrong; 11 questions were not needed.
• Turn-taking: ordinary settings split 48 of 120 sentences into more than one turn, each split a point where an ordinary agent would have started talking. The patient ear split 0.
• Recognition itself is hard here: word error rate 42% for the ordinary ear and 36% for the patient ear, against the prompt. We do not improve recognition; the point of the rules is what gets said anyway.
• The self-audit, on this speech: AssemblyAI's pre-recorded model had a word error rate of 35% here: about the same as the patient ear (36%), better than the ordinary one (42%). Of One Moment's 11 wrong relays it flagged 3, and it wrongly flagged 0 of 46 correct ones. On disordered speech every model can make the same mistake; the audit catches the cases where the live ears guessed from context, as in the documented failure below, not errors all models share. It is a backstop, not an oracle.
• What this is and is not: dysarthria is a motor speech disorder, not aphasia. This tests the listening and decision layers on hard, real speech; it does not test word-finding. Speakers read prompts, so the prompt is the reference even where a speaker changed or repeated a word; some relays scored wrong here may be what the speaker actually said, which makes these error counts, if anything, too high for every arm alike. Run 2026-09-19, model qwen3.5-4b-32k-fast, 148 model calls.
• Licence, and what is deliberately not in the repository: TORGO is free for academic, non-profit use. We use it for non-commercial evaluation, publish only aggregate results and our own transcripts, and redistribute no audio. The audio and the raw per-sentence run output stay on the machine that ran it, by .gitignore and .vercelignore, so reproducing this means fetching TORGO yourself with the first command below. What ships is the aggregate above, every row in it, and the scripts that made both.

./eval/spike/scripts/fetch-torgo.sh      # TORGO, from the University of Toronto
node --env-file=.env eval/audiobench-stream.mjs --per-speaker 15
node --env-file=.env eval/audiobench-decide.mjs

Everything above this section was measured at 16kHz, which is what a browser microphone gives. A phone line gives 8kHz mu-law: band-limited to about 3.4kHz and logarithmically companded, which discards exactly the high frequency detail that separates one consonant from another. Since the product is for phone calls, measuring it only on laptop audio would be measuring the easy case. So the same 120 TORGO sentences TORGO were streamed again through the same two ears, with the audio put through a real telephone encoder first and nothing else changed.

• Recognition gets worse, and the turn-taking result does not move. Both ears lose about four points of word error rate to the phone line, which is the honest cost of narrowband audio on speech that is already hard. But ordinary settings still cut 46 of 120 sentences off mid-way, and the patient ear still cuts 0. The central claim of this project is about when a turn ends, not about how well anything is heard, and that is why it survives the transcoding intact.
• The caution got better aimed, not worse. This is the result we did not expect. On worse audio One Moment spoke about as often and was wrong about as often (12 of 60 against 11 of 57), but of the questions it asked, the ones that were not needed fell from 11 to 5, and the ones that were needed rose from 52 to 55. Degrading the audio degrades confidence and makes the two ears disagree more, and those are exactly the signals the rules key on. The caution is driven by real uncertainty rather than by a guess about it, so when the line gets worse it lands on the right sentences.
• Why this run exists. Until it, the honest line on the limits page was that telephone audio was unmeasured, so nothing about phone calls was claimed. A product whose entire premise is a phone call cannot leave that unmeasured and still expect to be believed.
• What is not repeated here. The post-call self-audit is not re-run on this arm, so the audit numbers above are wideband only. Run 2026-09-27, model qwen3.5-4b-32k-fast.

node --env-file=.env eval/audiobench-stream.mjs --narrowband --out eval/results/audiobench-evidence-8k.json
node --env-file=.env eval/audiobench-decide.mjs --arm 8k

Everything above is about disordered speech, where waiting longer is obviously the right thing to do. The fair objection is the opposite one: a relay that holds a turn for six seconds and asks whenever it is unsure might be unusable for a speaker with no difficulty at all. If that were true, the design would be wrong rather than narrow. So we ran 60 sentences of ordinary fluent speech from 35 readers, taken from LibriSpeech LibriSpeech, through the same two ears, the same end-of-turn rules and the same engine.

• Patience is not the same as lag. The six-second window is a ceiling, not a wait. When both ears agree a sentence has finished, ForceEndpoint ends the turn about a second and a half later, which is why the median here is 1.7 seconds and 34 of 60 turns ended in under two. A fluent speaker pays for the patience only when they actually pause.
• The 7 flagged relays are worth reading, because none of them is an invention. Every word the scorer objected to was in what the ears actually heard, so the engine added nothing. Most are the corpus and the recogniser disagreeing about spelling rather than mistakes a listener would notice: LibriSpeech writes “MARCH TWENTY SECOND EIGHTEEN THIRTY SEVEN” where AssemblyAI heard “March 22nd, 1837”, and “ENQUIRIES” where it heard “inquiries”. The genuine ones are misheard names, such as “Emil” heard as “Amiel”, which the verbatim path passes on faithfully. We do not improve recognition and have never claimed to; this is what that costs on clean speech.
• The cost of caution, on people who do not need it. 11 of 60 became a question rather than a relay. On clean speech that is the system being careful about nothing, and it is the honest price of a design that would rather ask than guess.
• Even here, ordinary settings cut people off. The fast ear split 8 of 60 fluent, unimpaired sentences into more than one turn. The patient ear split 0.
• Corpus: LibriSpeech test-clean, read speech from public-domain audiobooks, CC BY 4.0, with the sentence each speaker read as the reference. We did not record it and did not choose which sentences exist in it. Run 2026-09-29, 42 model calls across 60 sentences, because 39 of them needed no model at all.

node --env-file=.env eval/fluentbench.mjs --n 60
node --env-file=.env eval/fluentbench-decide.mjs

One setting decides whether a sentence survives: min_turn_silence. It can be characterised by sweeping it against synthetic speech, and that tells you when a turn fires. It cannot tell you what firing costs, because a synthetic voice never stops in the middle of a word to look for one. So we swept it across 12 sentences from 7 speakers with dysarthria TORGO and scored each value by what the speaker loses.

• At the vendor default, the turn closes while the person is still talking. It did so on 4 of 12 sentences, with 10 words of 86 still to come. A voice agent on those settings is not waiting for a pause; it is answering before the sentence exists.
• And here is the result that argues against our own setting. At 2400ms nothing fired early, nothing split, and only 2 words were lost: the same as ours, for 3.7 seconds less waiting. On this corpus, 6000ms buys nothing that 2400ms has not already bought.
• Why we kept 6000 anyway, stated so you can disagree. TORGO is read speech: the pauses are short, so a 2400ms setting is enough to clear them. The pause this product exists for is a word-finding block in aphasia, which the literature puts at roughly 5 to 10 seconds, and no corpus of dysarthric read speech can show it. The 6000ms value is a ceiling for that case, not a delay we pay on every turn: when both ears agree a sentence has finished, ForceEndpoint closes it early, which is why the median wait on fluent speech measured 1.7 seconds rather than six.
• What this measures, and what it does not. One ear, no ForceEndpoint and no second stream, so each number is about the setting rather than about our orchestration on top of it. Twelve sentences is a small sample and is reported as one. Run 2026-09-29.

node --env-file=.env eval/turnbench.mjs --n 12
node eval/turnbench-summarise.mjs

The patient ear is told the caller's own words, so it hears them more easily. That is also the danger: a word it is listening for can be heard because it was expected. We tested it with four slurred ways of saying a medicine.

The boosted ear produced “amlodipine” 4 times out of 4; the unbiased ear, 0. So rule 11: a word from the caller's list that only the listening-for-it ear heard, or that only partly came out, is offered as a choice (“Amlodipine, or metformin?”) and never relayed on trust. Measured 18 September 2026 with eval/probe-vocabulary.mjs.

We publish this one on purpose. In an earlier run of the harder call, the caller said “I need to refill my ... am low dippy prescription, please.” In context, both live ears wrote “amlodipine”, so they agreed, no rule fired, and it was relayed.

When both ears are wrong the same way, no live check can know. The self-audit can, because the pre-recorded model listens more carefully and was not told what to expect. This is why every call is graded afterwards, and why the product is positioned as a first line with a human behind it.

Every figure above is derived from a run file in the repository rather than typed into the page. So the question is not whether we made a typo, it is whether those files still say what we claim. npm run verify scores every relay in both benchmarks again from scratch, against ground truth we did not write, rebuilds the totals from those fresh scores, and exits non-zero if anything differs. This table is that command's output, not a summary of it.

Run 2026-09-29, with 65 of 65 engine tests passing. One of the checks asserts that the unflattering numbers are still there, so a future run cannot quietly drop them.

git clone <this repository>
cd one-moment && npm install
npm run verify                   # re-derives every number above from the committed runs
cp .env.example .env            # add your AssemblyAI key
npm test                         # the engine: floor, evidence, Dissent, semantic patience
node --env-file=.env apps/orchestrator/scripts/record-sample-call.ts   # a full call, recorded
node --env-file=.env eval/negbench.mjs --n 16 --seed 11                # the benchmark

We have not tested this with a person who has aphasia. That is the honest limit of the whole project, and no amount of benchmarking substitutes for it. What we can do is say where each design decision came from, and then check it against research done by people who did work with them.

What this does not establish. None of these studies tested this system, or any automated conversation partner. They establish that the problem is real, that the waiting window we chose is the right order of magnitude, and that people with aphasia will use a voice assistant that earns it. They do not establish that this one helps anybody. Only testing with people who have aphasia would do that, and that is the next step rather than a claim.

The central inference, stated with its limit: the best-evidenced help for aphasia in conversation is a trained partner, and the partner on a phone call cannot be trained. No study has tested an automated partner. This is an untested application of a well-evidenced principle.

Our own accessibility audit is on the accessibility page.