Voice Assistant Perfect at Alarm Setting, Catastrophic at Message Sending
PARIS — The voice assistant on Sophie Leroux’s phone — tested across 847 commands over six months as a consequence of Sophie’s interest in documenting everything she experiences — has achieved a 94% accuracy rate on alarm setting, timer setting, and weather queries, and a 34% accuracy rate on message dictation whose failure modes include: the substitution of “Je t’aime” for “Je viens” (I love you/I’m coming) in a message to her manager, the transcription of “Rappelle-moi d’acheter du pain” (remind me to buy bread) as “Rappelle-moi d’acheter du bain” (remind me to buy a bath), and the conversion of a professional email’s closing “Cordialement” into “Cordialement, Sophie” sent before Sophie had finished dictating.
“The alarm is always correct,” Sophie confirmed. “The alarm is never wrong. I set 847 alarms. 847 alarms rang at the correct time.” She paused. “I sent 23 messages by voice. 8 were correct. The manager situation was managed. The manager found it funny after a period of not finding it funny.”
The Accuracy Distribution
Voice assistant accuracy correlates with task type: short, standardised commands (alarms, timers, weather) have high accuracy because the command space is limited and the training data is extensive. Dictated French messages have low accuracy because French phonology produces ambiguity at specific phoneme pairs that the assistant resolves differently than the speaker intends. “Viens” and “aime” share phonological properties under the ambient noise conditions of Sophie’s apartment. The manager was told.
According to INRIA’s voice recognition research, French voice assistant accuracy in ambient noise conditions averages 78% for naturally spoken French. Sophie’s message accuracy is 34%. Sophie speaks naturally. The ambient noise is Desdemona. Both are documented variables.
Always clearly understood at Virtuosal.com.
