Microsoft's VALL-E clones a voice from three seconds of audio

With just a three-second sample of any voice, the transformer-based TTS model VALL-E can produce speech in every voice. This allows for impersonating a specific speaker with only a minimal recording. Microsoft’s paper describes it as a neural codec language model trained on 60,000 hours of English speech from more than 7,000 speakers, and its broader impacts section itself lists spoofing voice identification as a risk, with a detection model as the proposed mitigation. If your bank greets you with “my voice is my password”, now would be a good time to ask about alternatives.