How AI technology is giving ALS patients their voice back

LEICESTER, Vt. (InvestigateTV) - ALS, often called Lou Gehrig’s disease, is a cruel illness. There is no cure, it’s always fatal, and Vermont has the highest rate per capita in the country with around 27 people living with it now in the state.

To help those losing independence and their, a nonprofit is working to restore the ability to communicate with loved ones and doctors.

Paul Francoeur’s days didn’t always start like this, but when they did, it was difficult. The cough-assist machine helped the 67-year-old clear his lungs when he couldn’t do it himself. His ALS weakened his muscles and motor skills too much.

“I think soon after diagnosis you realize how this road ends,” said Kris Francoeur, Paul’s wife.

That diagnosis came in 2019.

Kris turned into a caregiver, allowing her husband to remain at home. They get help from their son Ben. But it’s getting more challenging, especially as Francoeur loses his ability to communicate.

During a tele-medicine appointment with his doctor, the couple learned that with no better medicine or treatments, he would have to stay the course.

“Hardest part...” Francoeur said. “...Is worrying about other people,” Kris said, finishing his thought.

She helped him communicate. He was also good at eye-gaze technology, where eye movements control mouse clicks to spell out words. The software then generates synthesized speech.

“It is very flat and robotic,” Kris said.

But new technology by the company ElevenLabs uses AI to clone voices, replacing the robot sound. It even creates emotion in each word. The trouble is that Francoeur needed enough clear audio from before he started losing his voice, and they couldn’t find any old recordings.

But WCAX did.

The Channel 3 News featured the couple in a “Made in Vermont” story 14 years ago. At the time, Francoeur made spinning wheels, seven years before ALS stole his voice.

The footage takes the couple back to another time. “I think more than anything, it’s just to see Paul walking and talking... using tools. It’s amazing to see. I miss all that. He misses all that,” Kris said.

For Paul, it was hard to remember that level of physical freedom.

But in a couple of months, that WCAX story would give Francoeur back the voice he thought he’d lost.

Richard Cave with ElevenLabs said they were able to use 16 seconds of sound from Paul in that 2012 story to mirror his voice.

“What we’ve done here is use AI for good, to enable people to recreate their natural voice,” Cave said.

“The voice that will come out of the computer will sound like him,” Kris said.

“Like me,” added Francoeur.

“This disease robs people of so much of their lives,” said Dr. Mark Garret, who runs the ALS clinic at Dartmouth Hitchcock Medical Center and is also Francoeur’s doctor. “When speech really becomes affected by ALS, it’s one of the things that most profoundly impacts quality of life.”

Garret calls the couple inspirational for making the most of the time Francoeur has left, and is touched by their advocacy for others living with ALS.

“I think Paul, the way he confronted a terrible prognosis and a terrible disease and made the best of life throughout all of that has been extraordinary,” Garret said. And, he said, it may be part of the reason that he has survived so long after the diagnosis. Typically, ALS patients live two to five years.

Francoeur made it to year eight.

His first words through the new technology were a thank you message and raising awareness about ALS.

“Vermont has the highest rate per capita than any state in the country,” he said, the cloned voice a perfect match.

“That does sound just like you,” Fuller said.

“Made me feel more human,” Francoeur said.

“It’s been a long time since I’ve heard Paul’s voice,” Kris said.

His next big speech was for his grandkids, who’ve never heard his real voice. “The next time I see the grandkids, I can tell them I love them with actual emotion.”

ElevenLabs offers the voice cloning technology to ALS patients for free. The organization Bridging Voices helps families connect with the company.

Seven months after reporter Darren Perron talked to the couple, Paul would pass away. He is remembered by his wife and family.