Would you like to restore the voice of your loved one after death?
- Posted on
Every voice is unique.
When you hear someone's voice, it can be another identification card that represents that person. Your voice has an idiosyncratic connection to what makes you you, including your biology, habits, and personal and social history. The more accurately it specifies someone's character, the harder it is to imitate or replicate someone's voice in exactly the same way.
"Be Right Back"
Yet, for various reasons, people have tried to recreate the voice of a deceased person, reminiscing about the old days spent with loved ones and delivering the last message they were unable to convey when that person was alive. Sometimes this notion became real in a movie or drama, as in an episode of the famous series TV "Black Mirror". The ever-popular British science fiction anthology series "Black Mirror" aired an episode called "Be Right Back" in 2013, and we could see many different versions of "Be Right Back" in different styles. After the main character's husband died suddenly in a car accident, she decided to recreate her husband with all the assets from his social and chat history by uploading his profile pictures and videos. She basically created a new virtual version of her late husband.
A project requested by a television crew reminds us of this episode.A scene from the drama 'Be Right Back'
They wanted us to recreate the voice of a child who had died due to an illness.
At that time, a cell phone video of the child was shown, but we have no choice but to worry because it was only a one-minute video. Even more so, it was not a perfect sentence spoken by the little girl. There was a lot of concern that there would be an ethical controversy because we knew that recreating the voice would be difficult. Nevertheless, we decided to participate in the production as we kept thinking and wondering if we could dedicate ourselves to the good cause of the broadcast team. We completed the insufficient amount of data by dubbing more than 800 sentences with the voice of a child of the same age and went through a deep learning process. Our sophisticated technology and the passion of the professional team members resulted in a voice very similar to the girl.
The MBC documentary "Meeting you", which aired in February 2020, received a lot of attention from the VR content of the deceased little girl leaving many people in tears. When she met her mother in the VR world created by the broadcast team, she was wearing her favorite purple dress and had a nice dinner with her grieving mother. The little girl's appearance was recreated by analyzing facial expressions, gestures, and voice from photographs and video data. The child asked many questions with an innocent smile, saying, "Where have you been, Mom?", "Were you thinking about me?". Even though we are the ones who restored her voice, we couldn't help but shed tears watching this docu-show.
This was possible because Neosapience's original technology is able to speak differently depending on the situation. In the past, there was speech synthesis technology, but it was difficult to control different vocalizations. We have developed a technology that can detect the surrounding situation and conversation content and control emotions to produce a voice. Not only can the user adjust the timing and rhyme of the reading or ending, but it is also possible to adjust different prosodies according to the feelings or atmospheres of the content that the user generates.
Speaking was definitely not what an AI was good at. Even if an AI was able to speak, it barely speaks because it has no emotions. Human voices have different meanings depending on nuances and subtle changes in speech. With the question of whether it is possible to reproduce these voice characteristics with artificial intelligence technology, we dedicated ourselves to research.
Image: Channel 4, a scene from the drama "Be Right Back"