AI voice cloning tech used in kidnapping scam

Recent developments in the tech sector have taken the world by storm. Deep fakes and the invention of AI applications such as ChatGPT have people worried about the impact they might have on the world.

A case reported from Arizona, United States, has only added to the worries. A woman named Jennifer DeStefano claimed that scammers used AI to clone her 15-year-old daughter’s voice to demand a $1 million ransom from her.

According to a report by WKYTDeStefano’s daughter, Brie, was on a skiing trip when she received a call from an unknown number and heard her daughter say “mom” in a sobbing voice. “I never doubted for one second it was her,” DeStefano told WKYT. “That’s the freaky part that really got me to my core.”

“I pick up the phone, and I hear my daughter’s voice, and it says, ‘Mom!’ and she’s sobbing. I said, ‘What happened?’ And she said, ‘Mom, I messed up,’ and she’s sobbing and crying.”

She then went on to elaborate on how a man on the other side of the phone said that he has her daughter and that she needs to follow his instructions if she wants her daughter back.

“You call the police, you call anybody, I’m going to pop her so full of drugs. I am going to have my way with her, and I’m going to drop her off in Mexico,” the man said. And all this while DeStefano could hear her daughter crying for help.

The man asked for $1 million in ransom, however, he agreed to reduce the amount to $50,000 after the woman said that she did not have that kind of money. While the woman was on the call, one of her friends and husband tried to confirm if Brie had really been kidnapped. It was eventually revealed that it was not her daughter but an AI simulation.

However, the authorities have not confirmed if AI was indeed used in her case. But DeStefano believes that the scammers did clone her daughter’s voice. She has now appealed to people to spread awareness about the issue.

“Have a family emergency word or question that only you know so you can validate you are not being scammed with AI! Stay safe!” she wrote in a Facebook post.

AI tools can be used for highly sophisticated cyberattacks. AI apps have been able to generate fake images and videos. In fact, an app called Midjourney was able to generate a fake image of former President Donald Trump being arrested, which later went viral on social media.

Recently, the users of an online forum called 4chan managed to replicate British actress Emma Watson’s voice reading Adolf Hitler’s Mein Kampf. They used a voice-cloning app called Prime Voice.

Another fake video that went viral on social media platforms was of actor Leonardo Di Caprio speaking at the United Nations. Recently, a VICE journalist managed to access his bank account using a replica of his voice.

These hyper-realistic AI productions have ignited a debate around the technology’s disruptive power. Some key figures, including Elon Musk, previously wanted the training of powerful AI systems suspended, claiming it is a threat to humanity.

Apparently, they believe AI has the potential to become too powerful. As a result, OpenAI’s CEO and co-founder, Sam Altman, decided to postpone the development of GPT-5 until further notice.

There is also the possibility that virtual teachers and AI chatbots will become important tools for students in the times to come. Some people are concerned about the AI tutors being used to monitor or even control a student’s behaviour.

AI could also influence the job market by replacing humans. A report by a leading global investment banking firm, Goldman Sachsrevealed that AI could impact two-thirds of US and European Union jobs.