Snap boss wary of users treating his new chatbot as a friend

Social network Snapchat has given all its users, many of whom are teenagers, free access to a ChatGPT-powered bot called My AI, in an erosion of social media’s traditional reliance on user- generated content.

Snapchat co-founder and chief executive Evan Spiegel hopes users will not form quasi-relationships with the chatbot but use it as a creative tool, despite people developing personal attachments to other artificial intelligence tools.

Google engineer Blake Lemoine was sacked from the company last year after coming to believe that a bot he was working on had feelings. And a bot called Replicawhich was designed to simulate personal connection, maintained “relationships” with users for years until its settings were changed in a 2023 software update.

Asked last week, whether users might start to refer to the AI tool by a name and integrate it into group conversations, where it will respond to user prompts like they would a friend, Spiegel said: “I think that’s possible.”

“Although I worry a little bit about that, frankly, just the user experience that you’re describing. I think one of the things that’s really valuable about [instructing the bot to answer a question via] @ mentions is that My AI only participates when you want it to.”

He said My AI worked better as a “creative tool”.

Loading

Snapchat’s bot is not designed to mimic a human relationship and was first announced in February for members of Snapchat’s subscriber program. It has had several safety features integrated since, including responses tailored to a user’s age, and a ban on derogatory discussions or those around illegal activity.

In an April media release, Snapchat said that only 0.01 per cent of My AI’s responses had failed to conform with its rules. Parents can also see information on their child’s use of My AI on the app. Figures from Snapchat claim that 90 per cent of 13-to-24 year olds in Australia use its app.

Like most social media companies, Snapchat has competing imperatives in building its AI tools. The better a chat companion it is, the more users will spend time on it, giving Snapchat an alternative to relying on influencers who every social media platform is trying to court.

Loading

“I certainly do think that it’ll be valuable for My AI to get to know somebody over time, get to know their preferences and things like that,” Spiegel said. “That’s obviously very difficult. Today, My AI’s memory is not very good. It kind of only remembers the last few things you talked about, not everything you’ve ever talked about. So, that makes it harder to build a relationship.”

Widespread use of an AI bot could also help Snap gain advertisers. Spiegel would not say how Snapchat would seek to commercialise the AI, but did say links from the bot to places such as restaurants were useful. That suggests it could be used as a platform to sell ads.

But the more capable and human-like the bot becomes, the more it risks veering into luring users into quasi-relationships.