Meta’s test case of empowering artificial intelligence systems to do the work of human staff isn’t going so great.
Over the weekend, reports emerged that hackers had been able to gain access to many Instagram users’ accounts by simply asking Meta’s AI assistance bot to re-assign the email address for each account. That’s all it took for Meta’s AI system to hand over control to multiple accounts, including some with large followings. Meanwhile, Meta also reportedly, didn’t have staff available to review the issue as it arose.
That’s because Meta has been on a major staff cull over the past six months, as it works to rationalize costs due to its massive investment in AI infrastructure. Meta’s long-term view is that AI tools will eventually be able to take over from human staff, and undertake most of its internal engineering work. But it seems like Meta’s initial experiments here aren’t providing much assurance.
And while Meta said the issue had been addressed, many hackers are still using this exploit to steal IG account access, according to Android Authority.
Meanwhile, TechCrunch reported on June 3 that Instagram is now informing users impacted about the breach, and that those users will need to update their passwords as a result.
Meta hasn’t said how many accounts were impacted, but the ongoing problem, amid ongoing staff reductions, has raised concerns about Meta’s potential overreliance on AI tools for its own operations. It’s also brought up questions about how much trust Meta has placed in its AI models to undertake human tasks, given the security risks.
A key challenge here is that by empowering AI agents to undertake actions, and change internal systems, there’s actually no way of knowing what they might do, and how they could be persuaded to take action.
AI tools have been trained on human conversation, and various examples have shown that they can easily be tricked into lying, making up sources or breaking their own rules through user prompts.
And because engineers are dealing with a potentially infinite scope of commands, there’s really no way to lock this down entirely. These tools can be asked to undertake commands in a broad range of ways, using all types of different languages, so it’s impossible for Meta, or indeed any AI developer, to guarantee that such systems won’t be manipulated. That’s because developers can’t simply block a single command type, or remove a function button, to limit misuse.
If AI agents are empowered to take action on a user’s behalf, they will be susceptible to this type of exploit, and restricting such will be a never-ending process. It’s essentially a whack-a-mole type approach, because engineers can only restrict a line of questioning once they become aware of it. There are now so many ways that people can ask a question that the task becomes unmanageable, in a practical sense.
This happened with X’s efforts to limit the use of its chatbot to generate nude images. X tried to limit such requests, but users still asked it to nudify images. X seemingly admitting defeat and moved to restrict access to paying users only, which has limited the scope of risk.
But companies can’t just stop this activity outright, because there’s no single command line that defines such action. Users will come up with more ways to trick bots, using alternate parameters that will enable hackers to bend the system’s programming based on flawed AI logic.
Because that’s what AI tools are designed to do: Interpret broader context, and take action based on conversational queries.
As such, the potential problem here is as unlimited as human language in communicating a task, and that’s a much bigger challenge for engineers to solve.
But what does that then mean for expanded application of Meta’s AI systems, and the capacity for Meta to make money from its AI tools? The answer remains to be seen.
