Can we trust AI to use our computers?

Current AI agents are powerful but passive; they read what we say and make suggestions to us. But the next AI agents being developed are more active. Called ‘Computer Use Agents’, they don’t just give us answers, they go off and do tasks for us on our computer. But there’s a problem: they can’t distinguish between the instructions we give them, and malicious instructions that might be embedded in the data they read online. So how, and why, should we trust them?

Currently, many of us use AI without even thinking about it – for example, when we search Google maps for a route, get a music recommendation from Spotify, use a spam filter, or set up face recognition on our smartphones. And we also use AI’s like Claude or CoPilot or Chat GTP that read what we say and respond with suggestions. These AI’s are powerful but also passive. However, the next AI coming down the track is active. Computer Use Agents, as they’re called, don’t just tell you the answer, they actually go off and do it for you on your computer. This is AI that literally sees your screen and clicks, types, and scrolls. But it comes with a problem: it can’t distinguish between what you ask it to do, and malicious instructions that might be embedded in the data it’s reading.

You can ask a Computer Use Agent to find you a one-month weather forecast for your local area or set up a favourites folder on your browser bar. While such AI agents can help us get boring stuff done – like filling in forms or sending routine emails – they have a big drawback: they can get side-tracked or even be deliberately thrown off course by hackers feeding it malicious instructions. “As an analogy,” says researcher Hanna Foerster, “imagine you’ve hired a personal assistant, but every letter they open might contain a hypnotic command that they will obey (whether it’s a command from you or a malicious one).”

This is a problem that Hanna is currently researching and developing solutions for. Come and hear her talk about why she doesn’t trust AI – and offer tips for anyone interested in using or developing AI that’s safer to use.

These talks are taking place as part of the Department of Computer Science and Technology Open Day.