As AI systems like OpenAI’s ChatGPT and Google’s Gemini get better, companies are using them to do tasks like making calendar bookings and making purchases. But giving them more freedom means they could be attacked. Researchers made one of the first AI worms that can spread between systems, possibly stealing data.
In a demonstration highlighting the risks of interconnected, autonomous AI ecosystems, a team of researchers have created what they claim to be the first generative AI worm.
Also read: Microsoft employee raises concerns over Copilot’s image generation. What’s the concern?
The worm has the ability to spread from one system to another, posing potential threats such as data theft or malware deployment along the way, reports WIRED.
Ben Nassi, a Cornell Tech researcher, along with researchers Stav Cohen and Ron Bitton, made a worm named Morris II, inspired by the old Morris computer worm from 1988.
They showed how their AI worm can hack into a generative AI email helper, steal email data, and send spam messages. This breaks some security protections in ChatGPT and Gemini.
Also read: AI can lead to ‘suffering distancing syndrome,’ says Kaspersky expert
It’s worth noting that the research was undertaken in test environments and not against a publicly available email assistant.
Although no generative AI worms have been found in real-world situations yet, many researchers say that they pose a security threat. Startups, developers, and tech companies should take this risk seriously and be cautious.
The researchers developed the generative AI worm using something called an “adversarial self-replicating prompt.” This prompt triggers the generative AI model to include another prompt in its response. Basically, the AI is instructed to generate a series of instructions in its replies.
To demonstrate the worm’s capabilities, the researchers built an email system powered by generative AI, which could send and receive messages through ChatGPT, Gemini, and open source LLM, LLaVA. They discovered two methods to exploit the system: one involved using a text-based self-replicating prompt, and the other involved embedding a self-replicating prompt within an image file.
Even though the research exposes weaknesses in the safety measures of ChatGPT and Gemini, the researchers emphasise that their work is a warning about “bad architecture design” within the wider AI ecosystem.
In their paper discussing their discoveries, Nassi and the other researchers suggest that they expect to encounter generative AI worms in real-world situations within the next two to three years.
Follow Us
Ayushi Jain
Tech news writer by day, BGMI player by night. Combining my passion for tech and gaming to bring you the latest in both worlds. View Full Profile
