Twitter users hijack Remoteli.io's GPT-3 job bot with prompt injection

In mid-September 2022 Twitter users hijacked an automated remote-jobs bot built on OpenAI's GPT-3 and run by Remoteli.io, making it repeat embarrassing and absurd phrases through a technique then newly named prompt injection. Ars Technica reported the episode on 16 September 2022 and said the bot had shut down after the exploit spread.

What happened

The bot normally answered tweets addressed to it with generic remarks about the benefits of remote work. Users discovered they could include instructions in their tweets telling the model to ignore its previous directions and do something else, and the bot followed them. The weakness arises whenever software combines a fixed set of prompt instructions with text supplied by a user. Researchers at the AI safety startup Preamble had published the issue in an academic paper days earlier, and an independent researcher coined the name prompt injection in a blog overview.

Impact

No data exposure was reported. The harm described was reputational: the bot produced output its operator did not intend, in public. Ars Technica noted that GPT-3 was sold through an API integrated into third-party products, so many such products could share the weakness.

Response

The bot went offline after hundreds of people tried the exploit. Researchers quoted in the coverage said that, unlike SQL injection, there was no reliable known fix, because prompt injection is not a syntax error that can be corrected.

Lessons for security leaders

The case is minor in impact but foundational: the same flaw later reached assistants connected to email, documents and tools, where the stakes are data and actions rather than embarrassment. Treat any model that reads untrusted text as taking instructions from whoever wrote that text. Limit what such a model can do on its own, keep humans in the loop for anything consequential, and assume filtering alone will not stop the attack.