Ouch...
Welp, so much for AI ‘security’... h/t to Borepatch for this one!
Poisoning AI models might be way easier than previously thought if an Anthropic study is anything to go on.
Researchers at the US AI firm, working with the UK AI Security Institute, Alan Turing Institute, and other academic institutions, said today that it takes only 250 specially crafted documents to force a generative AI model to spit out gibberish when presented with a certain trigger phrase.
For those unfamiliar with AI poisoning, it’s an attack that relies on introducing malicious information into AI training datasets that convinces them to return, say, faulty code snippets or exfiltrate sensitive data.
Full article, HERE from The Register.
This one is truly going to put the fox among the chickens if this proves out...
Since LLMs are nothing more than algorithms, we know that they can be ‘damaged’ or fail to function by the introduction bad data either accidentally or on purpose.
This is yet one more avenue for hackers and or pissed off designers/programmers to take out your favorite AI, or make it spit out garbage depending on what it is asked to do.
And I don’t see how this one is going to get managed or prevented. There are too many security holes in LLM training, with things constantly being added to ‘train’ the LLMs.
Buyer beware? I dunno... what say you?
Didn't someone once mention something about "garbage in, garbage out"?.....
I've gotten grief for refusing to fully trust AI, but it's Friday. I might as well open myself up for more grief. 🤷♂️😁
To me, relying on AI is like relying on a neighbor's screwdriver that they may have ground down for a specific purpose, rendering it useless for yours. Or they may have moved, leaving you without a screwdriver when you need it. Or they might even start asking you for a cash deposit every time you show up asking to borrow a tool that no longer works that well.