[Caice-csse] AI security - an interesting study from Princeton

[N Narayanan]

A new study by researchers at Princeton University and Sentient shows it's surprisingly easy to trigger malicious behavior from AI agents by implanting fake "memories" into the data they rely on for making decisions. Unlike other attacks on AI - such as prompt injection, model poisoning, and adversarial attacks - a maliciously implanted fake memory can persist indefinitely through future sessions and keep causing damage until discovered.

Real AI Agents with Fake Memories: Fatal Context Manipulation Attacks on Web3 Agents
https://arxiv.org/pdf/2503.16248<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Farxiv.org%2Fpdf%2F2503.16248&data=05%7C02%7Ccaice-csse%40eng.auburn.edu%7C70137533953143a5a9d308dd93b1703c%7Cccb6deedbd294b388979d72780f62d3b%7C0%7C0%7C638829112480031679%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=EpCCiZj3%2B4Ns38cWi5Qdna4GbewJH8Q0NxVZx%2BJUkZI%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.eng.auburn.edu/pipermail/caice-csse/attachments/20250515/3bad63d8/attachment.htm>