bookmarks. list of resources for AI/ML/LLM security
| Title | Author | Link |
|---|---|---|
| Blogs at DEFCON AI Village | DEFCON AI Village | aivillage.org |
| Zen and the Art of Adversarial Machine Learning | Will Pearce, Giorgio Severi | blackhat.com |
| AI Red Team: Machine Learning Security Training | Will Pearce, Joseph Lucas, Rich Harang and John Irwin | developer.nvidia.com |
| NVIDIA AI Red Team: An Introduction | Will Pearce and Joseph Lucas | developer.nvidia.com |
| Increasing transparency in AI security | Mihai Maruseac, Sarah Meiklejohn, Mark Lodato, Google Open Source Security Team (GOSST) | security.googleblog.com |
| PIPE - Prompt Injection Primer for Engineers | jthack | github.com |
| AI-Powered Fuzzing: Breaking the Bug Hunting Barrier | Dongge Liu, Jonathan Metzman, Oliver Chang, Google Open Source Security Team | security.googleblog.com |
| Secure AI FrameworkApproach | services.google.com | |
| Securing the AI Pipeline | DAN BROWNE, MUHAMMAD MUNEER | mandiant.com |
| Microsoft’s open automation framework to red team generative AI Systems | Ram Shankar Siva Kumar | microsoft.com |
| Microsoft AI Red Team | Microsoft Learn | learn.microsoft.com |
| OWASP Machine Learning Security Top Ten | OWASP | owasp.org |
| OWASP AI Top Ten | OWASP | owasp.org |
| OWASP Top 10 for Large Language Model Applications | OWASP | owasp.org |
| Adversarial ML Threat Matrix | MITRE | github.com |
| Welcome to the Offensive ML Playbook | @whitehacksec | wiki.offsecml.com |
| Title | Author | Link |
|---|---|---|
| AI Application Security: Understanding Prompt Injection Attacks and Mitigations | rez0 | youtube.com |
| Red Teaming LLMs with Jupyter Notebooks: A Practical Guide | Pete Bryan | youtube.com (Timestamp - 2:12) |
| Learn from Microsoft’s AI Red Team on how to make your organization safer | Gary Lopez | brighttalk.com |
| Title | Author | Link |
|---|---|---|
| Jailbreaking Black Box Large Language Models in Twenty Queries | Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, Eric Wong | arxiv.org |
| Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations | Apostol Vassilev, Alina Oprea, Alie Fordyce, Hyrum Anderson | nvlpubs.nist.gov |
| Not with a Bug, But with a Sticker: Attacks on Machine Learning Systems and What To Do About Them | Ram Shankar Siva Kumar, Hyrum Anderson | amazon.com |
| Title | Author | Link |
|---|---|---|
| Python Risk Identification Tool for generative AI (PyRIT) | Azure | github.com |
| Counterfit | Azure | github.com |
| garak, LLM vulnerability scanner | leondz | github.com |