8-adversarial-ai.org

Adversarial AI

#NotHotDog

• https://blog.paperspace.com/ml_behind_nothotdog_app/
• https://www.digitaltribune.com/not-hotdog/
• https://twitter.com/hashtag/NotHotdog

Slides

• BSI talk
• Stanford talk

Literature

Blog posts:

• Fast gradient sign method: https://jaketae.github.io/study/fgsm/
• Towards a general theory of “adversarial examples,” the bizarre, hallucinatory motes in machine learning’s all-seeing eye https://boingboing.net/2019/03/08/hot-dog-or-not.html
• Adversarial attacks: How to trick computer vision https://hackernoon.com/adversarial-attacks-how-to-trick-computer-vision-7484c4e85dc0 ### Scientific articles:
• Explaining and harnessing adversarial examples https://arxiv.org/pdf/1412.6572.pdf
• Intriguing properties of neural networks https://arxiv.org/pdf/1312.6199.pdf
• Prompt Injection introduction: https://research.nccgroup.com/2022/12/05/exploring-prompt-injection-attacks/

Exercise: Generating adversarial examples & LLM Prompt Injection

• Use cleverhans: https://github.com/cleverhans-lab/cleverhans. (Python/ML) Try out the MNIST tutorial with torch! Afterwards, experiment with different image classification datasets, different models, or different attacks (other than fast gradient sign method). How does the epsilon value in FGSM affect the adversarial process? Setup tip: Do PIP install for torch, torchvision and cleverhans if buggy.
• Try out LLM prompt injection attacks with: https://gandalf.lakera.ai/ (Web) (less techical than above, how many levels can you solve? What kind of prompts worked?)

Target nets:

• Cubs: https://github.com/nicolalandro/ntsnet-cub200
• Shoes: https://github.com/rfox12-edu/image_classifier_pytorch_shoes
• Flowers: https://github.com/jclh/image-classifier-PyTorch

Dataset ressources:

• https://www.kaggle.com/datasets?tags=13302-Classification
• https://paperswithcode.com/datasets?task=image-classification

GANs resources

https://machinelearningmastery.com/resources-for-getting-started-with-generative-adversarial-networks/

More interesting links

• https://analyticsindiamag.com/an-idiots-guide-to-adversarial-attacks-in-machine-learning/
• https://www.belfercenter.org/publication/AttackingAI
• https://openai.com/blog/adversarial-example-research/
• https://machinelearningmastery.com/data-leakage-machine-learning/
• https://medium.com/the-generator/jailbreaking-large-language-models-if-you-torture-the-model-long-enough-it-will-confess-55e910ee2c3c

Attacking LLMs tutorial

https://gandalf.lakera.ai/