**[Jianqi Chen](https://windvchen.github.io/), [Hao Chen](https://scholar.google.com.hk/citations?hl=en&user=BEDNoZIAAAAJ&view_op=list_works&sortby=pubdate), [Keyan Chen](https://scholar.google.com.hk/citations?hl=en&user=5RF4ia8AAAAJ), [Yilan Zhang](https://scholar.google.com.hk/citations?user=wZ4M4ecAAAAJ&hl=en&oi=ao), [Zhengxia Zou](https://scholar.google.com.hk/citations?hl=en&user=DzwoyZsAAAAJ), and [Zhenwei Shi](https://scholar.google.com.hk/citations?hl=en&user=kNhFWQIAAAAJ)**
[](#License)
[](https://arxiv.org/abs/2305.08192)
### Share us a :star: if this repo does help
This repository is the official implementation of ***DiffAttack***. The newest version of the paper can be accessed in [IEEE](https://ieeexplore.ieee.org/abstract/document/10716799), the previous version of the paper can be accessed in [arXiv](https://arxiv.org/abs/2305.08192). (***Accepted by TPAMI 2024***)
If you encounter any question, please feel free to contact us. You can create an issue or just send email to me windvchen@gmail.com. Also welcome for any idea exchange and discussion.
## Updates
[**10/20/2024**] This paper is finally accepted by **TPAMI**. 👋 You can find the newest version of paper [here](https://ieeexplore.ieee.org/abstract/document/10716799) (with additional new results and experiments). For the previous version, please refer to [here](https://arxiv.org/abs/2305.08192). Please note that the attack methods compared in both versions differ slightly; for instance, the TPAMI version includes more recent methods, while some older ones were omitted. We recommend reviewing both versions to gain a comprehensive understanding of the comparisons with existing approaches.
[**10/14/2024**] Thanks for the contributions from @[AndPuQing](https://github.com/WindVChen/DiffAttack/pull/15) and @[yuangan](https://github.com/WindVChen/DiffAttack/pull/28), *DiffAttack* currently **supports the newest version** of `diffusers` (0.30.3). Please note that due to differences in package versions, the final evaluated results may vary slightly. To reproduce the results from our paper, we recommend installing `diffusers==0.9.0` and using the backed-up script [diff_latent_attack-0.9.0.py](diff_latent_attack-0.9.0.py).
[**11/30/2023**] Access **the latest version**, **v2**, of our paper on [Arxiv](https://arxiv.org/abs/2305.08192v2). 👋👋 In this updated release, we have enriched the content with additional discussions and experiments. Noteworthy additions include comprehensive experiments on diverse datasets (refer to Appendix I), exploration of various model structures (refer to Appendix H), and insightful comparisons with ensemble attacks (refer to Appendix G & K) as well as GAN-based methods (refer to Appendix J). Furthermore, we provide expanded details on the current limitations and propose potential directions for future research on diffusion-based methods (refer to Section 5).
[**09/07/2023**] Besides **ImageNet-Compatible**, the code now also supports generating adversarial attacks on **CUB_200_2011** and **Standford Cars** datasets. 🚀🚀 Please refer to [Requirements](#Requirements) for more details.
[**05/16/2023**] Code is public.
[**05/14/2023**] Paper is publicly accessible on [ArXiv](https://arxiv.org/abs/2305.08192) now.
[**04/30/2023**] Code cleanup done. Waiting to be made public.
## Table of Contents
- [Abstract](#Abstract)
- [Requirements](#Requirements)
- [Crafting Adversarial Examples](#Crafting-Adversarial-Examples)
- [Evaluation](#Evaluation)
- [Robustness on other normally trained models](#robustness-on-other-normally-trained-models)
- [Robustness on defensive approaches](#Robustness-on-defensive-approaches)
- [Results](#Results)
- [Citation & Acknowledgments](#Citation-&-Acknowledgments)
- [License](#License)
## Abstract
Many existing adversarial attacks generate $L_p$-norm perturbations on image RGB space. Despite some achievements in transferability and attack success rate, the crafted adversarial examples are easily perceived by human eyes. Towards visual imperceptibility, some recent works explore unrestricted attacks without $L_p$-norm constraints, yet lacking transferability of attacking black-box models. In this work, we propose **a novel imperceptible and transferable attack by leveraging both the generative and discriminative power of diffusion models**. Specifically, instead of direct manipulation in pixel space, we craft perturbations in latent space of diffusion models. Combined with well-designed content-preserving structures, we can generate human-insensitive perturbations embedded with semantic clues. For better transferability, we further "deceive" the diffusion model which can be viewed as an additional recognition surrogate, by distracting its attention away from the target regions. To our knowledge, our proposed method, ***DiffAttack***, is **the first that introduces diffusion models into adversarial attack field**. Extensive experiments on various model structures (including CNNs, Transformers, MLPs) and defense methods have demonstrated our superiority over other attack methods.
## Requirements
1. Hardware Requirements
- GPU: 1x high-end NVIDIA GPU with at least 16GB memory
2. Software Requirements
- Python: 3.8
- CUDA: 11.3
- cuDNN: 8.4.1
To install other requirements:
```
pip install -r requirements.txt
```
3. Datasets
- There have been demo-datasets in [demo](demo), you can directly run the optimization code below to see the results.
- If you want to test the full `ImageNet-Compatible` dataset, please download the dataset [ImageNet-Compatible](https://drive.google.com/file/d/1sAD1aVLUsgao1X-mu6PwcBL8s68dm5U9/view?usp=sharing) and then change the settings of `--images_root` and `--label_path` in [main.py](main.py)
4. Pre-trained Models
- We adopt `Stable Diffusion 2.0` as our diffusion model, you can load the pretrained weight by setting `--pretrained_diffusion_path="stabilityai/stable-diffusion-2-base"` in [main.py](main.py).
- For the pretrained weights of the adversarially trained models (Adv-Inc-v3, Inc-v3