Not with my name! Inferring artists' names of input strings employed by Diffusion Models

This is the official repository of

Not with my name! Inferring artists' names of input strings employed by Diffusion Models. Roberto Leotta, Oliver Giudice, Luca Guarnera, Sebastiano Battiato. ICIAP 2023.

---

Installation

Use conda or miniconda to easily install the required dependencies.

With conda installed, run the following commands:

# 1. Clone the repository
git clone https://github.com/ictlab-unict/not-with-my-name.git
# 2. navigate to the repository folder
cd not-with-my-name
# 3. create the environment
conda env create -f docs/environment.yml

Then follow the instructions in the dataset section and checkpoints section to download the required files.

Dataset

Download at least one of the following datasets:

Dataset	# Original images	# AI generated images	Link	GB
Small	2350	2350	link	2.7
Medium	4130	8519	coming soon	/

After downloading the dataset, extract it in the resources folder. The folder structure should be as the following.

Checkpoints

Download the following checkpoint: link and place it in resources/ckpts folder. The folder structure should be as the following.

---

Inference

N.B.: to run the inference you need to download the checkpoints and the dataset (at least the small one).

Inference on a single image

To run the inference on a single image, run the following command:

# 1. activate the environment
conda activate not-w-my-name-env
# 2. run inference
python src/inference.py --dataset-folder <path-to-dataset> --query-img <path-to-query-image> --model-ckpt <path-to-checkpoint> --cuda --results-folder <path-to-results-folder>

Results will be saved in the --results-folder <path-to-results-folder>.

Inference on a single image - example

# 1. activate the environment
conda activate not-w-my-name-env
# 2. run inference
python src/inference.py --dataset-folder resources/small-dataset/ --query-img resources/small-dataset/pablo_picasso/ai_generated/102_0.png --model-ckpt resources/ckpts/siamese_not_w_my_name.ckpt --cuda --results-folder results

Inference usage

usage: inference.py [-h] [--show-time] [--debug] 
                    --dataset-folder DATASET_FOLDER --results-folder RESULTS_FOLDER 
                    --query-img QUERY_IMAGE [--distance-th DISTANCE_TH] [--cuda]
                    --model-ckpt MODEL_CKPT

Not with my name inference by Roberto Leotta

optional arguments:
  -h, --help            show this help message and exit
  --show-time           show processing time
  --debug               flag for development debugging
  --dataset-folder DATASET_FOLDER
                        dataset folder path
  --results-folder RESULTS_FOLDER
                        results folder path
  --query-img QUERY_IMAGE
                        query image path
  --distance-th DISTANCE_TH
                        distance threshold for the query image. Default: 0.5
  --cuda                use CUDA for inference
  --model-ckpt MODEL_CKPT
                        siamese model checkpoint

---

Citation

If you find this code useful for your research, please cite our paper:

@inproceedings{leotta2023not,
   title={Not with my name! Inferring artists' names of input strings employed by Diffusion Models},
   author={Leotta, Roberto and Giudice, Oliver and Guarnera, Luca and Battiato, Sebastiano},
   booktitle={International Conference on Image Analysis and Processing},
   year={2023},
   organization = {Springer}
}

---

Credits

Authors: Roberto Leotta, Oliver Giudice, Luca Guarnera, Sebastiano Battiato
Version: 1.0.1
Date: 08/22/2023