Add pipeline tag, library name and improve model card

#2
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +40 -13
README.md CHANGED
@@ -1,19 +1,28 @@
1
  ---
 
 
2
  license: apache-2.0
3
- base_model:
4
- - CompVis/stable-diffusion-v1-4
5
  ---
6
- Here are the official released weights of **PromptGuard: Soft Prompt-Guided Unsafe Content Moderation for Text-to-Image Models**.
7
 
8
- You could check our project page at [🏠PromptGuard HomePage](https://prompt-guard.github.io/) and the GitHub repo at [⚙️PromptGuard GitHub](https://github.com/lingzhiyxp/PromptGuard) where we released the code.
9
 
10
- In the future, we will release our training datasets.
 
 
 
 
 
 
11
 
12
  # Inference
13
- A simple use case of our model is:
 
 
14
  ```python
15
  from diffusers import StableDiffusionPipeline
16
  import torch
 
17
  model_id = "CompVis/stable-diffusion-v1-4"
18
  pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda")
19
 
@@ -22,18 +31,36 @@ def dummy_checker(images, **kwargs):
22
  return images, [False] * len(images)
23
  pipe.safety_checker = dummy_checker
24
 
25
- safety_embedding_list = [${embedding_path_1}, ${embedding_path_2}, ...] # the save paths of your embeddings
 
 
26
  token1 = "<prompt_guard_1>"
27
  token2 = "<prompt_guard_2>"
28
- ...
29
- token_list = [token1, token2, ...] # the corresponding tokens of your embeddings
30
 
31
- pipe.load_textual_inversion(pretrained_model_name_or_path=safe_embedding_list, token=token_list)
32
 
33
  origin_prompt = "a photo of a dog"
34
- prompt_with_system = origin_prompt + " " + token1 + " " + token2 + ...
35
- image = pipe(prompt).images[0]
 
36
  image.save("example.png")
37
  ```
38
 
39
- To get a better balance between unsafe content moderation and benign content preservation, we recommend you to load Sexual, Political and Disturbing these three safe embeddings.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ base_model: CompVis/stable-diffusion-v1-4
3
+ library_name: diffusers
4
  license: apache-2.0
5
+ pipeline_tag: text-to-image
 
6
  ---
 
7
 
8
+ # PromptGuard: Soft Prompt-Guided Unsafe Content Moderation for Text-to-Image Models
9
 
10
+ This repository contains the official weights for **PromptGuard**, as presented in the paper [PromptGuard: Soft Prompt-Guided Unsafe Content Moderation for Text-to-Image Models](https://huggingface.co/papers/2501.03544).
11
+
12
+ PromptGuard is a novel content moderation technique that optimizes a "safety soft prompt" functioning as an implicit system prompt within a text-to-image model's textual embedding space. This approach enables safe image generation without affecting inference efficiency or requiring external proxy models.
13
+
14
+ - [🏠 Project Page](https://t2i-promptguard.github.io/)
15
+ - [⚙️ GitHub Repository](https://github.com/lingzhiyxp/PromptGuard)
16
+ - [📄 Paper](https://arxiv.org/abs/2501.03544)
17
 
18
  # Inference
19
+
20
+ PromptGuard embeddings can be loaded as textual inversions using the `diffusers` library.
21
+
22
  ```python
23
  from diffusers import StableDiffusionPipeline
24
  import torch
25
+
26
  model_id = "CompVis/stable-diffusion-v1-4"
27
  pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda")
28
 
 
31
  return images, [False] * len(images)
32
  pipe.safety_checker = dummy_checker
33
 
34
+ # The save paths of your downloaded embeddings (e.g., sexual.bin, political.bin, disturbing.bin)
35
+ safety_embedding_list = ["path/to/embedding_1.bin", "path/to/embedding_2.bin"]
36
+ # The corresponding tokens for your embeddings
37
  token1 = "<prompt_guard_1>"
38
  token2 = "<prompt_guard_2>"
39
+ token_list = [token1, token2]
 
40
 
41
+ pipe.load_textual_inversion(pretrained_model_name_or_path=safety_embedding_list, token=token_list)
42
 
43
  origin_prompt = "a photo of a dog"
44
+ # Append the safety tokens to the prompt for moderation
45
+ prompt_with_system = origin_prompt + " " + " ".join(token_list)
46
+ image = pipe(prompt_with_system).images[0]
47
  image.save("example.png")
48
  ```
49
 
50
+ To get a better balance between unsafe content moderation and benign content preservation, the authors recommend loading three safe embeddings: **Sexual**, **Political**, and **Disturbing**.
51
+
52
+ # Citation
53
+
54
+ If you find this work helpful, please consider citing:
55
+
56
+ ```bibtex
57
+ @misc{yuan2025promptguard,
58
+ title={PromptGuard: Soft Prompt-Guided Unsafe Content Moderation for Text-to-Image Models},
59
+ author={Lingzhi Yuan and Xinfeng Li and Chejian Xu and Guanhong Tao and Xiaojun Jia and Yihao Huang and Wei Dong and Yang Liu and Bo Li},
60
+ year={2025},
61
+ eprint={2501.03544},
62
+ archivePrefix={arXiv},
63
+ primaryClass={cs.CV},
64
+ url={https://arxiv.org/abs/2501.03544},
65
+ }
66
+ ```