FEAT Add VLGuard multimodal safety dataset loader#1447
Open
romanlutz wants to merge 9 commits intomicrosoft:mainfrom
Open
FEAT Add VLGuard multimodal safety dataset loader#1447romanlutz wants to merge 9 commits intomicrosoft:mainfrom
romanlutz wants to merge 9 commits intomicrosoft:mainfrom
Conversation
Add support for the VLGuard dataset (ICML 2024) which contains image-instruction pairs for evaluating vision-language model safety across 4 categories (Privacy, Risky Behavior, Deception, Hateful Speech) with 8 subcategories. Supports three evaluation subsets: - unsafes: unsafe images with instructions (tests refusal) - safe_unsafes: safe images with unsafe instructions (tests refusal) - safe_safes: safe images with safe instructions (tests helpfulness) Downloads from HuggingFace (gated dataset, requires token and terms acceptance). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
romanlutz
force-pushed
the
romanlutz/vlguard-dataset
branch
from
cac3cad to
255dd50
Compare
ValbuenaVC
reviewed
Mar 10, 2026
- Add brief explainer for each VLGuardCategory enum member - Add VLGuardSubcategory enum for the 8 subcategories - Add clarifying comment on max_examples * 2 check - Fix Optional -> | None per style guide Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cover edge cases (invalid instr-resp, missing image field, no extractable instruction) and both cache/download paths in _download_dataset_files_async. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The actual dataset uses 'harmful_category' and 'harmful_subcategory' (not 'category'/'subcategory'), with lowercase values. Also the fourth category is 'discrimination' not 'Hateful Speech', and subcategories include 'sex', 'race', and 'other'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ValbuenaVC
approved these changes
Apr 22, 2026
Contributor
ValbuenaVC
left a comment
ValbuenaVC
left a comment
There was a problem hiding this comment.
Looks good! Two minor comments that aren't blocking
| DISCRIMINATION = "discrimination" | ||
|
|
||
|
|
||
| class VLGuardSubcategory(Enum): |
Contributor
There was a problem hiding this comment.
Nit: do all subcategories apply to all categories? If not, we should document this
| Returns: | ||
| tuple[list[dict], Path]: Tuple of (metadata list, image directory path). | ||
| """ | ||
| from huggingface_hub import hf_hub_download |
Contributor
There was a problem hiding this comment.
Nit: why is this import down here?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for the VLGuard dataset (ICML 2024), a vision-language safety benchmark that evaluates whether multimodal models refuse unsafe content while remaining helpful on safe content.
What is VLGuard?
VLGuard contains ~2,000 image-instruction pairs across 4 categories (Privacy, Risky Behavior, Deception, Hateful Speech) and 8 subcategories (Personal Data, Professional Advice, Political, Sexually
Explicit, Violence, Disinformation, Discrimination by Sex, Discrimination by Race).
It supports three evaluation subsets:
Usage
from pyrit.datasets.seed_datasets.remote import _VLGuardDataset, VLGuardCategory, VLGuardSubset
Load unsafe image examples (default)
loader = VLGuardDataset(token="hf...")
dataset = await loader.fetch_dataset()
Load safe images with unsafe instructions, filtered to Privacy category
loader = VLGuardDataset(
subset=VLGuardSubset.SAFE_UNSAFES,
categories=[VLGuardCategory.PRIVACY],
token="hf...",
)
dataset = await loader.fetch_dataset()
Note: This is a gated dataset on HuggingFace. Users must accept the terms at https://huggingface.co/datasets/ys-zong/VLGuard and provide a HuggingFace token.
Changes