BREPS

Gold definitionUpdated Apr 2, 2026

Definition

BREPS is a method designed to generate adversarial bounding box prompts, reformulating the robustness evaluation of promptable segmentation models like SAM as a white-box optimization problem. It aims to assess model sensitivity to natural prompt variations, which synthetic prompts often fail to capture.

At a glance

Executive summary

BREPS is a new technique to test how well AI models that segment images based on user-drawn boxes can handle real-world variations in those boxes. It works by intelligently creating 'tricky' boxes that expose the model's weaknesses, rather than just using simple, fake ones. This helps make these AI models more reliable for actual users.

TL;DR

BREPS is a method to find tricky bounding box prompts that break image segmentation AI models, helping researchers make them more robust to real user input.

Key points

Reformulates robustness evaluation as a white-box optimization problem over bounding box prompt space.
Solves the problem of limited real-world robustness insight from synthetic prompts for segmentation models.
Used by researchers and ML engineers developing and evaluating promptable segmentation models like SAM.
Unlike simple heuristic-based prompt generation, BREPS systematically generates adversarial prompts.
Contributes to the growing trend of adversarial testing and robustness improvement for foundation models in computer vision.

Use cases

Benchmarking the robustness of new promptable segmentation models against diverse prompt variations.
Identifying specific failure modes of models like SAM when presented with naturally noisy bounding box inputs.
Guiding the fine-tuning or retraining of promptable segmentation models to improve their resilience to user input variability.
Developing more robust prompt encoding strategies for interactive segmentation systems.
Evaluating the impact of different prompt engineering techniques on model performance under adversarial conditions.

Also known as

Adversarial Bounding Box Prompt Generation, Robustness Evaluation via Prompt Space Optimization