Projected Gradient Method

Gold definitionUpdated Apr 2, 2026

Definition

The Projected Gradient Method is an iterative optimization algorithm that minimizes a function subject to constraints. It works by taking steps in the direction of the negative gradient and then projecting the result back onto the feasible set, ensuring the solution always satisfies the given constraints.

At a glance

Executive summary

The Projected Gradient Method is an optimization technique that helps find the best solution to a problem while making sure certain rules or limits are always followed. It does this by repeatedly moving towards a better answer and then adjusting it to fit within the allowed boundaries. This is often used to test how robust AI systems are by creating specific, limited "trick" inputs.

TL;DR

It's a math trick that finds the best answer to a problem by taking small steps and then forcing the answer to stay within specific rules.

Key points

Iteratively applies a gradient step followed by a projection onto a feasible set.
Solves constrained optimization problems, particularly useful for generating bounded adversarial examples and robust training.
Used by researchers in adversarial machine learning, robust AI, optimization, and control theory.
Unlike unconstrained gradient descent, PGM explicitly handles and enforces constraints at each step.
Central to developing and evaluating robustness in AI systems, especially against adversarial attacks.

Use cases

Adversarial Attack Generation: Crafting imperceptible perturbations (e.g., for images or audio) that cause deep learning models to misclassify, such as in the context of "AI-based error correction decoders" mentioned in the abstract.
Robust Machine Learning Training: Training models to be resilient against adversarial examples by incorporating PGM-generated perturbations into the training data.
Image Denoising and Reconstruction: Applying constraints (e.g., sparsity, non-negativity) during iterative image processing algorithms to recover cleaner or more accurate images.
Optimal Control Problems: Solving control policies where system states or control inputs must remain within specific physical or operational bounds.
Fairness in Machine Learning: Enforcing fairness constraints on model predictions or representations during training to prevent biased outcomes.

Also known as

PGM, Projected Gradient Descent, PGD, Proximal Gradient Method