Partial Least Squares (PLS)

Gold definitionUpdated Apr 2, 2026

Definition

Partial Least Squares (PLS) is a multivariate statistical method that constructs latent variables to model the relationship between predictor and response variables. It excels with high-dimensional, multicollinear data, maximizing the covariance between components and responses for robust predictive modeling.

At a glance

Executive summary

Partial Least Squares (PLS) is a statistical technique that simplifies complex data by finding underlying patterns (latent variables) that best predict an outcome. It's especially good for situations with many interconnected factors, helping to build robust predictive models and reduce inference costs.

TL;DR

PLS is a smart way to find hidden connections in complicated data to make better predictions, even when many factors are related, like mapping LLM states to satisfaction scores.

Key points

Constructs latent variables that maximize covariance between predictors and responses for predictive modeling.
Effectively handles high-dimensional data with multicollinearity, enabling robust prediction and reducing inference costs.
Used by researchers and ML engineers in conversational AI, chemometrics, social sciences, and process control.
Differs from PCA by being supervised (response-aware) and from OLS by handling multicollinearity and multiple responses.
Growing use in machine learning for latent space interpretation, feature engineering, and evaluating complex model outputs like LLM hidden states.

Use cases

Conversational AI Evaluation: Mapping LLM hidden states to continuous user satisfaction scores for high-fidelity evaluation and A/B testing, as seen in the BoRP framework.
Chemometrics: Predicting chemical properties (e.g., concentration, purity) from spectroscopic data (e.g., NIR, Raman spectra) in quality control and process monitoring.
Social Sciences and Marketing: Modeling customer satisfaction or brand loyalty from large survey datasets with many correlated demographic and behavioral variables.
Drug Discovery: Predicting drug efficacy or toxicity from molecular descriptors, where thousands of features might describe a compound.
Process Control: Monitoring and predicting product quality in manufacturing processes based on numerous sensor readings and operational parameters.

Also known as

PLS regression, PLS-R, Projection to Latent Structures