Automatic Laplace Collapsed Sampling: Scalable Marginalisation of Latent Parameters via Automatic Differentiation

Dimensions overall score 4.0

• Automatic Laplace Collapsed Sampling (ALCS) is a general framework for marginalizing latent parameters in Bayesian models using automatic differentiation.method 95%
  Evidencepartial

  We present Automatic Laplace Collapsed Sampling (ALCS), a general framework for marginalising latent parameters in Bayesian models using automatic differentiation

  Implicationpartial

  This is the core definition of ALCS provided in the abstract and introduction.

  Verificationpartial

  partial

• ALCS reduces the effective dimension for Bayesian evidence computation from d_θ + d_z to d_θ by collapsing high-dimensional latent variables z via MAP optimization and Laplace approximation.method 95%
  Evidencepartial

  This reduces the effective dimension from d_θ+ d_z to just d_θ, making Bayesian evidence computation tractable for high-dimensional settings

  Implicationpartial

  The abstract explicitly states this dimensionality reduction and the mechanism by which it is achieved.

  Verificationpartial

  partial

• The MAP optimization and Hessian evaluation in ALCS are parallelized across live points on GPU hardware, enabling practical application at scale.technical 90%
  Evidencepartial

  The MAP optimisation and Hessian evaluation are parallelised across live points on GPU-hardware, making the method practical at scale.

  Implicationpartial

  The abstract mentions parallelization on GPUs as a key enabler for scalability.

  Verificationpartial

  partial

• ALCS enables local approximations beyond Laplace, such as to the Student-t distribution, using automatic differentiation to improve evidence estimates for heavy-tailed latents.method 90%
  Evidencepartial

  We also show that automatic differentiation enables local approximations beyond Laplace to parametric families such as the Student-t, which improves evidence estimates for heavy-tailed latents.

  Implicationpartial

  The abstract explicitly states that autodiff enables approximations beyond Laplace and mentions the Student-t distribution as an example for heavy-tailed latents.

  Verificationpartial

  partial

• ALCS can be combined with nested sampling to explore the hyperparameter space robustly and efficiently.method 95%
  Evidencepartial

  which we combine with nested sampling to explore the hyperparameter space in a robust and efficient manner.

  Implicationpartial

  This combination is stated as a core aspect of the ALCS framework in the abstract.

  Verificationpartial

  partial

• The computational scaling of ALCS is O(d_θ^3 * d_z), representing a substantial improvement over O((d_θ + d_z)^3) for joint nested sampling when d_θ << d_z.technical 90%
  Evidencepartial

  This scales asO(d^3_θ ×d z), which ford θ ≪d z represents a substantial improvement over theO((d θ +d z)3)cost of joint nested sampling.

  Implicationpartial

  The paper explicitly states the computational scaling for ALCS and compares it to full nested sampling.

  Verificationpartial

  partial

• ALCS is designed for near-Gaussian latent spaces and primarily targets quadratic surfaces, with potential breakdown in anisotropic spaces.limitation 85%
  Evidencepartial

  ALCS is designed for near-Gaussian latent spaces, and as such we are primarily targeting quadratic surfaces. Although optimisation in such spaces scales well [Liu and Nocedal, 1989, Zhang et al., 2022], it breaks down in anisotropic spaces

  Implicationpartial

  The analysis section discusses the design intent and limitations regarding latent space geometry.

  Verificationpartial

  partial

• ALCS can achieve exact-Gaussian accuracy and scale to d=25,600 latent dimensions in hierarchical models.result 90%
  Evidencepartial

  a hierarchical supernova model (demonstrating exact-Gaussian accuracy andD= 25,600scaling)

  Implicationpartial

  The experiments section mentions a hierarchical supernova model demonstrating exact-Gaussian accuracy and scaling to 25,600 dimensions.

  Verificationpartial

  partial

Competitive landscape

A framework for scalable Bayesian inference using automatic differentiation to efficiently marginalize latent parameters in complex models.