GR-SAP: Generative Replay for Safety Alignment Preservation during Fine-Tuning

Dimensions overall score 8.0

• GR-SAP synthesizes domain-specific alignment data from LLMs to preserve safety alignment during fine-tuningmethod 95%
  Evidencepartial

  we propose Generative Replay for Safety Alignment Preservation (GR-SAP), a unified framework that synthesizes domain-specific alignment data from LLMs and integrate them during downstream adaption to preserve safety alignment

  Implicationpartial

  Directly stated in abstract as the core method of the paper

  Verificationpartial

  partial

• Safety alignment of LLMs can be easily compromised by seemingly non-adversarial fine-tuningtechnical 90%
  Evidencepartial

  Recent studies show that the safety alignment of large language models (LLMs) can be easily compromised even by seemingly non-adversarial fine-tuning

  Implicationpartial

  Directly stated in abstract as motivation for the research

  Verificationpartial

  partial

• Original alignment data is typically inaccessible even for open-weight LLMslimitation 85%
  Evidencepartial

  the original alignment data, which is typically inaccessible even for open-weight LLMs

  Implicationpartial

  Directly stated in abstract as a limitation of existing approaches

  Verificationpartial

  partial

• Synthetic alignment data serves as a reliable proxy for original alignment dataresult 88%
  Evidencepartial

  Theoretical and empirical analyses demonstrate this synthetic data serves as a reliable proxy for the original alignment data

  Implicationpartial

  Directly stated in abstract with theoretical and empirical support mentioned

  Verificationpartial

  partial

• GR-SAP substantially mitigates fine-tuning-induced safety degradationresult 92%
  Evidencepartial

  Experiments across various models and downstream tasks show that GR-SAP substantially mitigates fine-tuning-induced safety degradation

  Implicationpartial

  Directly stated in abstract with experimental validation across models and tasks

  Verificationpartial

  partial

• GR-SAP maintains comparable downstream performance while preserving safetyresult 90%
  Evidencepartial

  while maintaining comparable downstream performance

  Implicationpartial

  Directly stated in abstract as a key result of the method

  Verificationpartial

  partial

• GR-SAP is inspired by generative replay in continual learningmethod 95%
  Evidencepartial

  Inspired by generative replay in continual learning

  Implicationpartial

  Directly stated in abstract as the conceptual foundation

  Verificationpartial

  partial

• A widely used strategy to preserve safety alignment is joint optimization with original alignment datatechnical 85%
  Evidencepartial

  a widely used strategy is to jointly optimize safety and task objectives by mixing in the original alignment data

  Implicationpartial

  Directly stated in abstract as current practice that GR-SAP aims to improve upon

  Verificationpartial

  partial

Competitive landscape

GR-SAP is a framework that synthesizes domain-specific alignment data to preserve safety alignment in fine-tuning large language models.