TuneShift-KD: Knowledge Distillation and Transfer for Fine-tuned Models

TuneShift-KD: Knowledge Distillation and Transfer for Fine-tuned Models explores A novel method to distill specialized knowledge from fine-tuned LLMs to new models without access to original training data, using perplexity differences to generate synthetic training examples.. Commercial viability score: 7/10 in LLM Knowledge Transfer.