/knowledge-distillation
用知識蒸餾把大型語言模型從 teacher 壓縮到 student。部署保有效能的較小模型、把 GPT-4 能力轉移到開源模型，或降低推論成本時使用。
- 安裝次數
- 1000
- 專案星數
- 1.4 萬 ★
- 授權
- MIT
- 專案更新
- 2026-10-11
怎麼裝
npx skills add https://github.com/Orchestra-Research/AI-Research-SKILLs --skill knowledge-distillation安裝前請先看過原文內容。中文說明是 AI 翻譯，不保證安全。
原文
Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.