TRAINING

Distilling an LLM on One GPU: Offline Top-K Logits and a Fused Chunked KL Loss

LLM distillation on one GPU: cache the teacher's top-100 logits offline and use a fused chunked KL loss to cut a GPT-OSS 20B run from four nodes to one.