Article · Original
The article text is unavailable in this language; an existing version is shown.
– https://t.co/Z8ONXwgXeX
Title: "Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model-Skill Co-Evolution"
NYU and Amazon paper proposes SGUID to select a small set of skills for model trainingMachine translation
Automatically verified and published · Generated and evidence-checked automatically; not reviewed by a human.
The study proposes selecting skills that provide a useful training signal both early and late in training. On math contest tests, 3 of 4 models using 6 selected skills matched or beat models using the full bank of 30 to 71 skills. A second round with 3 new skills raised Qwen3-8B's score from 64.3% to 66.3%.
Good paper for selecting your skill files.
Before distilling a skill bank, log which skills give a steady training signal and drop the rest.
New NYU and Amazon paper finds that distilling a few skills that keep producing a useful training signal matches or beats distilling a skill bank up to 11× larger.
Skills are short written tips, like a rule for counting cases, that a model absorbs by learning from a copy of itself that reads them. Picked by topic match, under 25% of them gave any useful signal across 3 Qwen models.
SGUID keeps only skills that help early in training and still help late. With 6 such skills, 3 of 4 models matched or beat the full bank of 30 to 71 skills on math contest tests. A 2nd round with 3 new skills lifted Qwen3-8B from 64.3% to 66.3%.
View replied-to post on X