• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
AI
Report

Distilling skills with sustained training signals reportedly matches or beats banks up to 11 times larger

A post says fewer than 25% of topic-matched skills gave a useful signal across three Qwen models.

1 Source, 21m ago, first seen 21m ago

TLDR

A post about an NYU and Amazon paper describes skills as short written tips a model absorbs from a copy of itself that reads them. It says SGUID keeps skills that provide useful training signals both early and late: with six such skills, three of four models matched or beat banks of 30 to 71 skills on math contest tests. A second round with three new skills reportedly lifted Qwen3-8B from 64.3% to 66.3%.

Combined views

—

1 Source, first seen 21m ago

— likes— comments— saves— reposts

Combined views

—

1 Source, first seen 21m ago

— likes— comments— saves— reposts

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Featured Source

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

1 Source

Rohan Paul@rohanpaul_aiGood paper for selecting your skill files. Before distilling a skill bank, log which skills give a steady training signal and drop the rest. New NYU and Amazon paper finds that distilling a few skills that keep producing a useful training signal matches or beats distilling a skill bank up to 11× larger. Skills are short written tips, like a rule for counting cases, that a model absorbs by learning from a copy of itself that reads them. Picked by topic match, under 25% of them gave any useful signal across 3 Qwen models. SGUID keeps only skills that help early in training and still help late. With 6 such skills, 3 of 4 models matched or beat the full bank of 30 to 71 skills on math contest tests. A 2nd round with 3 new skills lifted Qwen3-8B from 64.3% to 66.3%.21m
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI

    1 Source

    Rohan Paul@rohanpaul_aiGood paper for selecting your skill files. Before distilling a skill bank, log which skills give a steady training signal and drop the rest. New NYU and Amazon paper finds that distilling a few skills that keep producing a useful training signal matches or beats distilling a skill bank up to 11× larger. Skills are short written tips, like a rule for counting cases, that a model absorbs by learning from a copy of itself that reads them. Picked by topic match, under 25% of them gave any useful signal across 3 Qwen models. SGUID keeps only skills that help early in training and still help late. With 6 such skills, 3 of 4 models matched or beat the full bank of 30 to 71 skills on math contest tests. A 2nd round with 3 new skills lifted Qwen3-8B from 64.3% to 66.3%.21m
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet