• Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    AI
    Announcement

    Next-Concept Modeling is claimed to be up to 31× faster than similar-sized diffusion models on summarization and translation

    Its creators also report 10–21× faster unconditional generation on three datasets, with comparable or better generative perplexity.

    Azalia MirhoseiniAM
    7 Sources, 2h ago, first seen 2h ago

    TLDR

    The team behind Next-Concept Modeling says its parallel, coarse-to-fine approach generated text up to 31× faster than similarly sized diffusion language models on tasks including summarization and machine translation, with comparable or better quality. It also reports 10–21× faster unconditional generation across TinyStories, LM1B and OpenWebText while using 3–5× fewer training tokens. The team says NCM outperformed autoregressive and diffusion baselines on Sudoku with fewer sampling steps.

    Combined views

    1.2K

    7 Sources, first seen 2h ago

    21 likes2 comments19 saves3 reposts

    Combined views

    1.2K

    7 Sources, first seen 2h ago

    21 likes2 comments19 saves3 reposts
    Image: @KumbongHermann on X

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Featured Source

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    7 Sources

    Hermann Kumbong@KumbongHermann🚀 Introducing Next-Concept Modeling (NCM): our new paradigm for fast, parallel, coarse-to-fine text generation. Compared to strong diffusion language models (DLMs) at similar parameter size, NCM achieves: ⚡ Up to 31× faster generation on downstream tasks like summarization and machine translation with comparable or better quality. 🏎️ 10–21× faster unconditional generation across TinyStories, LM1B, and OpenWebText, with comparable or better generative perplexity and using 3–5× fewer training tokens. 🧩 Better performance on reasoning tasks like Sudoku, outperforming both AR and diffusion baselines with fewer sampling steps. With @ishanskhare, @IddahMlauzi, @ekellbuch, @simran_s_arora, @Azaliamirh, and @HazyResearch. More details in the thread 🧵👇2h
    Simran Arora@simran_s_aroraRT @KumbongHermann: 🚀 Introducing Next-Concept Modeling (NCM): our new paradigm for fast, parallel, coarse-to-fine text generation. Compar…2h
    Yohei@yoheinakajimalook at the way this thing generates58m
    Azalia Mirhoseini@AzaliamirhIntroducing Next-Concept Models (NCMs), a new paradigm for language generation that is significantly faster and more data efficient than diffusion models. How it works: 1) First we learn a series of compressed representations of text from coarse to fine-grained, aka ‘concepts’. This step is done once, similar to training a tokenizer. 2) At inference, we generate text going from a coarse to fine hierarchy via these ‘concepts’. A transformer predicts each level of the hierarchy sequentially, but the codes within each level are generated in parallel. Compared to strong diffusion language models, NCM achieves: Up to 31× faster generation on downstream tasks like summarization and machine translation. 10-21× faster generation across TinyStories, LM1B, and OpenWebText, using 3-5× fewer training data. On reasoning tasks like Sudoku, NCM outperforms both autoregressive and diffusion baselines with fewer sampling steps. Lead authors: @KumbongHermann & @ishanskhare, together with a great team!39m
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI

    7 Sources

    Hermann Kumbong@KumbongHermann🚀 Introducing Next-Concept Modeling (NCM): our new paradigm for fast, parallel, coarse-to-fine text generation. Compared to strong diffusion language models (DLMs) at similar parameter size, NCM achieves: ⚡ Up to 31× faster generation on downstream tasks like summarization and machine translation with comparable or better quality. 🏎️ 10–21× faster unconditional generation across TinyStories, LM1B, and OpenWebText, with comparable or better generative perplexity and using 3–5× fewer training tokens. 🧩 Better performance on reasoning tasks like Sudoku, outperforming both AR and diffusion baselines with fewer sampling steps. With @ishanskhare, @IddahMlauzi, @ekellbuch, @simran_s_arora, @Azaliamirh, and @HazyResearch. More details in the thread 🧵👇2h
    Simran Arora@simran_s_aroraRT @KumbongHermann: 🚀 Introducing Next-Concept Modeling (NCM): our new paradigm for fast, parallel, coarse-to-fine text generation. Compar…2h
    Yohei@yoheinakajimalook at the way this thing generates58m
    Azalia Mirhoseini@AzaliamirhIntroducing Next-Concept Models (NCMs), a new paradigm for language generation that is significantly faster and more data efficient than diffusion models. How it works: 1) First we learn a series of compressed representations of text from coarse to fine-grained, aka ‘concepts’. This step is done once, similar to training a tokenizer. 2) At inference, we generate text going from a coarse to fine hierarchy via these ‘concepts’. A transformer predicts each level of the hierarchy sequentially, but the codes within each level are generated in parallel. Compared to strong diffusion language models, NCM achieves: Up to 31× faster generation on downstream tasks like summarization and machine translation. 10-21× faster generation across TinyStories, LM1B, and OpenWebText, using 3-5× fewer training data. On reasoning tasks like Sudoku, NCM outperforms both autoregressive and diffusion baselines with fewer sampling steps. Lead authors: @KumbongHermann & @ishanskhare, together with a great team!39m
    Today's Rank

    #18

    Today's Rank

    #18