Ido Amos

@amosaurusrex

MSc student at Tel-Aviv University working on ML/DL

following
followers

Byte-level H-Net reportedly reaches SmolLM3's six-task average with roughly one-tenth the training tokens

Unlike models with fixed tokenizers, H-Net learns to group raw bytes within the model, Cartesia says.

60 Sources, 14d ago, updated 14d ago

LIFT proposes deep-to-shallow feedback in Transformers while keeping pretraining parallel

A post introducing the preprint says LIFT uses teacher-supervised training to tackle a bottleneck in how language models pass information between layers.

3 Sources, 42d ago, updated 42d ago

Featured in

Byte-level H-Net reportedly reaches SmolLM3's six-task average with roughly one-tenth the training tokens

LIFT proposes deep-to-shallow feedback in Transformers while keeping pretraining parallel

Featured in

Byte-level H-Net reportedly reaches SmolLM3's six-task average with roughly one-tenth the training tokens

LIFT proposes deep-to-shallow feedback in Transformers while keeping pretraining parallel