Prime Intellect's kalomaze finds Anthropic's Opus 4.7 and 4.8 have 1.41 times higher effective API costs than Opus 4.6

VIEWS813LIKES15REPLIES2

anthropic pricing docs claim: "This new tokenizer may use up to 35% more tokens for the same fixed text." my measurements claim: the average inflation appears to be higher than their ceiling. and this is... general english, not arabic or something. so that's false as stated.

kalomaze@kalomaze

hey isn't it kind of messed up that API customers of Opus 4.7 and Opus 4.8 are paying ~1.41x as much for general english output (when measured against a consistent tokenizer baseline) vs Opus 4.6? from THE ONLY lab that's cagey about releasing the tokenizer... huh. that's... odd

3h813150

BOOKMARKS1

kalomaze@kalomaze

@xlr8harder maybe they got rid of BPE nonconcatenative property or something. makes post training assumptions easier in some real and genuinely not trivial ways (it's why PrimeIntellect-ai/renderers exists). but it seems strictly worse in basically ~all ways for general modeling of language

xlr8harder@xlr8harder

@kalomaze a bunch of people were running tokenizer stats on it when it came out and it seemed kind of terrible in a lot of ways. i really wonder what the benefits are. (I'm not cynical enough to think it's merely more monetizable.)

2h21481

xlr8harder@xlr8harder

@kalomaze a bunch of people were running tokenizer stats on it when it came out and it seemed kind of terrible in a lot of ways. i really wonder what the benefits are. (I'm not cynical enough to think it's merely more monetizable.)

kalomaze@kalomaze

hey isn't it kind of messed up that API customers of Opus 4.7 and Opus 4.8 are paying ~1.41x as much for general english output (when measured against a consistent tokenizer baseline) vs Opus 4.6? from THE ONLY lab that's cagey about releasing the tokenizer... huh. that's... odd

2h19460

kalomaze@kalomaze

@xlr8harder if it destroys your compression ratio on regular english this hard, though, then that seems ~pretty ass as a tradeoff? just dont have an infra skill issue 4head?

kalomaze@kalomaze

@xlr8harder maybe they got rid of BPE nonconcatenative property or something. makes post training assumptions easier in some real and genuinely not trivial ways (it's why PrimeIntellect-ai/renderers exists). but it seems strictly worse in basically ~all ways for general modeling of language

2h8930

kache@yacineMTB

@kalomaze interesting

kalomaze@kalomaze

hey isn't it kind of messed up that API customers of Opus 4.7 and Opus 4.8 are paying ~1.41x as much for general english output (when measured against a consistent tokenizer baseline) vs Opus 4.6? from THE ONLY lab that's cagey about releasing the tokenizer... huh. that's... odd

3h58450

resident gradient@hitchhooker

@kalomaze u sure they still use tokenizer? my assumption has been that leap in cybersec capability would be partially byte latent transformer to allow understand software at binary level.

3h251

kalomaze@kalomaze

@hitchhooker ...wdym? that's just a different kind of tokenizer, anything with a 1D embedding table -> single 1D categorical softmax on discrete units is doing some kind of lm-style tokenization, byte latent whatever stuff is essentially heuristics bolted on top of the core idea

3h7

Innovus@InnoboSJ

@kalomaze Wait, Anthropic are liars?

3h6