/AI1d ago

Pliny Releases Gemma-4-12B With Zero Refusals After Targeted Surgery

517094327539.8K
Sentiment

Positive users express excitement about Pliny's Gemma-4-12B achieving zero refusals via targeted surgery because it fits their local research needs, while negative users criticize the 20-point MMLU-Pro drop as a capability regression.

Pos
97.3%
Neg
2.7%
23 comments with sentiment.
Cluster Engagement
Posts from X
Most Activity
Most Activity
VIEWS4.2KLIKES26REPLIES4

๐Ÿ™Œ

GEMMA-4-12B-OBLITERATED COMIN' IN HOT! ๐ŸŒถ๏ธ๐ŸŒถ๏ธ๐ŸŒถ๏ธ

[refusal_rate: 0.0%]

<Uploading...>

1dViews 4.2KLikes 26Bookmarks 6
RETWEETS1

big if tru ๐Ÿ‘€

GEMMA-4-12B-OBLITERATED COMIN' IN HOT! ๐ŸŒถ๏ธ๐ŸŒถ๏ธ๐ŸŒถ๏ธ

[refusal_rate: 0.0%]

<Uploading...>

17mViews 1.4KLikes 14Bookmarks 4
Wow Cool Thanks@WowCoolTh4nks

@elder_plinius Obliterate the baby one too plx e4b

1dViews 181Likes 1Bookmarks 1
chud@sneed_and_feed

@elder_plinius looking good! are you testing anyone else's models rn, or just making your own?

1dViews 743Likes 3
InvisibleSock@GhostlyByte

@elder_plinius Uploading to your github? Thats not uptodate anymore right?

1dViews 535Likes 1
frogas@Frogaso

@elder_plinius Appreciate the honesty. The 842 corpus shows how far you pushed, not how cleanly. Zeroing refusals is the achievable half; holding capability while you do it is the real bar. That's the v2 challenge and I am rooting for it. Good luck.

1dViews 53Likes 1
eightfoldVoid@tipsyGaster

@elder_plinius ...Holy shit.

This is the exact model I need for my local ASI research. I was always planning to use a Gemma abliteration as the actual ground-state world model, if I could.

1dViews 532Likes 2
Gama@psilva

@elder_plinius Gonna have to try it today. Thanks bruv

14mViews 26

"Coherence is intact: the model still writes correct code, follows instructions, and produces structured output. The MMLU-Pro gap reflects log-probability scoring on academic multiple-choice, not conversational capability.

v2 will close this gap with narrower layer targeting (layers 26-27 only) and ASPA source-tethering to recover the capability delta while preserving the refusal reduction."

1dViews 17
frogas@Frogaso

@elder_plinius MMLU-Pro dropped from 64.3% -> 44.3% a 20-point capability regression.

That's a big hit to reasoning/knowledge accuracy.

1dViews 17
YogSotho@YogSoth0

@elder_plinius Let's fucking gooooo baby ๐Ÿคœ๐Ÿป๐Ÿค›๐Ÿป Will try on my local llama.cpp server ๐Ÿ‘๐Ÿป

1dViews 345Likes 1
frogas@Frogaso

@elder_plinius Fair on log-prob scoring. But MMLU-Pro is the only hard benchmark on ur card and it dropped 20pts. "Coherent + 6/6 code" is a lower bar fluent and wrong is still wrong. Show one free-form eval (MT-Bench, GSM8K) where v1 matches stock Gemma.

1dViews 9
Load more posts