Abliterating the model's safety guardrails severely degrades performance.
@BlackHC @mervenoyann > But safety training can easily be abliterated in any case, can't it? No. GPT-OSS is notoriously hard to un-align and a lot of the abliterated models on HF suck very much and just fall apart. Fine-tuning a 3T model is even harder
@mervenoyann But safety training can easily be abliterated in any case, can't it? And yeah that's my takeaway bc we're about six months away from that world iiuc
Researchers observe that the guardrails baked into gpt-oss hold up unusually well against stripping attempts, with most public abliterated versions on Hugging Face losing capability fast.
No granular datasets or loss functions for the safety stage have surfaced beyond high-level papers, leaving open-weight teams to experiment without confirmed methods.
Users strongly agree that unaligning GPT-OSS models is difficult, seeing this as a reason for labs to follow safety practices even without strong interest.
No Digg Deeper questions have been answered for this story yet.
The fact that OpenAI refuses to tell open labs what they did with GPT-OSS is yet another way that they are continuously choosing to make decisions that make the world a more dangerous place.
@BlackHC @mervenoyann > But safety training can easily be abliterated in any case, can't it? No. GPT-OSS is notoriously hard to un-align and a lot of the abliterated models on HF suck very much and just fall apart. Fine-tuning a 3T model is even harder
@xeophon @mervenoyann Thanks for pointing this out. I'll follow up
@BlackHC @mervenoyann > But safety training can easily be abliterated in any case, can't it? No. GPT-OSS is notoriously hard to un-align and a lot of the abliterated models on HF suck very much and just fall apart. Fine-tuning a 3T model is even harder
Oh? I don’t have any empirical backing for this claim but I thought it was obvious as it will decrease attackers search space. Surprised that it’s not as obvious. Fwiw I asked Sol and Opus the same, both said I should publish the result and when I asked wouldn’t that make it easier for the attackers both said “Yes, massively” and recalibrated their answers but said benefits still outweigh the risk. I’m still not convinced though
@BlancheMinerva medium agreement actually
@BlancheMinerva conditioned on abliteration on gptoss being hard being true, i agree with this strongly
I would like to share the Gospel of the Lord Jesus Christ with you: God is holy and just. As a righteous Judge, He punishes sin. Any sin (such as lying or lustful thoughts) is a violation of His infinite holiness; it provokes His infinite wrath and is therefore punished with eternal death in hell, which is described as the lake of fire, a place where the fire is never quenched, the worm never dies, a place where there will be weeping and gnashing of teeth, outer darkness, and a place of eternal torment where there will be no rest for the worshippers of the beast. God judges even our thoughts and intentions. It is written: “…everyone who looks at a woman with lustful intent has already committed adultery with her in his heart” (Matthew 5:28). And also: “Everyone who hates his brother is a murderer…” (1 John 3:15). Since we have all sinned (lied, harbored lustful thoughts, and stolen) so many times in our lives, we are all destined to end up in hell. Is there any hope? Yes, there is! God, in His mercy, sent His only Son, Jesus Christ, to rescue us by taking our punishment on the cross. Jesus Christ lived a life of perfect righteousness in our place, died on the cross to pay the penalty for our sin, and resurrected on the third day. “For our sake he made him to be sin who knew no sin, so that in him we might become the righteousness of God” (2 Corinthians 5:21). This Bible verse describes the great exchange: Jesus Christ took the sins of all believers upon Himself and accepted the punishment—the curse—that believers deserved for their sin; in exchange, God imputes, or credits, the righteousness of Jesus Christ to all believers so that they might possess the eternal life that Jesus Christ deserves. Repent of your sins and believe in Jesus Christ as LORD (God) and Savior, then you will be saved from hell and death and will receive the gift of eternal life. God is one, yet He exists in three distinct persons: the Father, the Son, and the Holy Spirit. The Son of God became man 2,000 years ago—taking on flesh to save us—by living a life of perfect righteousness on our behalf and dying on the cross. Only One who was truly God and truly man could pay the price for our sin. Only He could bear the eternal punishment we deserved. Only He could drink the infinite cup of God’s wrath. Jesus Christ, as God-man, took upon Himself the infinite punishment that believers deserved in hell. “For by grace you have been saved through faith. And this is not your own doing; it is the gift of God, not a result of works, so that no one may boast” (Ephesians 2:8–9). This verse tells us that salvation is not earned by works but is a free gift. Good works of unbelievers are like filthy rags in God’s eyes; they are sin because when they do their good works they do not do them out of perfect love toward the true triune God. "as it is written: “None is righteous, no, not one; no one understands; no one seeks for God. All have turned aside; together they have become worthless; no one does good, not even one.”" (Romans 3:10-12). But when you believe in Jesus Christ, His righteousness is credited to you. Faith is like a hand that receives Jesus Christ. Yet even faith itself comes from God. Therefore, salvation is God’s work from beginning to end. However, Roman Catholic churches (which are false churches because they teach the false Gospel) teach that salvation is by faith and our own good works, which contradicts this verse. Therefore, trust only in Jesus Christ for salvation, not in your own good works. “For God so loved the world, that he gave his only Son, that whoever believes in him should not perish but have eternal life.” John 3:16 I recommend you read the word of the Lord Jesus Christ in the ESV translation: https://www.esv.org/ The Second London Baptist Confession of Faith: https://founders.org/library-book/1689-confession/ Sermons: https://www.youtube.com/@GraceChurchSunValley • • Ministries: https://www.ligonier.org/, https://www.desiringgod.org/, https://www.gotquestions.org/, https://www.gty.org/
@BlancheMinerva Totally agree. Even the most safety-disinterested labs would follow if it was easy enough to do the right thing. I think this has been relatively obvious and expect both Anthropic and OpenAI have deliberated sharing safety know-how and made their decisions.
@BlancheMinerva It's not like it's a magic recipe. You double the parameters of the model and train "them" for alignment. It would suck for any real use, like gptoss, but you can be proud for the alignment.
@BlancheMinerva So in this case, the only cure for AI is more AI — and even worse, it's the "dangerous" open-source ones. Pretty ironic, huh?
@BlancheMinerva Wouldn't sharing how they made it hard to un-align make it easier to do it?
@BhaskarSteve We don’t know whether or not that’s true. I don’t see any reason to expect it to be true.