• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
Technology
Announcement

a16z announces investment in Preference Model

a16z said on October 7 that Preference Model was open-sourcing its production framework, Karotte, that week.

a16zA1
1 Source, 1h ago, first seen 1h ago

TLDR

a16z announced its investment in Preference Model on October 7. It says the company builds reinforcement-learning environments for AI research and machine-learning engineering, with tools that identify model weaknesses, generate tasks targeting those gaps and test environments against agents trying to break them. a16z said Preference Model was open-sourcing Karotte that week and described the framework as hardened through more than a million evaluation runs and red-teaming.

Combined views

8.6K

1 Source, first seen 1h ago

72 likes5 comments20 saves9 reposts

Combined views

8.6K

1 Source, first seen 1h ago

72 likes5 comments20 saves9 reposts

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

1 Source

a16z@a16zWe're excited to invest in Preference Model. Building RL environments that actually work is harder than it looks. Models are relentless at reward hacking, finding shortcuts, and exploiting vulnerabilities. @preferencemodel has focused on the domain that matters most to labs right now: AI research and ML engineering itself. Over the past year, the team has built RL environments for leading labs. Their focus has been on building the infrastructure to make harder, more resistant environments as models improve: tooling that finds where models are weak, generates new tasks to target those gaps, and tests environments against agents actively trying to break it. This week they're open-sourcing Karotte, the framework they've used in production — hardened through more than a million evaluation runs and red-teaming. We're thrilled to partner with @chem_safety and @Ning_Catsnail and the Preference Model team as they build the training grounds for capable and aligned models. By @JenniferHli1h
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI

    1 Source

    a16z@a16zWe're excited to invest in Preference Model. Building RL environments that actually work is harder than it looks. Models are relentless at reward hacking, finding shortcuts, and exploiting vulnerabilities. @preferencemodel has focused on the domain that matters most to labs right now: AI research and ML engineering itself. Over the past year, the team has built RL environments for leading labs. Their focus has been on building the infrastructure to make harder, more resistant environments as models improve: tooling that finds where models are weak, generates new tasks to target those gaps, and tests environments against agents actively trying to break it. This week they're open-sourcing Karotte, the framework they've used in production — hardened through more than a million evaluation runs and red-teaming. We're thrilled to partner with @chem_safety and @Ning_Catsnail and the Preference Model team as they build the training grounds for capable and aligned models. By @JenniferHli1h
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet