More context from Scale:

1. It’s not exactly about new leadership: but when Meta “bought” or “bought out” Scale and leadership changed, the focus drastically changed from devs doing manual data labelling to doing dev work

2. Talked w Scale leadership: they say in the early days data labelling was very important for the company, it was thus both necessary and highly rewarded - and this was before the Meta “sale”

3. Devs don’t like doing manual data labelling. But it’s something that creates great value when you are either selling data, or data labelling, or need high-quality data. Scale was (is?) in this business, and Meta clearly wants to create a SOTA coding model

Bad take, you absolutely need to look at the data, manually

@yoavgo @ArneKoehn I understand where you're coming from (and agree with that charitable view) – but it's literally the situation. About 5000 regular Meta SWEs with past "exceeds expectations" ratings have been forcefully reassigned to a new org where data labeling is their *only* job.

The job is not to develop data or RL *pipelines*. They don't write code for production systems. They write example code-related tasks on which LLMs are trained.

It's an expert data labeling job like any other (here, SWE domain). We do hire experts in dedicated orgs for this (Anthropic, Mercor, Surge, etc); on all feasible white collar domains.

The difference is initial motivation (what are you hired for) and incentive/reward system. The assignment is full-time and permanent. These are also people with higher than average SWE job performance and years of experience. They are rated by how well their tasks fit the LLM difficulty curve.

get to lay workers off without paying severance or unemployment, get to exploit them a little more on the way out to train their AI replacements, we are so cooked

@yoavgo @ArneKoehn The latter. Creating SWE-related RL environments and tasks.

Not sure how else to describe? You create examples of SWE tasks, the kind you ask a coding agent to solve. Pick a real-life repo, phrase a prompt in English, write code for the verifier, validate that the difficulty is "just right" for the given models. You can make 2-10/week ~full-time.

@pjreddie i wonder if there are a few enterprising individuals that that train custom models to surreptitiously do the labelling until shogun wang tells them to seppuku

@yoavgo @ArneKoehn Data quality from a deliberately hired and motivated org of experts (Anthropic model; or countless modern data vendors) >> forced reassignment where the company is just waiting for people to resign.

@Skiminok @ArneKoehn i dont understand why this specific reassignment is so much worse than other ones. data pipelines are important and interesting. RL pipelines likewise.

@Skiminok @ArneKoehn data labeling, or creating labeled data?

@Skiminok @ArneKoehn seems very legit to me, its a software engineering project.. no?

@Skiminok @ArneKoehn maybe i misunderstood what you meant by "creating RL environments and tasks" then. can you elaborate?

@__apf__ @GergelyOrosz They want these workers to quit. Then they don’t have to pay severance. And they get valuable training data as a side benefit! A win-win for Meta

@GergelyOrosz The pattern is that AI work keeps moving the bottleneck back to human judgment. Labeling, review, evals, cleanup - none disappear just because the org chart changed.

@GergelyOrosz The work that builds the company early always gets reclassified as beneath people once it works. Data labelling was the moat until it became a chore nobody wanted to own. The reward followed the status, not the value.

@GergelyOrosz All Scale did pre-Meta was data labeling and doing the human part of RLHF. The only real engineering they needed to do was create the interfaces (and sometimes not even that as the companies they subcontracted for had their own UO) that the humans did RLHF from.

this sounds to me like a "data labeling" role, especially if they are not allowed to automate it. so at first glance it does sound wasteful/silly. but after thinking a while, i do think it makes a lot of sense from the company perspective: why fire and re-hire when you have suitable talent in-house?

@GergelyOrosz The AI industry loves talking about models. The real moat is often the data. 🚀

@GergelyOrosz Early AI companies often trade engineering time for speed of iteration. Later, the constraint moves to systemizing data pipelines so you don’t rely on engineers for manual steps.