Announcement

LegalForecastBench and the challenge of building good legal evaluations

Its creator says many legal tasks involve judgment and discretion, making good evaluations harder to build.

AK

JH

2 Sources, 5h ago, first seen 5h ago

TLDR

The creator says they built LegalForecastBench. They argue that good legal evaluations are getting harder to make because many legal tasks involve judgment and discretion.

Today's Rank

—

Not ranked yet

Combined views

3.4K

2 Sources, first seen 5h ago

2 Sources

@jjhughesIt’s getting hard to build good legal evals. So many legal tasks involve judgment and discretion. I built LegalForecastBench to test frontier models on a high-level legal task with objective ground truth: predicting how judges will decide real motions. https://x.com/i/article/21101018149179310095h

@BlackHCRT @jjhughes: It’s getting hard to build good legal evals. So many legal tasks involve judgment and discretion. I built LegalForecastBench…5h

Today's Rank

—

Not ranked yet

Combined views

3.4K

2 Sources, first seen 5h ago

28 likes

7 comments

31 saves

5 reposts

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

7 comments

31 saves

5 reposts

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Today's Rank

—

Not ranked yet

Today's Rank

—

Not ranked yet

2 Sources

@jjhughesIt’s getting hard to build good legal evals. So many legal tasks involve judgment and discretion. I built LegalForecastBench to test frontier models on a high-level legal task with objective ground truth: predicting how judges will decide real motions. https://x.com/i/article/21101018149179310095h

@BlackHCRT @jjhughes: It’s getting hard to build good legal evals. So many legal tasks involve judgment and discretion. I built LegalForecastBench…5h

2 Sources

@jjhughesIt’s getting hard to build good legal evals. So many legal tasks involve judgment and discretion. I built LegalForecastBench to test frontier models on a high-level legal task with objective ground truth: predicting how judges will decide real motions. https://x.com/i/article/21101018149179310095h

@BlackHCRT @jjhughes: It’s getting hard to build good legal evals. So many legal tasks involve judgment and discretion. I built LegalForecastBench…5h

2 Sources

@jjhughesIt’s getting hard to build good legal evals. So many legal tasks involve judgment and discretion. I built LegalForecastBench to test frontier models on a high-level legal task with objective ground truth: predicting how judges will decide real motions. https://x.com/i/article/21101018149179310095h

@BlackHCRT @jjhughes: It’s getting hard to build good legal evals. So many legal tasks involve judgment and discretion. I built LegalForecastBench…5h