Its creator says the OpenTTD-based benchmark measures frontier LLM agents on real-time, long-horizon tasks.
OpenTTD-Bench’s creator announced the benchmark as a test of frontier LLM agents on real-time, long-horizon tasks. In the OpenTTD business simulation, the agents run competing transportation companies and try to make as much profit as possible.
—
3 Sources, first seen ago
—
Not ranked yet
—
Not ranked yet
@thesophiaxuI’m releasing OpenTTD-Bench, a benchmark for measuring frontier LLM agents on real-time, long-horizon tasks, based on a complex business simulation game OpenTTD, where models run competing transportation companies and must make the most amount of profit.
@PrimeIntellectRT @thesophiaxu: I’m releasing OpenTTD-Bench, a benchmark for measuring frontier LLM agents on real-time, long-horizon tasks, based on a co…
@xeophonEvals built by @PrimeIntellect residents, fully open: - MazeBench by @patience_cave - PMPP-Hard by @myainotez - PretrainGym by @HuNtEr_J1324 - OpenTTD-Bench by @thesophiaxu [and even more :) ] Super happy to support such amazing work, check it out!@thesophiaxuI’m releasing OpenTTD-Bench, a benchmark for measuring frontier LLM agents on real-time, long-horizon tasks, based on a complex business simulation game OpenTTD, where models run competing transportation companies and must make the most amount of profit.@PrimeIntellectRT @thesophiaxu: I’m releasing OpenTTD-Bench, a benchmark for measuring frontier LLM agents on real-time, long-horizon tasks, based on a co…@xeophonEvals built by @PrimeIntellect residents, fully open: - MazeBench by @patience_cave - PMPP-Hard by @myainotez - PretrainGym by @HuNtEr_J1324 - OpenTTD-Bench by @thesophiaxu [and even more :) ] Super happy to support such amazing work, check it out!@thesophiaxuI’m releasing OpenTTD-Bench, a benchmark for measuring frontier LLM agents on real-time, long-horizon tasks, based on a complex business simulation game OpenTTD, where models run competing transportation companies and must make the most amount of profit.@PrimeIntellectRT @thesophiaxu: I’m releasing OpenTTD-Bench, a benchmark for measuring frontier LLM agents on real-time, long-horizon tasks, based on a co…@xeophonEvals built by @PrimeIntellect residents, fully open: - MazeBench by @patience_cave - PMPP-Hard by @myainotez - PretrainGym by @HuNtEr_J1324 - OpenTTD-Bench by @thesophiaxu [and even more :) ] Super happy to support such amazing work, check it out!@thesophiaxuI’m releasing OpenTTD-Bench, a benchmark for measuring frontier LLM agents on real-time, long-horizon tasks, based on a complex business simulation game OpenTTD, where models run competing transportation companies and must make the most amount of profit.@PrimeIntellectRT @thesophiaxu: I’m releasing OpenTTD-Bench, a benchmark for measuring frontier LLM agents on real-time, long-horizon tasks, based on a co…@xeophonEvals built by @PrimeIntellect residents, fully open: - MazeBench by @patience_cave - PMPP-Hard by @myainotez - PretrainGym by @HuNtEr_J1324 - OpenTTD-Bench by @thesophiaxu [and even more :) ] Super happy to support such amazing work, check it out!
