People who care about AI: Free Systems needs your help! What would you like to know about the capabilities, dangers, and other characteristics of cutting-edge AI models?
Inspired by a suggestion from @Miles_Brundage we are working to design a simple, clean model card visualizer. The goal is to give people a quick, easy way to see what's different across models and what's new in the latest models.
But it turns out, it's pretty hard to boil down model cards to a single actual "card." New model cards can be many pages long. Most of the data provided in model cards is not directly comparable to other cards (there's almost no overlap in evals, we've found).
So we want to know: what are you looking for in a model card? What are the pieces of information that would be most valuable to you? And how would we standardize these cards across labs and models?
If you have a moment, please fill out our survey here:
https://forms.gle/9k6E5e11Z4WNUim88
We're hoping to ship our visualizer two weeks from today, and would really appreciate your input.
