Artificial Analysis announced AA-PersonalityBench on Oct. 19, 2026, to compare how language models communicate. It is not a test of which model gives the most correct answers. As the company explains, the scores describe how models tend to write and respond, and a trait may be desirable or undesirable depending on the reader.
How the traits are scored
Each model answers the same 900 questions once, with each question sent as a single user message and no system prompt, according to Artificial Analysis’s methodology. There are 30 questions written for each of 30 traits. The company uses the responses for both its personality judgments and its separate writing measurements. It keeps the full question set private but publishes sample questions.
For a given trait, one LLM judge compares two models’ answers to a question written for that trait. The judge sees the question, trait definition and answers, but not the model names. Answer order is randomized, and the judge chooses which answer shows more of the trait or calls a tie. Artificial Analysis uses the verdicts to calculate an Elo rating for each trait.
The traits feed eight indexes: Expressiveness, Organization, Informativeness, Transparency, Warmth, Conviction, Imagination and Slop. For each index, Artificial Analysis standardizes its traits’ Elo ratings against the evaluated models, averages them and assigns a percentile rank from 1 to 100. It reverses the contribution of sycophancy to Transparency. An index rank is relative to the models evaluated, not a verdict that a communication style is desirable.
What the software counts
Separately, Artificial Analysis calculates 21 writing characteristics across answers to the 900 questions. They cover length, structure, punctuation and emphasis, including word counts, tables, em dashes and emoji. These measurements are distinct from the LLM-judged traits.
In its announcement, the company reported that Qwen3.8 27B averaged about 1,030 words per response, compared with 270 for GPT-6 Luna. Those are averages across the benchmark questions, not lengths to expect from every reply. The evaluation page also lets readers select and weight traits to create their own ranking. None of those rankings measures whether a model’s answers are correct.
