Lead authors include Caleb Ziems, Dora Zhao, and Diyi Yang.
The next frontier of AI is not only more capable model; it is an AI that *humans* can meaningfully live and work with :)
With all students in my cs329x Human-Centered LLM class, we present 60+ pages of insights for developing Human-Centered LLMs (HCLLMs), from design & data sourcing to training, eval & deployment 🧵
Many users praise Stanford's new guide to human-centered large language models as impressive real progress and a worthwhile direction worth following.
No Digg Deeper questions have been answered for this story yet.
This is like the avengers but for human AI interaction
The next frontier of AI is not only more capable model; it is an AI that *humans* can meaningfully live and work with :)
With all students in my cs329x Human-Centered LLM class, we present 60+ pages of insights for developing Human-Centered LLMs (HCLLMs), from design & data sourcing to training, eval & deployment 🧵
The next frontier of AI is not only more capable model; it is an AI that *humans* can meaningfully live and work with :)
With all students in my cs329x Human-Centered LLM class, we present 60+ pages of insights for developing Human-Centered LLMs (HCLLMs), from design & data sourcing to training, eval & deployment 🧵
Tired of AI automating research and solving major open problems? Take a read at Diyi’s new masterpiece to gain more confidence on humans!
The next frontier of AI is not only more capable model; it is an AI that *humans* can meaningfully live and work with :)
With all students in my cs329x Human-Centered LLM class, we present 60+ pages of insights for developing Human-Centered LLMs (HCLLMs), from design & data sourcing to training, eval & deployment 🧵
Caleb Ziems (@cjziems) has a new paper out!
The next frontier of AI is not only more capable model; it is an AI that *humans* can meaningfully live and work with :)
With all students in my cs329x Human-Centered LLM class, we present 60+ pages of insights for developing Human-Centered LLMs (HCLLMs), from design & data sourcing to training, eval & deployment 🧵
Raise your hand if you have heard about "human-centered" AI or LLMs and not been sure what that means? 🙋♀️🙋♀️🙋♀️
Check out our paper for answers :)
The next frontier of AI is not only more capable model; it is an AI that *humans* can meaningfully live and work with :)
With all students in my cs329x Human-Centered LLM class, we present 60+ pages of insights for developing Human-Centered LLMs (HCLLMs), from design & data sourcing to training, eval & deployment 🧵
Link here: https://arxiv.org/pdf/2605.06901
The next frontier of AI is not only more capable model; it is an AI that *humans* can meaningfully live and work with :)
With all students in my cs329x Human-Centered LLM class, we present 60+ pages of insights for developing Human-Centered LLMs (HCLLMs), from design & data sourcing to training, eval & deployment 🧵
This paper was jointly led by @cjziems & @dorazhao9, w/ contributions from over 60 authors across @stanfordnlp and @StanfordHAI, and cs329x! We build on lots of wisdom & empirical contributions of countless researchers across HCI, NLP, and Responsible AI.
Thank you to everyone involved ❤️
In summary, we show human-centeredness is more than an alignment objective. It is a design approach across the entire HCLLM pipeline, from development to deployment. There are no simple answers.
To make this concrete, we conclude w/ a case study on HCLLMs and the Future of Work.
How do we center humans? We start with HCLLM DESIGN to understand (1) stakeholders, (2) design challenges, and (3) HCI solutions.
E.g., HCI interfaces can close the gap between user intentions and prompts -- a distinct AI design challenge called the "gulf of envisioning"
Link here: https://arxiv.org/pdf/2605.06901
Next, we look at HCLLM DEVELOPMENT from an NLP perspective.
First: training data reflects the people, institutions, cultures, histories, and social contexts that produced it. From data origins, we can understand human-centered concerns around bias, privacy, and data ownership.
How do we center humans? We start with HCLLM DESIGN to understand (1) stakeholders, (2) design challenges, and (3) HCI solutions.
E.g., HCI interfaces can close the gap between user intentions and prompts -- a distinct AI design challenge called the "gulf of envisioning"
In summary, we show human-centeredness is more than an alignment objective. It is a design approach across the entire HCLLM pipeline, from development to deployment. There are no simple answers.
To make this concrete, we conclude w/ a case study on HCLLMs and the Future of Work.
After design and development, we look at responsible HCLLM DEPLOYMENT. There is tension between 3 ideals: (1) interpretability, (2) steerability, & (3) safety.
E.g, certain alignment and steering methods can make models less interpretable; more steerable models may be less safe.
Second: in post-training, we highlight tensions and tradeoffs around preference tuning, personalization, & pluralism, tying concerns back to our earlier "data" discussion around bias & representation.
We also explore the limitations of scaling laws for human-centered objectives.
Next, we look at HCLLM DEVELOPMENT from an NLP perspective.
First: training data reflects the people, institutions, cultures, histories, and social contexts that produced it. From data origins, we can understand human-centered concerns around bias, privacy, and data ownership.
Third, we highlight pitfalls and best practices w/ evals at 3 different scopes: (1) model output, (2) human experience, (3) societal impact.
Currently, the field relies too much on surface heuristics and not enough on orienting HCLLMs towards long-term collective good objectives
Second: in post-training, we highlight tensions and tradeoffs around preference tuning, personalization, & pluralism, tying concerns back to our earlier "data" discussion around bias & representation.
We also explore the limitations of scaling laws for human-centered objectives.
After design and development, we look at responsible HCLLM DEPLOYMENT. There is tension between 3 ideals: (1) interpretability, (2) steerability, & (3) safety.
E.g, certain alignment and steering methods can make models less interpretable; more steerable models may be less safe.
Third, we highlight pitfalls and best practices w/ evals at 3 different scopes: (1) model output, (2) human experience, (3) societal impact.
Currently, the field relies too much on surface heuristics and not enough on orienting HCLLMs towards long-term collective good objectives
@Diyi_Yang woah that is a huge paper! where do you think are the biggest deltas between desirable properties of human-centered llms and the incentives of for-profit labs? stuff like sycophancy?
@Diyi_Yang @_Chuhan_Li Pretty cool that the class came together to make this happen! A good example for others to follow...
@Diyi_Yang @StanfordHAI This is what we should hill-climb on.
@p_jayannavar @_Chuhan_Li Yes i am very proud of the class😎 we did this as a class-level report in 2025 fall, and @cjziems @dorazhao9 put significant amount of efforts to put together the current version. i really hope this could be useful resources for people who're interested in this space
@ChengleiSi lol
Tired of AI automating research and solving major open problems? Take a read at Diyi’s new masterpiece to gain more confidence on humans!
This is interesting and I agree with the thesis that it’s also a design problem, not just an alignment problem. What I’m curious and would really like to start including in the thought and conversations is how to help people actually stop using tools when they’re done. Because the meaning is in the outside world and most of the tools are so optimized for accomplishment (rightfully so - and thanks for that) that we get so engaged in accomplishing - we forget to disconnect.
great question on engagement optimization, and sycophancy for sure. not the first time we have to think about this : ) just like platforms presenting controversial content might get short-term engagement but platforms with highly informative content may have a long-term edge. Same here - we need to bring in more objectives into the optimization process beyond task completion and the short-term user satisfaction, and we talked about how other goals such as user empowerment, mastery etc can be incorporated into the training target.
