/Tech5h ago

Andy Konwinski, Databricks co-founder, argues evaluations serve as the functional source code of AI agents

Matei Zaharia highlighted evaluations for continual learning systems.

4536257.2K

#441

Original post

Parth Asawa@pgasawa

Always fun to chat with @andykonwinski, see 41:33 for some of my brief takes on evaluating continual learning and academia / open science.

Laude Institute@LaudeInstitute

At @CAISconf last month, @andykonwinski sat down with researchers on the conference floor -- @matei_zaharia @istoica05 @lateinteraction @dawnsongtweets @gneubig @pgasawa @JonSaadFalcon @heathercmiller @ryanmart3n @alexgshaw @profjoeyg @swyx and Ioannis Ioannidis -- to talk open vs. closed, agent evals, compound AI systems, and where the frontier is headed 👀

https://www.youtube.com/watch?v=PbymvxpbWHg

8:13 PM · Jun 24, 2026 · 2.7K Views

Sentiment

Positive users thank contributors like Matei Zaharia for AI agent evals and academia insights while negative users dismiss standard eval metrics as optimized noise.

Pos

50.0%

Neg

50.0%

2 comments with sentiment.

Cluster Engagement

Digg Deeper

No Digg Deeper questions have been answered for this story yet.

YOUTUBEVia

Posts from X

Most Activity

VIEWS4.3KBOOKMARKS20LIKES24RETWEETS4REPLIES4

Andy Konwinski@andykonwinski

3 take-aways from chatting w/ top AI researchers last month:

- Evals are the “source code” of AI agents (48:35)

- BigAI labs: $ + anonymous impact. Academia -> impact w/ your own individual voice (44:00)

- Pedagogical RL: agent solves problem it knows the answer to (unintuitive!); reward solutions that don't take shortcuts, then distill them into a student model. (53:40)

Laude Institute@LaudeInstitute

https://www.youtube.com/watch?v=PbymvxpbWHg

3h4.3K2420

Andy Konwinski@andykonwinski

https://www.youtube.com/watch?v=PbymvxpbWHg

...and thanks again to @matei_zaharia @istoica05 @lateinteraction @dawnsongtweets @gneubig @pgasawa @JonSaadFalcon @heathercmiller @ryanmart3n @alexgshaw @profjoeyg @swyx and Ioannis Ioannidis!

Andy Konwinski@andykonwinski

3 take-aways from chatting w/ top AI researchers last month:

- Evals are the “source code” of AI agents (48:35)

- BigAI labs: $ + anonymous impact. Academia -> impact w/ your own individual voice (44:00)

- Pedagogical RL: agent solves problem it knows the answer to (unintuitive!); reward solutions that don't take shortcuts, then distill them into a student model. (53:40)

3h32870

Strata@ChainZenit

@andykonwinski Standard eval metrics are usually just optimized noise at this point.