DAIR.AI co-founder Elvis Saravia says effective AI agent steering requires human domain experts using verifiers and iterative feedback

You can see examples of HTML artifacts for finding unknowns here: https://thariqs.github.io/html-effectiveness/unknowns/

this is one of the posts that my AIE talk was based on

s/o to @geoffreylitt for an incredible discussion at AIE that helped me refine some of this, including adding quizzes

@omarsar0 thank you! i’m glad it resonated

One of the biggest levers for me has been interacting with agents using voice, text, and visual annotations. My prompts are now multimodal by default. It's the more natural approach to have this level of interaction with agents to get more from them and find those unknowns.

And I must add that this is strongly connected with something I have been sharing lately related to judges and verifiers. These verifiers require you to look at AI outputs and steer them towards the correct goals. It's not just prompts; it's deeper levels of understanding the inner workings of these models through richer interactions. This is also another indicator of the importance of developing expertise/knowledge in the specific domain/area where you are using AI agents.

@trq212 You're on fire these days and it's great to see.

@trq212 @hnshah amazing stuff

@Nabil_Alouani_ thanks man! means a lot

@trq212 This would be great... if I could audit my own code base for a simple health app without being flagged like I'm trying to hack China, you guys need to uncuck fable asap

@trq212 when do we get your HTML file skill?

@trq212 Models have unknowns too even Fable 5, you need multi-family to cover their blind spots if you want to be thorough and not just single-model vibing

@omarsar0 People gotta slow down, man. Always eager to jump in code ASAP. That's not the way. You gotta build into it, with at least one brainstorming session

@omarsar0 Autonomous loops have this problem. Something is constantly being built, but not exactly what we wanted… that’s even assuming we knew what we wanted in full detail.

@omarsar0 Counterpoint: at some point "prompt engineering" just becomes... communication skills with extra tabs open

@trq212 Love the metaphor!

@DimitriosMitsos @trq212 honestly multi-model ensembles feel like the only way to get any real confidence these days, single anything is just vibes.

@trq212 Banger article as usual, Thariq.

@floinkus @trq212 If you have me files. It is pretty much few lines of prompting, which you can adapt that suits you best. I have been using this for few months and it definitely helps to review vast amounts of info. Collapsible accordions are your friend.

@trq212 The problem is always to figure out “what you don’t know that you don’t know”, if you “know you don’t know” you can ask questions but how to solve the first? Best I figures out is to share what you already know and have your knowledge corrected or just read more

@trq212 Great insight my friend...but the problem is Fable5 (limited usage), I had to stop my work 3 times since morning and again waiting....