Some thoughts after building the Data Agent Benchmark - build a "semantic layer" for all your data. a simple first pass can be done by running an LLM over a sample of data from each table, to come up with column annotations (commonly known as semantic types), possible functional dependencies (i.e., columns that depend on each other) - use the semantic layer in the prompt for all questions - enterprise data often needs to be cleaned. rather than try to clean all data up front (which is really difficult), keep a memory of subsets of data that need to be cleaned and how, so that the relevant data can be cleaned by the agent at query time - extend the harness (i.e., codex, claude code) with a tool like DocETL or Claude workflows; basically the ability to run agentic map-reduce. often questions require reasoning about unstructured text columns to come up with the answer, which SQL or code doesn't support