late night on the dgx spark. (jetson)
no cs degree. no bootcamp. never took a programming class. everything i know about inference, docker, gpu memory budgets, and benchmarking i learned in the last 4 years by doing it with ai tools/agents
4 weeks ago i had one model running locally. tonight there are 19 models on disk, 665gb of weights.
the gap between "i can't code" and "i'm running local llm inference benchmarks and sanity checking other people's numbers" is smaller than it looks when you have the right tools and the patience to break things.
and boy do i break things.
dgx spark/hermes/modelofchoice/goal/sendit