ESP32-S3 Packs 28.9M Parameter LLM Into $8 AI Storytelling Device
Reactions from ranked influencers
2 posts
MY THOUGHTS ON THE $8 AI COMPUTER. I ran a few hours of experiments and I already have a few big ideas here! It always started with some called called a toy. I did not go looking for a personal knowledge system. I was simply testing the $8 AI Machine that improbable 28.9-million-parameter storyteller forced onto an ESP32-S3 when something quiet and unexpected happened. I gave the little chip a handful of my own short notes and a day’s schedule, then asked it to speak. What came back was not a list. It was a small narrative. The day had a beginning, a middle, and a gentle sense of intention. The notes were no longer static text; they had been woven into a story the machine told in its own limited, coherent voice. In that moment the device stopped being a curiosity and became a big possibility. I now have nodes that can narrate the shape of the day. Others hold clusters of personal notes and, when asked, re-tell them as short spoken tales. Still others convert ordinary reminders into soft story-framed nudges that feel more like guidance than alerts. This is a small swarm of several $8 boards, each given a narrow domain (schedule, notes, health intentions, project status), lightly networked so they can hand narrative threads to one another. One expert finishes speaking and invites the next. The result is not a single general intelligence. It is a quiet council of specialized storytellers that live entirely offline, own their own knowledge, and never phone home. These are the Personal Knowledge Nodes socialized domain experts built at the absolute edge of what the silicon can do. Do more and more with less and less. The constraints are severe, and that is precisely the point. This current model is trained on simple stories. It cannot reason deeply, retrieve large stores of fact, or follow complex instructions. So the system design must stay inside those limits. Knowledge is stored as short, well-structured text. Prompts are crafted as narrative seeds. The machine is asked only to continue a story, never to invent a world it cannot hold. Working inside those boundaries has proven more generative than working against them. This proof case worked. And it is already past toy stage. I am now training a new AI model specifically for this role: a very simple domain expert. It will be smaller still, deliberately specialized, and tuned only to the socialized data that make a personal knowledge node useful. The goal is not breadth. The goal is a reliable, low-power voice that can own one narrow slice of a life and speak it clearly, day after day, with no cloud and almost no energy. This is how real innovation happens. You push all the way to the smallest possible limit. You accept the toy. At first. You live inside its severe constraints until the constraints themselves begin to teach you. Only then do you work back outward, carrying what the limit revealed. Every device we now consider essential once existed in the mind of an expert as a toy. The first personal computers were toys. The first networks were toys. The first local language models that could run on a microcontroller were toys. The experts who dismissed them were not wrong about the present; they were simply blind to the future that the constraint made visible. The $8 AI Machine is a toy. That is its greatest strength. Because it is a toy, it forces honesty. Because it is a toy, it invites play. And because it is a toy that can already remember and narrate a fragment of a human life without ever leaving the desk, it has opened a door that larger, more “serious” systems have not. I will keep training the new domain-expert model. I will keep building the small councils of storytellers and domain experts. And I will keep treating the whole effort as a toy — the same way the people who once held the first microprocessors for a calculator in their hands treated those little chips, never quite realizing they were already holding the future.
THIS IS AN ENTIRE SELF CONTAINED $8 AI COMPUTER! ESP32-AI adds a 28.9M LLM to a microcontroller. It can store 28.9 million parameters; for context, the very first version of ChatGPT had 117 million parameters. Thus, this ESP32 project only has about a quarter of the smarts of OpenAI's first model, but given how this thing was squashed onto an ESP32 device, that's a far more impressive feat than it might sound at first. This opens up a massive opportunity for dozens of these devices in a network working on specific domains and returning results to a master ESP32. This is my plan…
MY THOUGHTS ON THE $8 AI COMPUTER. I ran a few hours of experiments and I already have a few big ideas here! It always started with some called called a toy. I did not go looking for a personal knowledge system. I was simply testing the $8 AI Machine that improbable 28.9-million-parameter storyteller forced onto an ESP32-S3 when something quiet and unexpected happened. I gave the little chip a handful of my own short notes and a day’s schedule, then asked it to speak. What came back was not a list. It was a small narrative. The day had a beginning, a middle, and a gentle sense of intention. The notes were no longer static text; they had been woven into a story the machine told in its own limited, coherent voice. In that moment the device stopped being a curiosity and became a big possibility. I now have nodes that can narrate the shape of the day. Others hold clusters of personal notes and, when asked, re-tell them as short spoken tales. Still others convert ordinary reminders into soft story-framed nudges that feel more like guidance than alerts. This is a small swarm of several $8 boards, each given a narrow domain (schedule, notes, health intentions, project status), lightly networked so they can hand narrative threads to one another. One expert finishes speaking and invites the next. The result is not a single general intelligence. It is a quiet council of specialized storytellers that live entirely offline, own their own knowledge, and never phone home. These are the Personal Knowledge Nodes socialized domain experts built at the absolute edge of what the silicon can do. Do more and more with less and less. The constraints are severe, and that is precisely the point. This current model is trained on simple stories. It cannot reason deeply, retrieve large stores of fact, or follow complex instructions. So the system design must stay inside those limits. Knowledge is stored as short, well-structured text. Prompts are crafted as narrative seeds. The machine is asked only to continue a story, never to invent a world it cannot hold. Working inside those boundaries has proven more generative than working against them. This proof case worked. And it is already past toy stage. I am now training a new AI model specifically for this role: a very simple domain expert. It will be smaller still, deliberately specialized, and tuned only to the socialized data that make a personal knowledge node useful. The goal is not breadth. The goal is a reliable, low-power voice that can own one narrow slice of a life and speak it clearly, day after day, with no cloud and almost no energy. This is how real innovation happens. You push all the way to the smallest possible limit. You accept the toy. At first. You live inside its severe constraints until the constraints themselves begin to teach you. Only then do you work back outward, carrying what the limit revealed. Every device we now consider essential once existed in the mind of an expert as a toy. The first personal computers were toys. The first networks were toys. The first local language models that could run on a microcontroller were toys. The experts who dismissed them were not wrong about the present; they were simply blind to the future that the constraint made visible. The $8 AI Machine is a toy. That is its greatest strength. Because it is a toy, it forces honesty. Because it is a toy, it invites play. And because it is a toy that can already remember and narrate a fragment of a human life without ever leaving the desk, it has opened a door that larger, more “serious” systems have not. I will keep training the new domain-expert model. I will keep building the small councils of storytellers and domain experts. And I will keep treating the whole effort as a toy — the same way the people who once held the first microprocessors for a calculator in their hands treated those little chips, never quite realizing they were already holding the future.
THIS IS AN ENTIRE SELF CONTAINED $8 AI COMPUTER! ESP32-AI adds a 28.9M LLM to a microcontroller. It can store 28.9 million parameters; for context, the very first version of ChatGPT had 117 million parameters. Thus, this ESP32 project only has about a quarter of the smarts of OpenAI's first model, but given how this thing was squashed onto an ESP32 device, that's a far more impressive feat than it might sound at first. This opens up a massive opportunity for dozens of these devices in a network working on specific domains and returning results to a master ESP32. This is my plan…