UnconvAI releases Un-0, an image generation model built on a backbone of coupled oscillators

We’re at the beginning of the unconventional AI journey, but we are releasing artifacts to the community to enable us all to build together. We’ve open-sourced the Un-0 weights, training scripts, and ablation code.

If you build physics-based models, plug them into our scaffold and share your success!

Read the blog: https://unconv.ai/blog/introducing-un-0-generating-images-with-coupled-oscillators

Visit our GitHub: https://github.com/unconv-ai/Un-0

Another paper in the same vein (oscillators) but for classification and reasoning: https://jiawen-dai.github.io/WONN_Project_Page/

It seems that oscillators allows for good performance at much smaller parameter count. This is a super exciting line of work.

harmonic revolution gaining momentum

it does seem like the right medium for constraint propagation it always has

How does it work? Un-0 uses a system of coupled Kuramoto oscillators.

Initial random phases evolve under the pull of other oscillators through learned coupling strengths.

At Unconventional AI, we're building a new kind of computer that runs AI on the dynamics of a physical system, at a fraction of the energy today's machines need.

We’re leveraging the dynamics of a physical system, such as the noisy, time-varying behavior of analog circuits that compute with analog voltage and current instead of conventional digitized numbers.

Un-0 illuminates a path towards running modern AI workloads on physical substrates that are more efficient than today's hardware.

Is the physics actually doing the work? Yes.

Our ablations show that the trained dynamics provide value over a decoder only baseline and a random Kuramoto feature reservoir, and that increasing the number of integration steps increases model quality.

The results? We scale Un-0 up to 16k oscillators and 322M parameters and achieve an FID of ~6.74 on ImageNet 64x64.

To our knowledge, this is the most capable model based on a simulation of a physical system to date, expanding the Pareto frontier for small generative models. Though there is still remaining work to scale model quality as a function of model size.

@unconvAI In 2021 I worked on something similar for classification: https://arxiv.org/pdf/1808.08412 but the readout was through the synchronization (or lack thereof) of certain "output" pairs of oscillators, no trained readout network. Happy to see renewed interest in oscillator-based computation!

we do something very similar with our Metriplector, but instead use a more general metriplectic dynamics and Noether stress energy tensor readout...also our model works across multiple modalities with matched performance comapred to sota deep learning methods but using 10x less params.

https://arxiv.org/abs/2603.29496

@StphTphsn1 @unconvAI We tried to put quite a bit of detail in the blog. We'll do a tech report soon.

@unconvAI Coupled oscillators instead of attention is a wild swing. Kuramoto dynamics carrying the load is cool, but where does it break first, can it scale past toy resolutions or does the sync go unstable as params grow? Rooting for the weird approach!

@unconvAI this approach sounds interesting, is there a paper associated to this work?

@unconvAI If @QualiaRI is correct this further increases the odds of instantiating consciousness and I hope that’s something you’re all considering ‘:)

Amazing work though, very cool

@unconvAI Is this called Middle-out?

i think they will deliver eeeven better parameter efficiency once they support harmonic coupling (ie allow octaves - integer multiple frequencies) to couple as resonant modes. higher dimensionality latent

if i understand correctly this implementation is matching freqs directly

@unconvAI Kuramoto neural network with drifting loss(secretly score matching) ?

@unconvAI oh that's beautiful

@mcarbin @unconvAI thanks! yes I went through the blog but did not see much info about the training procedure. Is it backprop through time?

@unconvAI Pretty cool!

@jm_alexia the one you shared seems especially exciting since if i understand correctly it can learn harmonic coupling (octave resonant modes) which would allow for higher dimensional representations with few params