• Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    AI

    The case for moving computation closer to memory to reduce processor wait times

    A post outlines processing-in-memory, dataflow and neuromorphic designs, along with the practical limits of each.

    1 Source, 1h ago, first seen 1h ago

    TLDR

    A post argues that processors spend much of their time waiting for data to arrive from memory. It describes processing-in-memory as doing work on memory chips, dataflow designs as reusing values within a processing grid, and neuromorphic chips as keeping stored information close to where it is used. The author notes trade-offs: slower processing-in-memory cores, dataflow's difficulty with irregular tasks, and limited software tools for neuromorphic systems.

    Combined views

    —

    1 Source, first seen 1h ago

    — likes— comments— saves— reposts

    Combined views

    —

    1 Source, first seen 1h ago

    — likes— comments— saves— reposts
    Image: @BrianRoemmele on X

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Featured Source

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    1 Source

    Brian Roemmele@BrianRoemmeleToday CPUs spend most of thier existance waiting. The fastest processors idle for most of their cycles because they cannot pull data from memory as quickly as they can compute. The gap is structural. A conventional machine keeps arithmetic units and storage on opposite sides of a narrow bus, so every calculation requires a round trip that the processor must wait out. Processing-in-memory moves the arithmetic units onto the memory chips themselves. When a database updates millions of rows or an image filter touches every pixel, the work happens in place across the array. The data never leaves the module, so the transfer delay disappears. Commercial prototypes such as UPMEM place simple processors next to DRAM arrays and demonstrate gains on memory-bound kernels that conventional CPUs and GPUs cannot match once the data volume exceeds what fits in cache. Dataflow designs take a different cut at the same problem. Instead of fetching an instruction, loading a value, computing, and writing the result back, they stream data through a grid of simple processing nodes. A value loaded once can pass through several successive operations before it is stored again. The pattern suits the dense matrix and tensor work that dominates modern machine-learning training; the memory traffic drops because intermediate results stay inside the fabric. Neuromorphic chips go further and abandon the separation entirely. In a biological brain a synapse both stores a weight and performs the local computation that uses it. Intel’s Loihi family builds an equivalent circuit: asynchronous spiking cores hold their own synaptic state and exchange spikes directly with neighboring cores. On large systems such as Hala Point, the integrated memory-and-compute fabric reaches bandwidths measured in petabytes per second while drawing a few kilowatts for more than a billion neurons. Early results on sparse optimization and certain inference tasks show energy reductions of roughly two orders of magnitude relative to conventional accelerators on the same workloads. Each approach still carries practical limits. Processing-in-memory cores run at lower clock rates than a modern CPU and require software that can be partitioned across thousands of modest units. Dataflow fabrics excel at regular, streaming arithmetic and lose their advantage on irregular control flow. Neuromorphic systems remain research platforms whose programming models and toolchains are narrower than those of CPUs or GPUs. The shared premise is that the bus itself is the constraint; once computation and storage share the same silicon, the waiting time that dominates today’s machines becomes optional.1h
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI

    1 Source

    Brian Roemmele@BrianRoemmeleToday CPUs spend most of thier existance waiting. The fastest processors idle for most of their cycles because they cannot pull data from memory as quickly as they can compute. The gap is structural. A conventional machine keeps arithmetic units and storage on opposite sides of a narrow bus, so every calculation requires a round trip that the processor must wait out. Processing-in-memory moves the arithmetic units onto the memory chips themselves. When a database updates millions of rows or an image filter touches every pixel, the work happens in place across the array. The data never leaves the module, so the transfer delay disappears. Commercial prototypes such as UPMEM place simple processors next to DRAM arrays and demonstrate gains on memory-bound kernels that conventional CPUs and GPUs cannot match once the data volume exceeds what fits in cache. Dataflow designs take a different cut at the same problem. Instead of fetching an instruction, loading a value, computing, and writing the result back, they stream data through a grid of simple processing nodes. A value loaded once can pass through several successive operations before it is stored again. The pattern suits the dense matrix and tensor work that dominates modern machine-learning training; the memory traffic drops because intermediate results stay inside the fabric. Neuromorphic chips go further and abandon the separation entirely. In a biological brain a synapse both stores a weight and performs the local computation that uses it. Intel’s Loihi family builds an equivalent circuit: asynchronous spiking cores hold their own synaptic state and exchange spikes directly with neighboring cores. On large systems such as Hala Point, the integrated memory-and-compute fabric reaches bandwidths measured in petabytes per second while drawing a few kilowatts for more than a billion neurons. Early results on sparse optimization and certain inference tasks show energy reductions of roughly two orders of magnitude relative to conventional accelerators on the same workloads. Each approach still carries practical limits. Processing-in-memory cores run at lower clock rates than a modern CPU and require software that can be partitioned across thousands of modest units. Dataflow fabrics excel at regular, streaming arithmetic and lose their advantage on irregular control flow. Neuromorphic systems remain research platforms whose programming models and toolchains are narrower than those of CPUs or GPUs. The shared premise is that the bus itself is the constraint; once computation and storage share the same silicon, the waiting time that dominates today’s machines becomes optional.1h
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet