The Dreaming Fix

The classical Hopfield model suffers from catastrophic forgetting: store too many patterns and the network fails to retrieve any of them. The transition is sharp. Below the capacity threshold, memory works. Above it, everything collapses at once.

Introducing synaptic clipping — bounding how strong any synapse can become — eliminates the catastrophe. The clipped network forgets gracefully: older patterns fade as newer ones are stored. No cliff. But the cost is severe: memorization capacity drops dramatically compared to the unclipped model.

Feinstein, Hopfield, and Palmer proposed a solution inspired by dreaming: between learning phases, the network generates random patterns and unlearns them. The new analysis shows this works. Alternating learning and dreaming improves the clipped network’s capacity while preserving its graceful forgetting. The dreaming doesn’t add information — the random patterns are noise. What it does is sculpt the synaptic landscape: unlearning random configurations pushes the network away from spurious attractors, clearing space for real memories.

The structural insight: the problem with bounded synapses isn’t the bound itself but the interference pattern the bound creates. Every stored pattern partially overwrites previous ones in a clipped system. Dreaming acts as a form of interference management — not by protecting stored patterns but by degrading the accidental structure that noise creates. The random patterns during dreaming are like a controlled burn: destroy the undergrowth so the forest can grow.

What makes the result biologically interesting is its evolutionary plausibility. Finding the optimal unclipped Hopfield capacity requires fine-tuning that natural selection can’t easily achieve. The clipped-plus-dreaming system reaches near-optimal performance through a simple alternation that’s robust to parameter variation. The path to good memory goes through planned forgetting — not as a bug but as the mechanism that makes bounded resources sufficient.