Robust to noise, fragile to faults: what stuck weights mean for at-array decoders

What the work claims

Dangarikar and Merkel claim to be the first to study three reliability demands on a network at once: compression by magnitude pruning, robustness to adversarially perturbed inputs, and tolerance of stuck-at-zero hardware weight faults. Their headline results run against intuition in both directions. Training for adversarial robustness, the standard defense against malicious or noisy inputs, makes the network more sensitive to stuck weights. And pruning, which one would expect to concentrate information into fewer, more fragile weights, has no clear effect on fault sensitivity at all.¹

This is a simulation study on a deliberately small system: a three-layer convolutional network on MNIST, with faults injected in software. It is the cleanest possible experimental sandbox, which is both its strength (clean isolation of effects) and its limit (nothing here ran on real neuromorphic silicon).

How it works

The network is three convolutional layers with 32, 64, and 64 filters followed by a ten-unit fully connected layer, trained on MNIST with RMSProp. Three model variants are built: naturally trained; adversarially trained with projected gradient descent adversaries (step size 0.01, 40 iterations, perturbation bound 0.3, clean and adversarial losses averaged); and adversarially trained plus magnitude pruning at 20, 40, 60, or 80 percent of weights, globally, with retraining after pruning. Robustness is then stress-tested two ways: FGSM and PGD attacks at perturbation magnitudes from 0 to 0.5, and stuck-at-zero fault injection that progressively zeroes 10 percent of the remaining active weights per step, up to 80 percent total faults. Every configuration is run five times with different random initializations.¹

The results table tells the story. Fault-free, everything is stable: clean accuracy sits near 98.3 percent at every pruning level, and the adversarially trained model holds above 89 percent accuracy under both FGSM and PGD at the training perturbation bound. Under 40 percent stuck weights, the unpruned adversarially trained model collapses to a 70.41 percent clean accuracy with a standard deviation of 31.10 points across runs, while the 40 percent pruned variant retains 93.60 plus or minus 1.36 percent. At 80 percent faults, every configuration is at chance level near 11 to 13 percent. The natural-versus-adversarial comparison shows the two training regimes diverging noticeably once roughly 20 percent of weights are stuck at zero.¹

The proposed explanation is geometric. Adversarial training pushes decision boundaries away from the training data, which is why larger input perturbations are needed to cross them. Stuck-at-zero faults distort the weights and pull those carefully placed boundaries back toward the data, so smaller perturbations now cause misclassifications. The authors are careful to label this consistent with, not proof of, that mechanism.

Where a skeptic should push

The load-bearing weakness is scale, in every sense. MNIST is a ten-class digit task; the network has three convolutional layers; the faults are injected into a floating-point software model, which the authors themselves note cannot capture analog noise or device-specific fault patterns of real neuromorphic hardware. Stuck-at-one faults, plausibly more damaging, are left entirely to future work. And the elephant in the results table is the variance: at intermediate fault rates of 20 to 40 percent, standard deviations across the five runs exceed 30 percentage points, and the authors concede their own means may not support strong significance claims there. Five runs is a small sample for surfaces this noisy. The apparent advantage of 40 percent pruning at 40 percent faults (93.60 plus or minus 1.36 versus 70.41 plus or minus 31.10 unpruned) is dramatic precisely where the unpruned variance explodes, which invites caution about reading a reproducible effect into one row.

Also note what the 80 percent fault rows actually tell you: at that damage level everything is dead, so the useful engineering signal lives between 20 and 50 percent faults, which is a far higher fault rate than any sane fabrication process ships. The result is a directional warning about a trade-off, not a specification you can design tolerances from.

Fault models for compute next to the array

Why does an MNIST fault-injection study belong on a microelectrode array site? Because the endpoint of MEA instrumentation is increasingly to close the loop: decode spike streams on or near the array and drive stimulation back, within a latency budget, on hardware that lives inches from living tissue. That compute is exactly the kind this paper models. Memristive crossbars and analog in-memory fabrics, the natural candidates for embedding weight matrices next to a high-density array, suffer permanent stuck defects as a routine failure mode, and the paper itself cites stuck-at faults as common in memristive hardware. A stuck device is a stuck weight. The sensor side has the same physics in a different costume: in any large electrode array, a subset of channels is dead or degraded on any given day, and the system is expected to tolerate it silently.

The non-obvious implication cuts against a habit the field is quietly developing. As MEA decoders are trained to be robust to biological variability, drifting baselines, and prep-to-prep differences, that training is the noise-robustness analog of adversarial training. This paper's central finding suggests that pushing robustness in that direction can quietly spend down the network's tolerance for permanent hardware faults. A decoder that shrugs off electrode drift because its decision boundaries were pushed away from the training data may be precisely the decoder whose boundaries a handful of stuck crossbar devices can pull back across the data. Two robustness budgets, one weight budget, and no free lunch between them.

The pruning result is the good news, and it matters because at-array compute is resource-starved: a decoder that must fit beside a high-density front end will be compressed by necessity, and until now one could reasonably fear that compression multiplies fragility. It apparently does not, at least in this sandbox. The honest caveat is that the same noisy data that produced that reassurance limits its strength, so treat compression as neutral to fault tolerance, not as proven-safe.

The actionable lesson is procedural rather than numerical: evaluate MEA decoders jointly. A decoder qualified only against noise, drift, and biological variability, and separately only against quantization and compression, has not been qualified against the combination. If it will ever be mapped onto analog fabric with real stuck devices, fault injection belongs in the training and test loop from the start, and the decision-boundary geometry the authors propose gives a concrete reason why bolt-on fixes after the fact may fail.

The bottom line

Established, within a toy but well-controlled sandbox: adversarial training and stuck-at-zero fault tolerance trade off against each other, pruning level had little effect, and at 80 percent stuck weights nothing survives. Hypothesis: the trade-off transfers to noise-robust MEA decoders mapped onto imperfect analog hardware, where it would be worse, because real stuck-device faults are spatially and statistically structured, not random. What would confirm it is the same experiment run on a real memristive or mixed-signal fabric with a spike-stream decoding task and realistic one to five percent fault rates. What would soften it is evidence that fault-aware training recovers both robustness budgets simultaneously. Until then: train for noise robustness if you must, but test for stuck weights before you trust the loop.