Neural Cellular Automata for
Mel-Spectrogram Enhancement

ML for Audio Workshop · ICML 2026 Anonymous Submission Code

Can local, cell-by-cell computation denoise speech? We show that a 9k-parameter Neural Cellular Automaton — where each mel-spectrogram cell updates by looking only at its neighbours — produces globally coherent enhancement with anytime quality control.

AAbstract

We investigate Neural Cellular Automata (NCA) for speech enhancement in the mel-spectrogram domain — a proof of concept for local iterative computation and anytime inference in audio. An NCA treats each time-frequency cell as an independent agent that updates its state by perceiving only its local neighbourhood via fixed Sobel filters, producing globally coherent denoising through iterative application of a single shared MLP. We introduce a conditioned NCA variant that receives the noisy mel as a read-only reference at every step, preventing state drift, and a 1D column NCA in which each cell is an entire mel column. We further show that supervising every intermediate step rather than only the final output improves PESQ by +0.053 at equal 50k budget. Trained on VoiceBank+DEMAND, our 9k-parameter model achieves PESQ 2.437 and STOI 0.926, with a matched-horizon ablation checkpoint reaching PESQ 2.447 — outperforming the similarly-sized U-Net (12k params) in PESQ and STOI while trailing slightly in SI-SNR, and not significantly different in PESQ from one 5× larger (paired bootstrap, p = 0.09).
9k params · PESQ 2.437–2.447 +0.055 vs. 12k U-Net in PESQ T = 8 – 64 anytime VoiceBank+DEMAND

⚙How NCA Works

Each step of the NCA is a local message-passing operation applied in parallel to every cell of the mel spectrogram.

👁️

Perceive

Fixed Sobel-x, Sobel-y and identity kernels extract frequency and time gradients from the cell's neighbourhood. The noisy mel is also read at every step as a stable reference.

🧠

Update

A shared 2-layer MLP maps the perception vector to a state delta. A stochastic mask (p=0.5) randomly silences cells each step, preventing synchronisation artifacts.

🔁

Iterate

After T steps, the visible state channel is the enhanced mel. T is chosen freely at test time — more steps give higher quality up to the training horizon.

⚡

Anytime

All-steps supervision trains every intermediate state to be a valid estimate. This directly bakes the anytime property into the model — stop early for speed, run longer for quality.

📊Results

Quality (PESQ) as a function of inference steps and GPU latency. The NCA curve is anytime — each point is the same model stopped at a different step.

Quality vs steps and latency

▶NCA Denoising Process

Watch the NCA evolve step-by-step. Left: noisy input (static reference). Centre: NCA output at the current step. Right: absolute error from the clean reference — watch it shrink as the NCA converges.

♪Audio Examples

NCA at T=8 (fast), T=32 and T=64 (best quality) demonstrate anytime inference. All reconstructed via ISTFT with the noisy-input phase.

⚠Failure Cases

These two utterances were already near-clean before enhancement (noisy PESQ ≈ 4.0–4.2).