Noise-Aware Adaptive Sampling for Robust Diffusion Models on Analog Compute-in-Memory

Diffusion models achieve state-of-the-art image generation but impose heavy computational burdens on digital computers. Compute-in-memory (CIM) architectures offer promising acceleration, but inherent noise causes severe performance degradation through weight perturbations. We find that reducing sampling steps improves robustness but limits generation versatility, and that noise at earlier steps causes more severe degradation due to error accumulation. Based on these insights, we propose EtaMix, a novel noise-aware sampling strategy that interpolates between stochastic and deterministic sampling without requiring training or hardware modifications. EtaMix applies more stochastic sampling initially to offset weight perturbations, then gradually transitions to deterministic sampling. Experimental results show EtaMix achieves up to 2.01× and 5.12× FID improvements under different noise conditions for DDPM and DDIM, respectively.