Anti-Bandit for Neural Architecture Search

Neural Architecture Search (NAS) is a highly challenging task that requires consideration of search space, search efficiency, and adversarial robustness of the network. In this paper, to accelerate the training speed, we reformulate NAS as a multi-armed bandit problem and present Anti-Bandit NAS (ABanditNAS) method, which exploits Upper Confidence Bounds (UCB) to abandon arms for search efficiency and Lower Confidence Bounds (LCB) for fair competition between arms. Based on the presented ABanditNAS, the adversarially robust optimization and architecture search can be solved in a unified framework. Specifically, our proposed framework defends against adversarial attacks based on a comprehensive search of denoising blocks, weight-free operations, Gabor filters, and convolutions. The theoretical analysis on the rationality of the two confidence bounds in ABanditNAS are provided and extensive experiments on three benchmarks are conducted. The results demonstrate that the presented ABanditNAS achieves competitive accuracy at a reduced search cost compared to prior methods.