InfiNAS: Architecture Search of Infinite-Dimensional Neural Networks

As AI enters the era of large foundation models, neural network architecture design remains a critical research focus. Recent studies emphasize high-order computation and infinite-dimensional feature interactions, introducing new computation-memory patterns and challenges for neural architecture search (NAS). In this work, we analyze the search space of high-order neural networks under infinite-dimensional design, investigating how scaling factors such as width, depth, rank, and order affect performance and efficiency. We then propose InfiNAS, a NAS framework that automatically discovers effective high-order architectures. Experiments demonstrate that InfiNAS achieves superior accuracy and the best computation-memory trade-off compared to state-of-the-art networks, reducing memory access by up to 69% while maintaining high performance.