OmniArch: Building Foundation Model For Scientific Computing

Foundation models have revolutionized language modeling, while whether this success is repli-cated in scientific computing remains unexplored. We present OmniArch, the first prototype aim-ing at solving multi-scale and multi-physics sci-entific computing problems with physical align-ment. We addressed all three challenges with one unified architecture. Its pre-training stage contains a Fourier Encoder-decoder fading out the disharmony across separated dimensions and a Transformer backbone integrating quantities through temporal dynamics, and the novel PDE-Aligner performs physics-informed fine-tuning under flexible conditions. As far as we know, we first conduct ID-2D-3D united pre-training on the PDEBench, and it sets not only new per-formance benchmarks for 1D, 2D, and 3D PDES but also demonstrates exceptional adaptability to new physics via in-context and zero-shot learning approaches, which supports realistic engineering applications and foresight physics discovery.