ReLU-based nonlinear tensor decomposition for sparse nonnegative data

Using low-rank approximation to find intrinsic structures in tensor data is a standard approach. However, sparse nonnegative tensors present a challenge, as their sparse structures do not directly fit the low-rank assumption required by these standard methods. Therefore, we propose a ReLU-based nonlinear tensor decomposition (ReLU-NTD) model. This approach decomposes a sparse nonnegative tensor M into the form M≈max(0,X), where X is a low-rank latent tensor. We develop an alternating minimization framework, and incorporate QR decomposition with the dimension tree (DT) strategy to enhance numerical stability and scalability. Our theoretical analysis establishes convergence, demonstrating that using DT to reduce QR overhead yields a lower overall complexity compared to standard QR decomposition. Extensive experiments on both synthetic and real datasets show that the proposed ReLU-NTD achieves lower reconstruction errors and more compact representations than conventional low-rank models, providing a robust tool for analyzing sparse nonnegative tensor data.