Skip to main navigation Skip to search Skip to main content

HPD: Hybrid Projection Decomposition for Robust State Space Models on Analog CIM Hardware

  • Yuannuo Feng*
  • , Wenyong Zhou
  • , Yuexi Lyu
  • , Hanjie Liu
  • , Zhengwu Liu
  • , Ngai Wong
  • , Wang Kang
  • *Corresponding author for this work
  • Beihang University
  • Hicun Research Lab
  • The University of Hong Kong

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

State Space Models (SSMs) are efficient alternatives to traditional sequence models, excelling at processing long sequences with lower computational complexity. Their reliance on matrix multiplications makes them ideal for compute-in-memory (CIM) architectures, which improve energy efficiency by computing within memory arrays. However, device non-idealities in CIM introduce weight perturbations that can degrade inference accuracy. In this paper, we systematically analyze the robustness of SSMs under noisy conditions, identifying that the final block and output projection layers are more susceptible to perturbations compared to other components. Building on these insights, we propose HPD, a Hybrid Projection Decomposition strategy for the last output projection layer. We replace the original weight matrix with the multiplication of U and Σ in its SVD to ensure compatibility with existing hardware architectures, while offloading V to digital hardware for precise and robust correction. Comprehensive tests on Mamba models show that our method reduces perplexity by up to 99.57% under various noise conditions compared to baseline models, with accuracy gains of up to 96.67% on the PIQA benchmark for commonsense reasoning.

Original languageEnglish
Title of host publication2025 IEEE 16th International Conference on ASIC, ASICON 2025
PublisherIEEE Computer Society
ISBN (Electronic)9798331539177
DOIs
StatePublished - 2025
Event2025 IEEE 16th International Conference on ASIC, ASICON 2025 - Kunming, China
Duration: 21 Oct 202524 Oct 2025

Publication series

NameProceedings of International Conference on ASIC
ISSN (Print)2162-7541
ISSN (Electronic)2162-755X

Conference

Conference2025 IEEE 16th International Conference on ASIC, ASICON 2025
Country/TerritoryChina
CityKunming
Period21/10/2524/10/25

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

  1. SDG 7 - Affordable and Clean Energy
    SDG 7 Affordable and Clean Energy

Keywords

  • Compute-in-Memory
  • State Space Models

Fingerprint

Dive into the research topics of 'HPD: Hybrid Projection Decomposition for Robust State Space Models on Analog CIM Hardware'. Together they form a unique fingerprint.

Cite this