跳到主要导航 跳到搜索 跳到主要内容

HPD: Hybrid Projection Decomposition for Robust State Space Models on Analog CIM Hardware

  • Yuannuo Feng*
  • , Wenyong Zhou
  • , Yuexi Lyu
  • , Hanjie Liu
  • , Zhengwu Liu
  • , Ngai Wong
  • , Wang Kang
  • *此作品的通讯作者
  • Beihang University
  • Hicun Research Lab
  • The University of Hong Kong

科研成果: 书/报告/会议事项章节会议稿件同行评审

摘要

State Space Models (SSMs) are efficient alternatives to traditional sequence models, excelling at processing long sequences with lower computational complexity. Their reliance on matrix multiplications makes them ideal for compute-in-memory (CIM) architectures, which improve energy efficiency by computing within memory arrays. However, device non-idealities in CIM introduce weight perturbations that can degrade inference accuracy. In this paper, we systematically analyze the robustness of SSMs under noisy conditions, identifying that the final block and output projection layers are more susceptible to perturbations compared to other components. Building on these insights, we propose HPD, a Hybrid Projection Decomposition strategy for the last output projection layer. We replace the original weight matrix with the multiplication of U and Σ in its SVD to ensure compatibility with existing hardware architectures, while offloading V to digital hardware for precise and robust correction. Comprehensive tests on Mamba models show that our method reduces perplexity by up to 99.57% under various noise conditions compared to baseline models, with accuracy gains of up to 96.67% on the PIQA benchmark for commonsense reasoning.

源语言英语
主期刊名2025 IEEE 16th International Conference on ASIC, ASICON 2025
出版商IEEE Computer Society
ISBN(电子版)9798331539177
DOI
出版状态已出版 - 2025
活动2025 IEEE 16th International Conference on ASIC, ASICON 2025 - Kunming, 中国
期限: 21 10月 202524 10月 2025

出版系列

姓名Proceedings of International Conference on ASIC
ISSN(印刷版)2162-7541
ISSN(电子版)2162-755X

会议

会议2025 IEEE 16th International Conference on ASIC, ASICON 2025
国家/地区中国
Kunming
时期21/10/2524/10/25

联合国可持续发展目标

此成果有助于实现下列可持续发展目标:

  1. 可持续发展目标 7 - 经济适用的清洁能源
    可持续发展目标 7 经济适用的清洁能源

指纹

探究 'HPD: Hybrid Projection Decomposition for Robust State Space Models on Analog CIM Hardware' 的科研主题。它们共同构成独一无二的指纹。

引用此