TY - GEN
T1 - Restricted Boltzmann Machines and Deep Belief Networks on Sunway Cluster
AU - Song, Kaida
AU - Liu, Yi
AU - Wang, Rui
AU - Zhao, Meiting
AU - Hao, Ziyu
AU - Qian, Depei
N1 - Publisher Copyright:
© 2016 IEEE.
PY - 2017/1/20
Y1 - 2017/1/20
N2 - Deep learning models have showed great potential in classification and recognition over the last decade. Deep Belief Networks (DBNs) have been applied in visual, voice fields due to their great feature presentation capability. However, there are a vast number of time consuming calculations in the training of DBNs. Many researches have accelerated the training of DBNs with good speedups on CPU, GPU, FPGA, etc. At the same time, the latest published Sunway(SW) many-core processor has high computing performance and dedicated heterogeneous architecture. This paper provides a DBNs training system on SW cluster and verifies SW cluster's applicability of training DBNs. We firstly optimize the Restricted Boltzmann Machines and Deep Belief Networks on Sunway processor, then build a parallelism model with linear topology to train DBNs on multiple processors. The system is implemented on the TaihuLight supercomputer and evaluated by training a DBN with 2.8 million parameters with MNIST dataset. Experimental results show that our system achieves considerable speedups on Sunway processors as compared with CPUs.
AB - Deep learning models have showed great potential in classification and recognition over the last decade. Deep Belief Networks (DBNs) have been applied in visual, voice fields due to their great feature presentation capability. However, there are a vast number of time consuming calculations in the training of DBNs. Many researches have accelerated the training of DBNs with good speedups on CPU, GPU, FPGA, etc. At the same time, the latest published Sunway(SW) many-core processor has high computing performance and dedicated heterogeneous architecture. This paper provides a DBNs training system on SW cluster and verifies SW cluster's applicability of training DBNs. We firstly optimize the Restricted Boltzmann Machines and Deep Belief Networks on Sunway processor, then build a parallelism model with linear topology to train DBNs on multiple processors. The system is implemented on the TaihuLight supercomputer and evaluated by training a DBN with 2.8 million parameters with MNIST dataset. Experimental results show that our system achieves considerable speedups on Sunway processors as compared with CPUs.
KW - Data parallelism model
KW - Deep Belief Networks
KW - Restricted Boltzmann Machines
KW - Sunway processor
KW - TaihuLight supercomputer
UR - https://www.scopus.com/pages/publications/85013628842
U2 - 10.1109/HPCC-SmartCity-DSS.2016.0044
DO - 10.1109/HPCC-SmartCity-DSS.2016.0044
M3 - 会议稿件
AN - SCOPUS:85013628842
T3 - Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016
SP - 245
EP - 252
BT - Proceedings - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016
A2 - Yang, Laurence T.
A2 - Chen, Jinjun
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 18th IEEE International Conference on High Performance Computing and Communications, 14th IEEE International Conference on Smart City and 2nd IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2016
Y2 - 12 December 2016 through 14 December 2016
ER -