TY - GEN
T1 - Performance optimization of small file I/O with adaptive migration strategy in cluster file system
AU - Li, Xiuqiao
AU - Dong, Bin
AU - Xiao, Limin
AU - Ruan, Li
PY - 2010
Y1 - 2010
N2 - While cluster file systems exploit data striping scheme to boost large file I/O throughput, small file performance is impaired and neglected. Common metadata-based optimizations introduce obstacles such as metadata server overload and migration latency. In this paper, a novel adaptive migration strategy is incorporated into metadata-based optimization to alleviate these side effects by migrating file dynamically. Guided by proposed adaptive migration threshold model, two types of file migration are applied to reduce metadata server load without degrading current performance of file system obviously. Schemes of latency hiding and migration consistency are also introduced to reduce overhead induced by small file optimization. Our results indicate that proposed optimization can substantially improve file creation and deletion performance, and boost small file I/O throughput by more than 20%. Moreover, side effects on overall performance produced by file migration are slight and can be absorbed by improvements.
AB - While cluster file systems exploit data striping scheme to boost large file I/O throughput, small file performance is impaired and neglected. Common metadata-based optimizations introduce obstacles such as metadata server overload and migration latency. In this paper, a novel adaptive migration strategy is incorporated into metadata-based optimization to alleviate these side effects by migrating file dynamically. Guided by proposed adaptive migration threshold model, two types of file migration are applied to reduce metadata server load without degrading current performance of file system obviously. Schemes of latency hiding and migration consistency are also introduced to reduce overhead induced by small file optimization. Our results indicate that proposed optimization can substantially improve file creation and deletion performance, and boost small file I/O throughput by more than 20%. Moreover, side effects on overall performance produced by file migration are slight and can be absorbed by improvements.
UR - https://www.scopus.com/pages/publications/77951567768
U2 - 10.1007/978-3-642-11842-5_33
DO - 10.1007/978-3-642-11842-5_33
M3 - 会议稿件
AN - SCOPUS:77951567768
SN - 3642118410
SN - 9783642118418
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 242
EP - 249
BT - High Performance Computing and Applications - Second International Conference, HPCA 2009, Revised Selected Papers
T2 - 2nd International Conference on High-Performance Computing and Applications, HPCA 2009
Y2 - 10 August 2009 through 12 August 2009
ER -