Skip to main navigation Skip to search Skip to main content

QoSFM: QoS support for metadata I/O in parallel file systems

  • Limin Xiao
  • , Ke Xie
  • , Guoying Li
  • , Li Ruan*
  • , Xiuqiao Li
  • *Corresponding author for this work
  • Beihang University

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Large parallel file systems (PFSs) must service multiple independent workloads concurrently. Workloads often require different quality of service (QoS) of metadata I/O. However, existing PFSs are unable to differentiate metadata I/O from different workloads. There are many researches on QoS support of storage system, they do not focus on metadata I/O of PFSs as one metadata I/O can have several sub-operations and different sub-operations have different access size. In addition, users cannot specify metadata QoS requirements without a suitable QoS specification. This paper presents a novel scheme QoSFM (QoS Support for Metadata) to support metadata I/O QoS in PFSs. QoSFM allows users to specify workloads' metadata QoS requirements through our two-layered QoS specification mechanism. QoSFM can dynamically translate the workload-level goals into system resource needs such as service time per time window and sub-operation latency. Client-side controllers throttle the metadata I/O based on service time allocation to meet the throughput goal while metadata servers (MDSs) reorder the sub-operations based on deadlines to meet the latency goal. Extensive simulations show that QoSFM can effectively satisfy workload-level metadata I/O QoS requirements. Meanwhile, results also show that QoSFM has low overhead.

Original languageEnglish
Title of host publicationProceedings - 2013 IEEE International Conference on High Performance Computing and Communications, HPCC 2013 and 2013 IEEE International Conference on Embedded and Ubiquitous Computing, EUC 2013
PublisherIEEE Computer Society
Pages29-38
Number of pages10
ISBN (Print)9780769550886
DOIs
StatePublished - 2014
Event15th IEEE International Conference on High Performance Computing and Communications, HPCC 2013 and 11th IEEE/IFIP International Conference on Embedded and Ubiquitous Computing, EUC 2013 - Zhangjiajie, Hunan, China
Duration: 13 Nov 201315 Nov 2013

Publication series

NameProceedings - 2013 IEEE International Conference on High Performance Computing and Communications, HPCC 2013 and 2013 IEEE International Conference on Embedded and Ubiquitous Computing, EUC 2013

Conference

Conference15th IEEE International Conference on High Performance Computing and Communications, HPCC 2013 and 11th IEEE/IFIP International Conference on Embedded and Ubiquitous Computing, EUC 2013
Country/TerritoryChina
CityZhangjiajie, Hunan
Period13/11/1315/11/13

Keywords

  • Metadata I/O
  • Parallel file system
  • QoS
  • Resource allocation

Fingerprint

Dive into the research topics of 'QoSFM: QoS support for metadata I/O in parallel file systems'. Together they form a unique fingerprint.

Cite this