Skip to main navigation Skip to search Skip to main content

IBB: Improved K-resource aware backfill balanced scheduling for HTCondor

  • Beihang University

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

HTCondor, a batch system characterized by its matchmaking mechanism, schedules job in FCFS way, so its performance is not ideal as expected. Backfilling is a technique to address the above problem. Most backfilling algorithms are based on CPU information and have large room for improvements with considering other resource information. The K-resource aware scheduling algorithm Backfill Balanced (BB) selects backfill job which can best balance the usage of all resources and achieve better performance compared with the classical backfilling algorithm. However, BB does not realize that small jobs’ impacts on resource utilization are negligible and they mainly contribute to reduce the average response time. Here we propose the IBB algorithm, which utilizes the characteristics of small jobs to guide a better job selection. We implemented IBB on HTCondor to improve its performance. Experiments results show that IBB can provide up to 60% performance gains in most performance metrics compared with BB.

Original languageEnglish
Title of host publicationNetwork and Parallel Computing - 13th IFIP WG 10.3 International Conference, NPC 2016, Proceedings
EditorsXinbo Gao, Barbara Chapman, Depei Qian, Wenguang Chen, Guang R. Gao
PublisherSpringer Verlag
Pages85-92
Number of pages8
ISBN (Print)9783319470986
DOIs
StatePublished - 2016
Event13th IFIP WG 10.3 International Conference on Network and Parallel Computing, NPC 2016 - Xi’an, China
Duration: 28 Oct 201629 Oct 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9966 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference13th IFIP WG 10.3 International Conference on Network and Parallel Computing, NPC 2016
Country/TerritoryChina
CityXi’an
Period28/10/1629/10/16

Fingerprint

Dive into the research topics of 'IBB: Improved K-resource aware backfill balanced scheduling for HTCondor'. Together they form a unique fingerprint.

Cite this