Dynamic scheduling of parallel jobs with QoS demands in multiclusters and grids

Ligang He*, Stephen A. Jarvis, Daniel P. Spooner, Xinuo Chen, Graham R. Nudd

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

47 Citations (Scopus)

Abstract

This paper addresses the dynamic scheduling of parallel jobs with QoS demands (soft-deadlines) in multiclusters and grids. Three metrics (over-deadline, makespan and idle-time) are combined with variable weights to evaluate the scheduling performance. These three metrics are used to measure the extent of the jobs' QoS demand compliance, the resource throughput and the resource utilization. Two levels of performance optimisation are applied in the multicluster. At the multicluster level, a scheduler (which we call MUSCLE) allocates parallel jobs with high packing potential to the same cluster; it also takes the jobs' QoS requirements into account and employs a heuristic to allocate suitable workloads to each cluster to balance the overall system performance. At the single cluster level, an existing workload manager, called TITAN, utilizes a genetic algorithm to further improve the scheduling performance of the jobs previously allocated by MUSCLE. Extensive experimental studies are conducted to verify the effectiveness of the scheduling mechanism as well as the effect of the prediction accuracy on the scheduling performance. The results show that compared with traditional distributed workload allocation policies, the comprehensive scheduling performance of parallel jobs is significantly improved across the multicluster, and the presence of prediction errors does not dramatically weaken the performance advantage.

Original languageEnglish
Title of host publicationProceedings - Fifth IEEE/ACM International Workshop on Grid Computing
EditorsR. Buyya
Pages402-409
Number of pages8
DOIs
Publication statusPublished - 2004
EventProceedings - Fifth IEEE/ACM International Workshop on Grid Computing - Pittsburgh, PA, United States
Duration: 8 Nov 20048 Nov 2004

Publication series

NameProceedings - IEEE/ACM International Workshop on Grid Computing
ISSN (Print)1550-5510

Conference

ConferenceProceedings - Fifth IEEE/ACM International Workshop on Grid Computing
Country/TerritoryUnited States
CityPittsburgh, PA
Period8/11/048/11/04

ASJC Scopus subject areas

  • General Engineering

Fingerprint

Dive into the research topics of 'Dynamic scheduling of parallel jobs with QoS demands in multiclusters and grids'. Together they form a unique fingerprint.

Cite this