Performance-oriented workload management for multiclusters and grids

This chapter addresses the dynamic scheduling of parallel jobs with QoS demands (soft-deadlines) in multiclusters and grids. Three performance metrics (over-deadline, makespan and idle-time) are combined with variable weights to evaluate the scheduling performance. These three metrics are used for measuring the extent of jobs’ QoS demands compliance, resource throughput and resource utilization, respectively. Therefore, clusters situated in different administrative organizations can utilize different weight combinations to represent their different performance requirements. Two levels of performance optimisations are applied in the multicluster. At the multicluster level, a scheduler, (which we call MUSCLE), allocates parallel jobs with high packing potential to the same cluster; MUSCLE also takes the jobs’ QoS requirements into account and employs a heuristic to allocate suitable workloads to each cluster to balance the performance. At the local cluster level, a workload manager, called TITAN, utilizes a genetic algorithm to further improve the scheduling performance of the jobs sent by MUSCLE. The extensive experimental studies are conducted to verify the effectiveness of the scheduling mechanism in MUSCLE; the results show that comparing with the traditional workload allocation policies in distributed systems (Dynamic Least Load and Weighted Random), the comprehensive scheduling performance (in terms of overdeadline, makespan and idle-time) of parallel jobs is significantly improved and well balanced across the multicluster.