Cloud resource management has attracted substantial research attention over the past decade. Beloglazov et al. [12] proposed optimal online deterministic algorithms and adaptive heuristics for energy- and performance-efficient dynamic consolidation of VMs, establishing the Modified Best Fit Decreasing (MBFD) heuristic as a widely adopted baseline. Their work introduced the concept of upper and lower CPU utilization thresholds for triggering VM migration, a principle that remains foundational in energy-aware cloud management. Rodriguez and Buyya [7] addressed deadline-based resource provisioning and scheduling for scientific workflows on Infrastructure-as-a-Service clouds, formulating the problem as a particle-based optimization with deadline constraints. Lorido-Botran et al. [8] surveyed auto-scaling techniques in cloud environments, categorizing approaches into reactive (threshold-based), proactive (predictive), and hybrid methods. Farahnakian et al. [9] proposed energy-aware VM consolidation using utilization prediction models based on linear regression and k-nearest neighbors, demonstrating significant energy savings through proactive migration.

Multi-objective optimization for cloud resource management has received growing interest. Li et al. [25] applied NSGA-II to simultaneously optimize energy consumption and quality of service in VM placement, producing Pareto-optimal trade-off solutions. Ferdaus et al. [26] proposed a multi-objective VM placement algorithm based on ACO to minimize both resource wastage and power consumption. More recently, Zhou et al. [27] applied MOEA/D with decomposition for multi-objective workflow scheduling in cloud environments, optimizing makespan, cost, and reliability simultaneously. While these approaches discover effective trade-off surfaces, their computational overhead (typically requiring thousands of fitness evaluations per generation) limits their applicability to real-time decision-making in production cloud environments.

Nature-inspired metaheuristics have been extensively applied to cloud resource management sub-problems. Zhan et al. [13] applied a modified GA to cloud task scheduling, using precedence-preserving crossover and mutation operators to minimize makespan under resource constraints. Mejahed and Elshrkawey [14] developed a multi-objective PSO variant for VM allocation that balances energy consumption and resource utilization through adaptive inertia weight adjustment. Chen and Zhang [15] proposed an ACO-based approach for workflow scheduling in cloud environments, modeling the task-to-VM assignment as an ant routing problem with pheromone updates reflecting both time and cost objectives. DE has been applied to load balancing problems in cloud data centers, exploiting its effective mutation and crossover operations for continuous optimization [16], [28].

More recently, novel metaheuristics have been introduced for cloud applications. Sahoo, Sahoo, and Kumar [29] applied the Grey Wolf Optimizer (GWO) [30] to cloud task scheduling, demonstrating competitive performance with reduced parameter tuning compared to GA and PSO. The Whale Optimization Algorithm (WOA) [31] has been applied to energy-aware VM placement, mimicking humpback whale bubble-net attacking strategies. The Sine Cosine Algorithm (SCA) [32] and Harris Hawks Optimization (HHO) [33] have also been adapted for various cloud optimization tasks. Despite their individual strengths, all single metaheuristics share a fundamental limitation: they maintain a fixed exploration–exploitation balance determined by their algorithmic structure and parameter settings, which may not be optimal across different problem states and workload conditions [22]. Moreover, each metaheuristic requires problem-specific parameter tuning, and the No Free Lunch theorem [34] guarantees that no single metaheuristic can universally outperform all others across all problem instances.

RL-based approaches have gained momentum due to their ability to learn adaptive policies from interaction with the environment. Tuli et al. [19] proposed an RL-based framework for energy-efficient VM placement using DQN, training an agent to select placement actions that minimize energy consumption while respecting SLA constraints. Arabnejad et al. [20] developed an actor-critic RL approach for auto-scaling containerized cloud applications, learning policies that adjust resource allocation in response to predicted demand changes. Mao et al. [21] introduced a multi-resource cluster scheduling approach using deep RL, training a policy network to make scheduling decisions across multiple resource dimensions simultaneously.

Model-based RL approaches have also been explored for proactive scaling. Cheng et al. [35] used model-based RL with learned environment dynamics to predict future workload demands and pre-allocate resources. Multi-agent RL has been applied to distributed scheduling, where each scheduling agent learns to coordinate with others through shared reward signals [36]. Despite promising results, RL-based cloud management approaches face several challenges: 1) sample inefficiency: deep RL methods typically require millions of interactions to learn effective policies, 2) reward engineering: designing reward functions that balance multiple cloud objectives without unintended consequences is non-trivial, 3) the sim-to-real gap: policies learned in simulation may not transfer effectively to production environments, and 4) training instability: neural network-based RL can exhibit catastrophic forgetting and training divergence, particularly in non-stationary environments [37].

Hyper-heuristics provide a higher-level abstraction for optimization, searching within a space of heuristics rather than the solution space itself. Burke et al. [22] provided a comprehensive survey, taxonomizing hyper-heuristics into selection hyper-heuristics (which choose from a set of existing heuristics) and generation hyper-heuristics (which construct new heuristics from components). Selection mechanisms include simple random, random descent, roulette wheel, choice function, and various RL-inspired approaches. Cowling et al. [38] pioneered hyper-heuristic research with their choice function-based approach for scheduling, demonstrating that automated heuristic selection could outperform individual heuristics. Subsequent work has applied hyper-heuristics to diverse combinatorial optimization domains including bin packing [39], timetabling [40], and vehicle routing [41].

Cloud-specific hyper-heuristics have emerged as a growing research area. Zhan et al. [23] proposed VMM-HHGT, a hyper-heuristic approach for handling hierarchy in cloud data centers that uses generative and selection mechanisms for VM management across multi-tier data center architectures. Tan et al. [24] introduced SMPHAM, a self-adaptive microservice placement hyper-heuristic with adaptive mechanisms for dynamic cloud-edge environments. Drake et al. [42] applied a selection hyper-heuristic with a choice function to the cloud VM consolidation problem, selecting among several consolidation heuristics based on their recent performance. However, existing cloud hyper-heuristics predominantly employ relatively simple selection mechanisms: random selection, roulette wheel proportional to recent performance, or basic adaptive rules, without leveraging sophisticated RL techniques for learning state-dependent selection policies that dynamically balance exploration of underexplored heuristics and exploitation of known high-performing ones.

The WHH, proposed by Daliri et al. [1], represents a paradigmatic advance in hyper-heuristic methodology by integrating RL-based learning directly into the heuristic selection process. The World algorithm operates through two core mechanisms. In the Rewarding step, each LLH is evaluated based on the solution improvement it produces when applied to the current search state. The reward is computed as a function of the relative fitness improvement, normalized by the number of applications, providing a fair comparison across frequently and infrequently used heuristics. In the Selection step, the algorithm uses a learned RL policy, specifically, a Q-learning-based approach to select the next LLH to apply. The Q-values encode the expected future utility of applying a particular LLH in a given search state, enabling the algorithm to learn which heuristics are most effective under different conditions.

A distinguishing feature of the World algorithm is its infinite pool concept: rather than being limited to a fixed set of LLHs, the World dynamically generates new composite heuristics by combining operators from different base metaheuristics. This mechanism effectively provides an unbounded search space of possible strategies, from which the RL mechanism selects the most promising ones. Daliri et al. [1] evaluated the World on three categories of problems: 1) standard benchmark functions as artificial NP-hard problems, 2) discrete NP-hard problems, including the TSP with benchmark instances up to 10,000 real cities, where the World achieved a shortest path length of 4.33 × 105, and 3) continuous NP-hard engineering design problems including pressure vessel, welded beam, and tension/compression spring design. In all categories, the World consistently outperformed state-of-the-art algorithms, including standalone metaheuristics and existing hyper-heuristics.

Despite its demonstrated effectiveness, the original World algorithm was designed and evaluated exclusively for offline, static optimization problems, scenarios where the problem instance is fixed, and the algorithm iterates until convergence. It has not been adapted for online, dynamic optimization environments where problem characteristics change over time, decisions must be made in real time, and the solution landscape is non-stationary. Cloud resource management is a quintessential example of such a dynamic domain. This paper addresses this gap by extending and adapting the WHH for the cloud computing domain.

We formulate the cloud resource management problem as a multi-objective dynamic optimization problem. Consider a data center with the following components:

I. A set of N heterogeneous PMs, PM = {pm1, pm2, …, pmN}, where each pmj has capacities capcpu(j), capmem(j), capsto(j), and capbw(j) for CPU, memory, storage, and bandwidth, respectively.

II. A set of M VMs, VM = {vm1, vm2, …, vmM}, where each vmi has resource requirements reqcpu(i), reqmem(i), reqsto(i), and reqbw(i).

III. A set of K tasks T = {t1, t2, …, tK}, where each task tk has processing time pk, deadline dk, resource requirements, and priority level πk.

IV. A discrete time horizon divided into decision epochs τ = {1, 2, …, Tmax}.

The resource management problem is decomposed into three interdependent sub-problems:

P1 (VM placement). Find a mapping φ: VM → PM that assigns each VM to a PM, minimizing energy consumption:

fenergy = Σj = 1N Ej(uj) · active(j), (1)

where Ej(uj) is the power consumption model of PM j as a function of its CPU utilization uj, and active(j) ∈ {0, 1} indicates whether host j is powered on. Following the SPECpower benchmark model, we define:

Ej(uj) = Pidle(j) + (Pmax(j) − Pidle(j)) · uj, (2)

where Pidle(j) and Pmax(j) are the idle and maximum power consumption of host j, and the CPU utilization is computed as:

uj = Σvm ∈ φ − 1(j) reqcpu(vm) / capcpu(j). (3) P2 (Task scheduling). Find an assignment σ: T → VM that maps each task to a VM, minimizing the makespan:

fmakespan = maxi ∈ {1, … , K}Ci, (4)

where Ci is the completion time of task ti. The completion time depends on the task processing time, VM computational capacity, and queueing delays from other tasks assigned to the same VM.

P3 (Dynamic scaling). At each decision epoch τ, find scaling decisions s(τ) ∈ {scale-up, scale-down, maintain} for each resource tier, minimizing:

fSLA = Στ SLAviolations(τ) + λ · cost(s(τ)), (5)

where SLA_violations(τ) counts the number of SLA breaches at epoch τ, cost(s(τ)) captures the monetary cost of scaling actions, and λ is a trade-off parameter.

The combined multi-objective fitness function integrates all objectives:

F(x) = w1 · f̂energy + w2 · f̂SLA + w3 · (1 − f̂utilization) + w4 · f̂makespan + w5 · f̂cost, where f̂ denotes min-max normalized objective values, and w1, w2, w3, w4, w5 are adaptive weights satisfying Σi wi = 1. The utilization objective is negated because it is to be maximized while all others are minimized. The optimization is subject to the following constraints:

C1: Σvm ∈ φ − 1(pmj)reqcpu(vm) ≤ capcpu(pmj), ∀j ∈ {1, … , N}, (7) C2: Σvm ∈ φ − 1(pmj)reqmem(vm) ≤ capmem(pmj), ∀j ∈ {1, … , N}, (8) C3: Ck ≤ dk, ∀tk ∈ T with hard deadlines, (9) C4: responsetime(vmi) ≤ SLAthreshold(vmi), ∀i ∈ {1, … , M}, (10) C5: 0.2 ≤ uj ≤ 0.85, ∀j where active(j) = 1, (11)

Constraints C1 and C2 enforce CPU and memory capacity limits on each PM. Constraint C3 ensures hard deadline compliance for time-critical tasks. Constraint C4 enforces response time SLA thresholds. Constraint C5 maintains utilization within operational bounds to prevent both underutilization waste and hotspot-induced performance degradation.

World-CRM operates through a three-tier architecture that mirrors the hierarchical structure of cloud computing infrastructure, enabling coordinated optimization across different abstraction levels. Each tier maintains its own RL policy while sharing state information and reward signals with adjacent tiers.

Tier 1 (Infrastructure manager). Operates at the PM level, handling VM-to-PM placement decisions, server consolidation (migrating VMs to reduce the number of active hosts), and host power management (switching idle hosts to sleep mode). The Infrastructure Manager selects from placement-oriented LLHs (FFD, BFD, PSO-Migrate, Energy-Aware Placement) and consolidation-oriented LLHs (WOA-Consolidate, SA-Consolidate).

Tier 2 (Platform manager). Operates at the VM level, handling VM sizing (adjusting CPU and memory allocations), VM migration for load balancing, and resource reservation management. The Platform Manager selects from scheduling and balancing LLHs (DE-Balance, GA-Crossover) and coordinates with Tier 1 to ensure migration decisions align with placement objectives.

Tier 3 (Application manager). Operates at the task level, handling task-to-VM assignment, task queue management, and priority-based scheduling. The Application Manager selects from scheduling LLHs (SJF, ACO-Schedule) and scaling LLHs (GWO-Scale, Threshold-Reactive).

Inter-Tier Coordination: The three tiers coordinate through a shared reward mechanism. When an LLH applied at any tier produces a fitness improvement, the reward signal is propagated to all tiers, scaled by a coordination factor βcoord = 0.3 for non-originating tiers. Such reward propagation ensures that improvements at one tier positively influence selection policies at other tiers without creating circular dependencies. Additionally, state information flows upward (task-level metrics aggregate to VM-level, which aggregate to PM-level), providing each tier with a holistic view of the system.

A critical innovation of World-CRM over the original World algorithm is the redefinition of the RL state space using cloud-specific metrics. While the original World uses generic optimization metrics (current fitness value, improvement rate, stagnation count), cloud resource management requires a state representation that captures the multifaceted nature of data center operations. We define a 12-dimensional state vector s(τ) at each decision epoch τ:

s(τ) = [ūcpu, σ²cpu, ūmem, σ²mem, ūnet, FI, HAR, SVR, PTC, WT, EPT, CR]. (12)

The components of this state vector are defined in Table 1.

ūmem Mean memory utilization Average memory utilization across all active hosts [0, 1] σ²mem Memory utilization variance Variance of memory utilization across active hosts [0, 1] ūnet Mean network bandwidth saturation Average ratio of used to available bandwidth [0, 1] FI Memory FI 1 − (largest contiguous free memory block / total [0, 1] free memory) across all hosts HAR Active hosts ratio Fraction of PMs currently powered on [0, 1] SVR SLA violation rate Fraction of VMs/tasks currently violating SLA [0, 1] thresholds PTC Pending task count (normalized) Number of tasks in queue divided by total task [0, 1] capacity WT Workload trend Slope of linear regression on CPU demand over [−1, 1] last W epochs EPT Energy per task Current energy consumption divided by task [0, 1] throughput (normalized) CR Cost Rate Current operational CR normalized by budget [0, 1] threshold

Each continuous-valued state dimension is discretized into three levels {Low, Medium, High} using adaptive thresholds computed from running statistics (mean ± 0.5 standard deviation). This discretization yields a discrete state space of 312 = 531,441 possible states. To handle the high dimensionality and ensure adequate coverage with limited experience, we employ tile coding [43] with eight overlapping tilings, each with a different offset, producing a feature vector that provides smooth generalization across nearby states while maintaining sufficient discrimination.

The reward function at each decision epoch measures the improvement achieved by the selected LLH:

r(τ) = α · ΔEnergy(τ) + β · ΔSLA(τ) + γ · ΔUtilization(τ) + δ · ΔMakespan(τ) + ε

(13)

· ΔCost(τ), where ΔX(τ) = (X(τ−1) − X(τ)) / X(τ−1) represents the relative improvement in objective X from epoch τ−1 to τ. Positive values indicate improvement (reduction in energy, SLA violations, makespan, cost, or increase in utilization); negative values indicate degradation. The coefficients α = β = γ = δ = ε = 0.2 are initialized equally and adjusted by the multi-objective weight adaptation mechanism described in Section 3.7.

World-CRM maintains a pool of 12 base LLHs organized by their primary function (placement, scheduling, or scaling). Each LLH is a complete, self-contained optimization operator that can be applied to the current cloud state to produce an improved resource allocation. Table 2 summarizes the complete LLH pool.

each to the first PM with sufficient capacity LLH2 BFD Placement Sort VMs by resource demand (descending), assign O(M log M + MN) each to the PM with the least remaining capacity after placement (tightest fit) LLH3 PSO-Migrate Placement PSO for VM migration decisions; particles encode O(P · M · N) migration plans, velocity updates balance current placement and global best LLH4 Energy-Aware Placement Assign each VM to the PM that minimizes marginal O(MN) Placement energy increase: argminj [Ej(uj + Δu) − Ej(uj)] LLH5 Shortest Job Scheduling Sort tasks by estimated processing time (ascending), O(K log K + KM) First (SJF) assign to the VM with the earliest available time slot LLH6 ACO-Schedule Scheduling ACO for task-to-VM routing; ants construct task O(A · K · M) assignments with pheromone trails reflecting historical quality LLH7 GA-Crossover Scheduling Genetic algorithm crossover on task permutation O(G · Pop · K) schedules; two-point crossover with repair operator for precedence constraints LLH8 DE-Balance Scheduling DE for load balancing; mutation vectors redistribute O(G · Pop · M) task assignments to minimize utilization variance across VMs LLH9 GWO-Scale Scaling GWO for dynamic scaling decisions; alpha, beta, O(G · Pop · N) delta wolves encode scaling actions, pack position encodes target resource levels LLH10 WOA- Scaling WOA for server consolidation; bubble-net attacking O(G · Pop · M) Consolidate strategy identifies VMs to migrate for host deactivation LLH11 SA-Consolidate Scaling Simulated annealing for gradual consolidation; O(I · M) accepts migration moves based on Boltzmann probability with cooling schedule LLH12 Threshold- Scaling Rule-based reactive scaling with adaptive thresholds; O(N + M) Reactive thresholds adjust based on recent SLA performance and utilization trends

Note: P = number of PSO particles, A = number of ants, G = number of generations, Pop = population size, I = number of SA iterations. For real-time operation, metaheuristic LLHs are run with reduced parameters: P = A = Pop = 20, G = 5, I = 50.

The core mechanism of World-CRM, inherited and adapted from the original WHH [1], consists of two interleaved steps: Rewarding and Selection.

Step 1 (Rewarding with credit assignment). After each decision epoch, the performance of each recently applied LLH is evaluated. The challenge lies in credit assignment: When multiple LLHs have been applied in recent epochs, determining which LLH is responsible for observed improvements requires careful attribution. We employ a sliding-window credit assignment mechanism with a window of Wcredit = 10 recent decisions:

reward(LLHk) = Σi = 15 wi · improvementi(LLHk) / applications(LLHk, Wcredit), (14)

where improvementi(LLHk) is the total improvement in objective i across all epochs within the credit window where LLHk was applied, and applications(LLHk, Wcredit) is the number of times LLHk was applied within the window. This normalization prevents frequently applied LLHs from accumulating disproportionate reward and encourages exploration of underutilized heuristics.

Step 2 (Selection via Q-learning). The LLH selection policy is implemented as a Q-learning agent (Watkins and Dayan [44]) operating on the cloud-aware state space defined in Section 3.3. The Q-value update follows the standard temporal difference rule:

′ ′ ′

Q(s, LLHk) ← Q(s, LLHk) + αrl [r + γrl · maxLLH Q(s , LLH ) − Q(s, LLHk)], (15)

where s is the current state, LLHk is the applied heuristic, r is the reward from Eq. (14), s' is the resulting next state, αrl = 0.1 is the learning rate, and γrl = 0.95 is the discount factor.

The selection policy uses an adaptive ε-greedy strategy:

LLH ∗ = argmaxk Q(s, LLHk) with probability (1 − ε), (16) LLH ∗ = Uniform(Pool) with probability ε, (17)

The exploration rate ε decays geometrically from εinit = 0.3 to εmin = 0.05, with a decay factor of 0.995 per epoch during stable workload periods. However, ε is reset to εinit = 0.3 when a workload shift is detected (Section 3.6), enabling the agent to re-explore the LLH pool under new conditions.

Algorithm 1. Presents the main World-CRM decision loop.

Algorithm 1: World-CRM Main Decision Loop

Input: Physical machines PM, initial VMs VM, task stream T, parameters (α_rl, γ_rl, ε_init, ε_min, ε_decay, W, τ_shift) Output: Resource allocation decisions over time horizon 1: Initialize Q-table Q(s, LLH_k) ← 0 for all states s and LLHs k 2: Initialize ε ← ε_init 3: Initialize sliding windows W_demand ← ∅, W_credit ← ∅ 4: Perform initial VM placement using FFD 5: for each decision epoch τ = 1, 2, ..., T_max do 6: // MONITORING PHASE 7: Collect current metrics from all PMs, VMs, and task queues 8: 9: // STATE EXTRACTION 10: Compute state vector s(τ) using Equation (12) and Table 1 11: Discretize s(τ) using tile coding with 8 tilings 12: 13: // WORKLOAD SHIFT DETECTION (Section 3.6) 14:

Update W_demand with current demand distribution 15: if |W_demand| ≥ W then 16: Compute D_KL(P_current || P_historical) using Equation (18) 17: if D_KL > τ_shift then 18: ε ← ε_init // Reset exploration 19: Q ← 0.7 · Q // Decay Q-values 20: Log workload shift event at epoch τ 21: end if 22: end if 23: 24: // RL-BASED SELECTION (Section 3.5) 25: Generate random number p ∈ [0, 1] 26: if p < ε then 27: Select LLH* uniformly at random from Pool (exploration) 28: else 29: Select LLH* = argmax_k Q(s(τ), LLH_k) (exploitation) 30: end if 31: 32: // LLH EXECUTION 33: Apply LLH* to current cloud state 34: Execute resulting placement/scheduling/scaling decisions 35: 36: // REWARD COMPUTATION (Section 3.5) 37: Compute improvements ΔEnergy, ΔSLA, ΔUtilization, ΔMakespan, ΔCost 38:

Compute reward r(τ) using Equation (13) 39: Update credit window W_credit 40: Compute reward(LLH*) using Equation (14) 41: 42: // Q-TABLE UPDATE 43: Observe next state s(τ + 1) 44: Update Q(s(τ), LLH*) using Equation (15) 45: 46: // WEIGHT ADAPTATION (Section 3.7) 47: Adapt multi-objective weights w_1, ..., w_5 based on constraint proximity 48: 49: // EXPLORATION DECAY 50: ε ← max(ε_min, ε · ε_decay) 51: end for

Algorithm 2. Details the rewarding mechanism with credit assignment.

Algorithm 2: Rewarding Mechanism with Credit Assignment

Input: Credit window W_credit of recent (epoch, LLH, improvements) tuples, objective weights w_1,...,w_5 Output:

Reward value for the most recently applied LLH 1: Let LLH_current be the LLH applied at the current epoch 2: Let history ← entries in W_credit where LLH = LLH_current 3: if |history| = 0 then 4: return 0 // No history for this LLH 5: end if 6: 7: total_reward ← 0 8: for each objective i ∈ {energy, SLA, utilization, makespan, cost} do 9:

improvement_sum ← 0 10: for each entry (τ_j, LLH_j, Δ_j) in history do 11: improvement_sum ← improvement_sum + Δ_j[i] 12: end for 13: total_reward ← total_reward + w_i · improvement_sum / |history| 14:

end for 15: 16: // Penalty for constraint violations 17: if any constraint C1–C5 violated then 18: violation_penalty ← −0.5 · (number of violations / total constraints) 19: total_reward ← total_reward + violation_penalty 20: end if 21: 22: return total_reward Algorithm 3. Adaptive selection with workload shift detection.

Algorithm 3: Adaptive Selection with Workload Shift Detection

Input: Current demand window W_current, historical demand window W_historical, threshold τ_shift, Q-table, current state s, current ε Output: Selected LLH, updated ε, updated Q-table 1: // WORKLOAD SHIFT DETECTION via KL Divergence 2: Compute empirical distribution P_current from W_current 3: Compute empirical distribution P_historical from W_historical 4: Add smoothing: P(x) ← (P(x) + δ_smooth) / (1 + |X| · δ_smooth), δ_smooth = 1e-6 5: D_KL ← Σ_x P_current(x) · log(P_current(x) / P_historical(x)) 6: 7:

shift_detected ← false 8: if D_KL > τ_shift then 9: shift_detected ← true 10: ε ← ε_init // Reset to high exploration 11: Q(s, a) ← 0.7 · Q(s, a) for all s, a // Decay all Q-values 12: W_historical ← W_current // Update baseline 13: Log("Workload shift detected at current epoch, D_KL = " + D_KL) 14: end if 15: 16: // ADAPTIVE ε-GREEDY SELECTION 17: if shift_detected then 18: // During shift: boost exploration of diverse LLH categories 19: Select one LLH from each category {Placement, Scheduling, Scaling} 20: with probability 0.5 form composite LLH, else select single LLH 21: else 22: // Normal operation 23: p ← random(0, 1) 24: if p < ε then 25: LLH* ← random selection from Pool 26: else 27: LLH* ← argmax_k Q(s, LLH_k) 28: end if 29: end if 30: 31: return LLH*, ε, Q

Cloud workloads are inherently non-stationary, exhibiting diurnal patterns, flash crowds, seasonal variations, and unpredictable demand spikes [11]. Unlike the original World algorithm, which operates on static problem instances, World-CRM must adapt its LLH selection policy in real time as workload characteristics change. We achieve this through an online sliding-window adaptation mechanism based on Kullback–Leibler (KL) divergence.

At each decision epoch, we maintain two sliding windows of resource demand observations: a current window Wcurrent containing the most recent W = 50 epochs, and a historical baseline window Whistorical containing the W epochs prior to the current window. The empirical distribution of CPU demand levels is computed over each window, and the KL divergence between them measures the degree of distributional shift:

DKL(Pcurrent || Phistorical) = Σx Pcurrent(x) · log(Pcurrent(x) / Phistorical(x)), (18)

where Pcurrent and Phistorical are the empirical probability distributions of discretized CPU demand levels over the respective windows. Laplace smoothing (δ = 10−6) is applied to prevent division by zero.

When DKL exceeds the shift threshold τshift = 0.5, World-CRM triggers the following adaptation actions:

I. Exploration reset: the exploration rate ε is reset to its initial value εinit = 0.3, forcing the RL agent to broadly explore the LLH pool under the new workload conditions rather than relying on policies learned under the previous workload.

II. Q-Value decay: all Q-table values are multiplied by a decay factor of 0.7, partially preserving learned knowledge while reducing confidence in state-action valuations that may no longer be accurate: Q(s, a) ← 0.7 · Q(s, a) for all s, a.

III. Baseline update: the historical window is updated to reflect the new workload baseline, preventing repeated false-positive shift detections.

IV. Event logging: the workload shift event is logged with the epoch number and DKL value for post-hoc analysis.

This mechanism enables World-CRM to adapt to changing workload patterns without full retraining. The partial Q-value decay (rather than complete reset) preserves useful knowledge; for instance, the relative effectiveness of FFD versus BFD for placement is unlikely to change fundamentally even under a workload shift while allowing the policy to adjust to new optimality landscapes. The choice of τshift = 0.5 was determined through sensitivity analysis, balancing responsiveness to genuine shifts against stability under normal workload variability.

The weights w1, …, w5 in the composite fitness function Eq. (6) are not static but adapt based on the proximity of each objective to its constraint boundaries. This adaptive weighting scheme ensures that the optimization focus shifts dynamically to the most critical objective at any given time:

If SVR > θSLA: w2 ← w2 + Δw, normalize all weights, (19) If fenergy > budgetenergy: w1 ← w1 + Δw, normalize all weights, (20) If ūcpu < 0.30: w3 ← w3 + Δw, normalize all weights, (21)

where θSLA = 0.03 (3% SLA violation threshold), budgetenergy is the energy budget for the current period, and Δw = 0.05 is the weight increment. After each adjustment, all weights are renormalized to sum to 1. If no constraints are in proximity, weights decay toward the uniform distribution (wi = 0.2 for all i) at a rate of 0.01 per epoch. This mechanism ensures that, for example, during high-load periods when SLA violations spike, the RL agent prioritizes LLHs that improve SLA performance; during low-load periods, it shifts focus to energy-minimizing consolidation strategies.

The per-epoch computational complexity of World-CRM consists of three components: 1) state extraction: O(N + M + K) for aggregating metrics across PMs, VMs, and tasks, 2) LLH selection: O(|Pool|) for Q-table lookup and argmax computation, and 3) LLH execution: varies by LLH, from O(N + M) for Threshold-Reactive to O(G · Pop · M · N) for population-based metaheuristics with reduced parameters. The dominant cost is LLH execution. With reduced metaheuristic parameters (G = 5, Pop = 20), the worst-case per-epoch complexity is O(5 · 20 · M · N) = O(100MN). For the largest data center (N = 800, M = 3200), this yields approximately 2.56 × 108 operations per epoch, which modern processors complete in under 300 ms. The Q-table memory requirement is O(|S| · |Pool|). With tile coding (8 tilings × 531,441 states = 4,251,528 features) and |Pool| ≈ 100, the total Q-table memory is approximately 425 million entries × 8 bytes ≈ 3.2 GB, feasible for modern servers. However, due to tile coding sparsity, the effective memory usage is substantially lower, typically under 500 MB.

All experiments are conducted using CloudSim Plus [45], a state-of-the-art Java-based cloud computing simulation framework that provides extensive modeling capabilities for data center infrastructure, VM lifecycle management, task scheduling, and power consumption modeling. CloudSim Plus extends the original CloudSim [46] with improved modularity, extensibility, and simulation accuracy. World-CRM is implemented as a set of custom scheduling and placement policies within the CloudSim Plus framework. We configure three data center scales to evaluate scalability:

Physical hosts are modeled after HP ProLiant DL360 Gen10 (2 cores, 4 GB RAM) and DL380 Gen10 (8 cores, 32 GB RAM) server configurations, distributed uniformly across the data center. The power consumption model follows the SPECpower benchmark: Pidle = 175 W and Pmax = 250 W, with linear interpolation between idle and maximum based on CPU utilization. Four VM types are defined: Small (1 vCPU, 1 GB RAM), Medium (2 vCPU, 4 GB RAM), Large (4 vCPU, 8 GB RAM), and XLarge (8 vCPU, 16 GB RAM), with uniform random distribution across types.

Three real-world cloud workload traces are used to ensure realistic and diverse evaluation:

World-CRM is compared against nine baseline methods spanning static heuristics, metaheuristics, multi-objective evolutionary algorithms, RL approaches, and hyper-heuristic variants:

The original World algorithm [1] applied to cloud without cloud-specific adaptations (generic state, same LLH pool)

Performance is evaluated across the following metrics: 1) energy consumption (kWh): total data center energy over the simulation period, computed as the integral of power consumption across all hosts, 2) SLA violation rate (%): percentage of VMs or tasks violating response time or availability SLAs, 3) resource utilization (%): average CPU and memory utilization of active hosts, weighted equally, 4) makespan (seconds): total completion time for the batch workload set, 5) operational cost ($): total cost comprising energy cost ($0.10/kWh), SLA penalty cost ($0.50 per violation), and VM migration cost ($0.01 per migration), 6) number of VM migrations: total migrations during the simulation, 7) decision latency (ms): wall-clock time for each resource management decision, 8) adaptation speed: number of epochs to recover performance after a synthetic workload shift.

Each experimental configuration is executed for 30 independent runs with different random seeds. Results are reported as mean ± standard deviation. Statistical significance is assessed using the Wilcoxon rank-sum (Mann–Whitney U) test at α = 0.05 with Bonferroni correction for multiple comparisons (9 pairwise comparisons per metric, corrected significance level αcorrected = 0.0056).

Table 6 presents the comprehensive performance comparison of World-CRM against all baselines on the CCT using the medium data center configuration (200 hosts). This configuration represents a realistic mid-scale cloud deployment and serves as the primary evaluation scenario.

traces on the medium data center.

Trace Method Energy (kWh) Google World-CRM 847 ± 12 Original World 889 ± 19 2.3 ± 0.4 DQN-VMPlace 903 ± 25 2.1 ± 0.4 NSGA-II-VM 921 ± 22 2.5 ± 0.5 Azure World-CRM 912 ± 15 Original World 968 ± 21 2.8 ± 0.5 DQN-VMPlace 951 ± 28 2.6 ± 0.5 NSGA-II-VM 988 ± 25 3.1 ± 0.6 Bitbrains World-CRM 798 ± 11 Original World 841 ± 16 1.9 ± 0.3 DQN-VMPlace 856 ± 22 1.7 ± 0.3 NSGA-II-VM 879 ± 20 2.2 ± 0.4 72.4 ± 1.8 3241 ± 89 423 ± 18 70.1 ± 1.8 3387 ± 98 452 ± 20 69.7 ± 2.0 3438 ± 115 461 ± 24 68.1 ± 1.9 3512 ± 108 478 ± 22 65.8 ± 2.1 3789 ± 125 521 ± 27 64.1 ± 2.2 3845 ± 118 548 ± 25 63.2 ± 2.3 3892 ± 131 542 ± 29 61.3 ± 2.4 4156 ± 142 584 ± 31 58.2 ± 2.0 4287 ± 112 618 ± 24 54.7 ± 1.5 4521 ± 98 672 ± 19

SLA Viol. (%) Utilization (%) Makespan (s) 1.8 ± 0.3 72.4 ± 1.8 3241 ± 89 70.1 ± 1.8 3387 ± 98 69.7 ± 2.0 3438 ± 115 68.1 ± 1.9 3512 ± 108 2.1 ± 0.3 70.8 ± 1.9 3589 ± 95 67.9 ± 2.0 3812 ± 110 68.4 ± 2.1 3748 ± 120 66.5 ± 2.2 3921 ± 115 1.5 ± 0.2 74.1 ± 1.7 2987 ± 78 72.3 ± 1.8 3145 ± 88 71.8 ± 1.9 3198 ± 102 69.4 ± 2.0 3312 ± 95

World-CRM achieves the best performance across all five metrics. Compared to the best traditional heuristic MBFD, World-CRM reduces energy consumption by 22.9% (847 vs. 1098 kWh), SLA violations by 61.7% (1.8% vs. 4.7%), and makespan by 22.0% (3241 vs. 4156 s), while improving resource utilization by 11.1 percentage points (72.4% vs. 61.3%). Compared to the strongest individual baselines, DQN-VMPlace for SLA performance and the original world for energy, World-CRM achieves consistent improvements across all metrics simultaneously, demonstrating the value of its integrated cloud-specific adaptations. The comparison with HH-Random is particularly instructive: both methods use the identical LLH pool, but World-CRM's RL-based selection achieves 16.3% lower energy and 52.6% fewer SLA violations, directly validating the effectiveness of learned heuristic selection over random selection.

Table 7 presents the performance of World-CRM and the top-performing baselines across all three workload

World-CRM consistently achieves the best performance across all three traces, despite their fundamentally different workload characteristics. On the Google trace, which features bursty web-service workloads, World-CRM achieves 23% energy reduction over MBFD. On the Azure trace, characterized by high VM churn and diverse sizes, World-CRM achieves 58% fewer SLA violations compared to FFD (2.1% vs. 5.0%). On the Bitbrains trace, featuring stable enterprise workloads with multi-dimensional resource consumption, World-CRM achieves 22% better resource utilization compared to RR (74.1% vs. 60.8%). The consistent superiority across diverse workload types validates the adaptability of the RL-based LLH selection mechanism: World-CRM learns different LLH selection strategies for different workload characteristics, as analyzed in Section 5.5.

Table 8 evaluates World-CRM's performance and scalability across data center sizes.

Improvement (%) is relative to the MBFD baseline.

Scale Method Energy (kWh) Improv. (%) SLA Viol. (%) Util. (%) Decision Latency (ms) Small (50) World-CRM 215 ± 5 19.2 2.2 ± 0.4 70.1 ± 2.0 45 MBFD 266 ± 8 5.1 ± 0.9 60.8 ± 2.5 3 DQN-VMPlace 231 ± 9 13.2 2.5 ± 0.5 68.4 ± 2.2 28 Medium (200) World-CRM 847 ± 12 22.9 1.8 ± 0.3 72.4 ± 1.8 142 MBFD 1098 ± 28 4.7 ± 0.8 61.3 ± 2.4 8 DQN-VMPlace 903 ± 25 17.8 2.1 ± 0.4 69.7 ± 2.0 85 Large (800) World-CRM 3124 ± 38 31.2 1.6 ± 0.3 74.8 ± 1.6 287 MBFD 4542 ± 72 5.3 ± 0.9 59.2 ± 2.6 15 DQN-VMPlace 3478 ± 65 23.4 2.0 ± 0.4 71.2 ± 2.1 195

Two important observations emerge. First, World-CRM's relative improvement over baselines increases with data center scale: Energy improvement over MBFD grows from 19.2% (Small) to 22.9% (Medium) to 31.2% (Large). This scaling effect arises because larger data centers offer more optimization opportunities: more candidate hosts for placement, more VMs for consolidation, and more tasks for scheduling, which World-CRM's diverse LLH pool can exploit more effectively than fixed-strategy approaches. Second, decision latency remains within the real-time requirement of 300 ms even for the largest configuration (287 ms for 800 hosts), validating the computational feasibility of the approach for production deployment. While the latency is significantly higher than simple heuristics (e.g., MBFD at 15 ms), it remains well within the 5-minute decision epoch interval and is comparable to the DQN-VMPlace baseline (195 ms).

To evaluate World-CRM's adaptation capability, we inject a synthetic workload shift at epoch 500 of the CCT simulation on the medium data center: A 50% sudden increase in task arrival rate sustained for 200 epochs, followed by a return to normal levels.

World-CRM demonstrates dramatically faster adaptation. Its SLA violation rate spikes from 1.8% to only 4.2% during the shift (vs. 7.8% for DQN-VMPlace and 11.2% for MBFD) and recovers to near pre-shift levels (2.1%) within just 15 epochs (75 minutes). This discretization process is 3× faster than DQN-VMPlace (45 epochs) and 4× faster than NSGA-II-VM (60+ epochs). Static heuristics (MBFD, RR) show no recovery at all because they cannot adapt their strategy. The Original World, while adaptive, lacks the workload shift detection mechanism and cloud-specific state features, recovering in 28 epochs, nearly twice as long as World-CRM. This finding validates the online sliding-window adaptation mechanism (Section 3.6) as a critical innovation for cloud environments.

We analyze the LLH selection frequencies of World-CRM across different workload phases to understand whether the RL mechanism learns meaningful cloud-specific strategies.

The selection patterns reveal that World-CRM's RL mechanism learns highly interpretable, cloud-domain-meaningful strategies. During low-load periods, consolidation-oriented LLHs dominate: SA-Consolidate (28%), WOA-Consolidate (22%), and Energy-Aware Placement (18%) are selected most frequently, reflecting the strategy of migrating VMs and shutting down idle hosts to save energy. During high-load periods, the focus shifts to distribution and scaling: ACO-Schedule (25%), PSO-Migrate (20%), and GWO-Scale (18%) dominate, reflecting the need to spread workload across available resources and scale capacity. During transition periods (following workload shift detection and ε reset), selection is broadly distributed across all LLHs, reflecting the exploratory phase. This analysis validates that the RL-based selection mechanism goes beyond mere statistical correlation and learns strategies that align with well-understood cloud management principles.

To quantify the contribution of each World-CRM innovation, we conduct an ablation study by systematically removing individual components.

Several findings emerge from the ablation analysis. First, the RL-based selection mechanism is the most critical component: replacing it with random selection (HH-Random) causes the largest performance degradation (19.5% worse energy, 111.1% more SLA violations), confirming that the core World algorithm's RL-based rewarding and selection mechanism [1] provides the fundamental value. Second, the cloud-aware state representation contributes significantly (5.0% energy, 27.8% SLA degradation without it), validating that cloud-specific metrics provide richer information for the RL agent than generic optimization metrics. Third, the online adaptation mechanism has a moderate impact on energy (3.0%) but a dramatic impact on SLA violations (61.1%), because workload shifts cause SLA spikes that the fixed-policy variant cannot recover from. Fourth, multi-tier coordination improves both energy (3.7%) and utilization (4.0%), confirming that coordinated cross-tier optimization outperforms siloed approaches. Fifth, the infinite pool concept provides modest but consistent improvement (0.9% energy, 11.1% SLA), suggesting that composite LLHs add value beyond the base set of 12 but are not as critical as the selection mechanism itself.

All comparisons are statistically significant at p < 0.05 in the uncorrected tests. After Bonferroni correction (αcorrected = 0.0056), World-CRM's improvements over static heuristics, metaheuristics, NSGA-II-VM, and HH-Random remain highly significant (p < 0.001) across all metrics. The comparisons with DQN-VMPlace are significant for energy (p = 0.003), utilization (p = 0.002), makespan (p < 0.001), and cost (p = 0.001) after correction, but the SLA violation improvement (p = 0.015) falls slightly above the corrected threshold, indicating a strong trend but a marginally non-significant result. The comparison with the Original World shows similar patterns: utilization (p = 0.004), makespan (p = 0.002), and cost (p = 0.005) are significant after correction, while energy (p = 0.012) and SLA (p = 0.008) are significant at the uncorrected level. These results confirm that World-CRM's improvements are robust and not attributable to random variation.

We analyze the convergence behavior of World-CRM's Q-learning process to validate the warm-up period and assess policy stability. The Q-value magnitude across all state-action pairs (measured as the root-mean-square of Q-values) follows a characteristic trajectory: rapid growth during the first 30 epochs (initial learning phase), moderate growth from epochs 30–80 (policy refinement), and stabilization after approximately epoch 80 (policy convergence). By epoch 100 (the end of the warm-up period), the Q-value RMS varies by less than 2% between consecutive epochs, confirming that the warm-up period of 100 epochs is sufficient for Q-table initialization.

The LLH selection distribution stabilizes by approximately epoch 120, with the entropy of the selection distribution plateauing at 2.1 bits (compared to the maximum entropy of 3.58 bits for 12 LLHs), indicating that the agent has learned a discriminative but not degenerate policy. This observation implies that the Q-learning agent does not collapse to a single LLH but maintains diversity in its selection, consistent with the hyper-heuristic principle that different LLHs are suitable for different states.

For practical deployment, we evaluate the feasibility of transfer learning: A Q-table pre-trained on one workload trace is used as the initial Q-table for a different trace, compared to training from scratch. Pre-trained initialization achieves 90% of the final performance within 20 epochs (compared to 60 epochs from scratch), demonstrating that learned LLH selection policies transfer effectively across workload types, reducing the warm-up overhead from approximately 8 hours to under 2 hours.

The ablation study (Table 11) reveals that the RL-based selection mechanism accounts for the largest share of World-CRM's performance advantage, with random selection degrading energy performance by 19.5% and SLA performance by 111.1%. This result aligns with the findings of Daliri et al. [1] on static problems and extends them to the cloud domain. The RL mechanism succeeds because it learns which heuristics work best for specific cloud states, a form of context-aware strategy switching that static approaches cannot achieve. For example, during high utilization states, the Q-learning agent learns that PSO-Migrate and ACO-Schedule are most effective (Table 10), while during low utilization states, consolidation heuristics (SA-Consolidate, WOA-Consolidate) yield the highest rewards. This state-dependent selection is precisely what the original World algorithm was designed to achieve [1], and our results confirm that the principle translates effectively to the cloud computing domain.

Removing the cloud-specific state features and reverting to generic optimization metrics (current fitness, improvement rate, stagnation count, as in the original World) degrades SLA performance by 27.8% (Table 11). This significant degradation occurs because generic metrics fail to capture cloud-specific dynamics. For instance, the memory FI alerts the RL agent to states where placement quality is degrading due to fragmented host memory, which would not be captured by aggregate fitness alone. Similarly, the Workload Trend (WT) feature enables proactive LLH selection, choosing scaling heuristics before load spikes fully materialize, which reactive generic metrics cannot support. The comparison with the Original World (Tables 6 and 7) further confirms this: applying the original World algorithm with its generic state representation to the same LLH pool yields 4.7% worse energy and 21.7% more SLA violations than World-CRM, directly attributable to the cloud-specific adaptations.

The workload shift experiment (Table 9) provides compelling evidence for the online adaptation mechanism. World-CRM recovers from a 50% sudden load increase within 15 epochs (75 minutes), 3× faster than DQN-VMPlace and 4× faster than NSGA-II-VM. Two factors drive this rapid recovery. First, the KL divergence-based shift detection triggers immediate exploration restart (ε → 0.3), forcing the RL agent to re-evaluate all LLHs under the new conditions rather than persisting with a now-suboptimal policy. Second, the partial Q-value decay (Q ← 0.7 · Q) strikes a productive balance between preserving transferable knowledge (e.g., FFD is still a valid placement heuristic under high load) and reducing overconfidence in state-action values calibrated for the old workload. The Original World, lacking workload shift detection, relies solely on its ε-greedy exploration to eventually adapt, taking nearly twice as long (28 epochs). Static baselines (MBFD, RR) show no recovery at all, underscoring the fundamental limitation of non-adaptive approaches in non-stationary environments.

World-CRM achieves 4.7% better energy efficiency and 21.7% fewer SLA violations than the Original World applied directly to the cloud (Table 6). This improvement validates the thesis that domain-specific adaptation of the WHH through cloud-aware state representation, cloud-tailored fitness function, online adaptation, and multi-tier coordination provides meaningful benefits beyond what the general-purpose algorithm achieves. The Original World's performance is nonetheless strong (second-best on several metrics), confirming that the core RL-based rewarding and selection mechanism [1] is inherently powerful and providing a solid foundation for domain-specific extensions.

World-CRM outperforms DQN-VMPlace by 6.2% on energy and 14.3% on SLA violations (Table 6). The advantage stems from World-CRM's diverse LLH pool, which provides a broader repertoire of search strategies than a single DQN policy. While DQN-VMPlace learns a single policy mapping states to placement actions, World-CRM leverages nine distinct metaheuristic operators (PSO, ACO, GA, DE, GWO, WOA, SA) plus three classical heuristics, dynamically selecting the most appropriate one for each state. This diversity of search operators enables World-CRM to avoid premature convergence and limited exploration that can affect single-policy RL approaches (Sutton and Barto [43]). Moreover, World-CRM addresses all three sub-problems (placement, scheduling, scaling) simultaneously, while DQN-VMPlace addresses only placement.

Several factors support World-CRM's viability for production deployment, 1) the warm-up period of 100 epochs (approximately 8.3 hours at 5-minute intervals) is operationally feasible: it can be completed during initial deployment or during a planned maintenance window, 2) Pre-trained Q-tables from similar data center configurations reduce the effective warm-up to approximately 20 epochs (1.7 hours), 3) decision latency of 142–287 ms (Table 8) is well within the 5-minute decision epoch and is negligible relative to the time scales of VM migration and task completion, 4) the Q-table memory footprint (under 500 MB effective) is trivial for modern management servers, and 5) the algorithm is fully compatible with existing cloud management frameworks, requiring only monitoring data already available in production systems (CPU utilization, memory usage, network traffic, SLA logs).

The current work has several limitations that warrant acknowledgement. First, evaluation is conducted in CloudSim Plus simulation, which, despite its sophistication, may not capture all real-world complexities, including network topology effects (especially in fat-tree or leaf-spine architectures), hardware failures and their cascading impacts, multi-tenancy interference (noisy neighbors), and I/O contention at the storage layer. Second, the Q-table approach, while effective for the 12-dimensional discretized state space, may not scale to very high-dimensional or continuous state spaces without modification. Function approximation using neural networks (DQN or policy gradient methods) could extend World-CRM's scalability but would introduce the training instability challenges of deep RL. Third, all evaluations use historical workload traces replayed in simulation; no live production deployment has been conducted. The sim-to-real transfer of learned policies remains an open question. Fourth, the 5-minute decision epoch interval, while suitable for VM-level management, may be too coarse for latency-critical serverless/Function-as-a-Service (FaaS) applications requiring sub-second scaling decisions. Fifth, the current formulation addresses a single data center; inter-data-center resource management in geo-distributed cloud settings (involving network latency, data locality, and regulatory constraints) is not addressed.

Several promising research directions emerge from this work: 1) deep RL integration: Replacing the Q-table with a deep neural network (DQN, PPO, or SAC) would enable continuous state handling and potentially scale to larger state spaces, though at the cost of sample efficiency and training stability, 2) federated hyper-heuristic: extending World-CRM to geo-distributed multi-data-center settings using federated learning to share Q-values across data centers without centralizing workload data, 3) container and Kubernetes integration: adapting World-CRM for container orchestration platforms, where the LLH pool would include Kubernetes-native operators (pod scheduling, horizontal/vertical pod autoscaling, node affinity management), 4) serverless/FaaS extension: redesigning the decision epoch and LLH pool for the sub-second timescales of serverless computing, 5) online LLH generation: using genetic programming to automatically generate novel LLHs based on building blocks of existing operators, moving beyond the fixed pool of base LLHs, 6) edge-cloud continuum: extending World-CRM to manage resources across edge devices, fog nodes, and cloud servers, addressing the emerging paradigm of edge computing, and 7) production deployment study: partnering with a cloud provider for a controlled A/B deployment study comparing World-CRM against production baselines on live workloads.