Draft: Add stop_housekeeping_cpuslocked()

Merge Request Required Information

Summary of Changes

CPU hotplug interferes with CPU isolation and introduces latency to real-time tasks.

The test:

  rtla timerlat hist -c 1 -a 500 &
  echo 0 > /sys/devices/system/cpu/cpu2/online

The RTLA tool reveals the following blocking thread stack trace:

               -> multi_cpu_stop
               -> cpu_stopper_thread
               -> smpboot_thread_fn

This happens because multi_cpu_stop() disables interrupts for EACH online CPU since takedown_cpu() indirectly invokes take_cpu_down() through stop_machine_cpuslocked(). I'm omitting the detailed description of the call chain.

Proposal: Limit the stop operation to housekeeping CPUs.

Approved Development Ticket(s)

All submissions to CentOS Stream must reference a ticket in Red Hat Jira.

Click for formatting instructions

Please follow the CentOS Stream contribution documentation for how to file this ticket and have it approved.

List tickets each on their own line of this description using the format "Resolves: RHEL-1234", "Related: RHEL-2345" or "Reverts: RHEL-3456", as appropriate.

kernel/stop_machine.c

@@ -680,8 +680,12 @@ int stop_machine_from_inactive_cpu(cpu_stop_fn_t fn, void *data,
 	struct multi_stop_data msdata = { .fn = fn, .data = data,
 					    .active_cpus = cpus };
 	struct cpu_stop_done done;
+	cpumask_var_t stop_cpus;
 	int ret;
 
+	if (!alloc_cpumask_var(&stop_cpus, GFP_KERNEL))
+		return -ENOMEM;
+
 	/* Local CPU must be inactive and CPU hotplug in progress. */
 	BUG_ON(cpu_active(raw_smp_processor_id()));
 	msdata.num_threads = num_active_cpus() + 1;	/* +1 for local */
@@ -692,9 +696,12 @@ int stop_machine_from_inactive_cpu(cpu_stop_fn_t fn, void *data,
 
 	/* Schedule work on other CPUs and execute directly for local CPU */
 	set_state(&msdata, MULTI_STOP_PREPARE);
-	cpu_stop_init_done(&done, num_active_cpus());
-	queue_stop_cpus_work(cpu_active_mask, multi_cpu_stop, &msdata,
-			     &done);
+	/*
+	 * don't stop isolated CPUs
+	 */
+	cpumask_and(stop_cpus, housekeeping_cpumask(HK_TYPE_MANAGED_IRQ), cpu_active_mask);
+	cpu_stop_init_done(&done, cpumask_weight(stop_cpus));
+	queue_stop_cpus_work(stop_cpus, multi_cpu_stop, &msdata, &done);
 	ret = multi_cpu_stop(&msdata);
 
 	/* Busy wait for completion. */
@@ -702,5 +709,6 @@ int stop_machine_from_inactive_cpu(cpu_stop_fn_t fn, void *data,
 		cpu_relax();
 
 	mutex_unlock(&stop_cpus_mutex);
+	free_cpumask_var(stop_cpus);
 	return ret ?: done.ret;
 }