Configure the scheduler - Qualcomm Dragonwing Documentation

The Qualcomm^® Linux kernel supports the standard Linux scheduler solution. The kernel uses the scheduler to choose the right CPU for task placement based on the CPU energy consumption. The scheduler decides the order in which the processes must run. The scheduler runs the processes from the runqueue of each CPU. The kernel baseline supports the standard Linux scheduler solution. The kernel uses Energy Aware Scheduling (EAS) to choose the right CPU for task placement based on the CPU energy consumption. EAS overrides the completely fair scheduler (CFS) task wake-up balancing code. It uses the energy model (EM) of the CPUs and the per entity load tracking (PELT) signals to choose an energy-efficient target CPU during wake-up balance in a system with asymmetric CPU topology. For basic scheduler documentation, see Scheduler.

Understand CPU topology and EAS

The Qualcomm SoCs have a heterogeneous CPU topology that is differentiated in terms of CPU capacity metrics used in EAS. EAS uses the concept of capacity to differentiate CPUs with different computing capabilities. The capacity of a CPU represents the amount of work it finishes when running at its highest frequency compared to the most capable CPU of the system. Capacity values are normalized in a 1024 range (the most powerful CPU/cluster is configured at 1024). EAS builds the capacity of a cluster based on the Dhrystone million instructions per second (DMIPS) value specified in the CPU node and the maximum frequency supported by the cluster. The following are the sysfs nodes for CPU topology and capacity: Table: CPU topology and capacity

Commands	Purpose
cat /sys/devices/system/cpu/cpu*/cpu_capacity	To get the capacity associated with each CPU in the system.
cat /sys/devices/system/cpu/cpufreq/policy*/related_cpus	To get a list of the CPUs associated with each cluster in the system.

For more information about how a scheduler uses CPU capacity, see Capacity Aware Scheduling.

Change the default CPU frequency governor

The CPU frequency governor can be changed at runtime or set statically during build compilation.

Kconfig configuration: To set the CPU frequency governor, enable the corresponding driver in the kernel defconfiguration file. To set PERFORMANCE governor as the default CPU frequency governor, set CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y in the defconfiguration file.
Kernel command-line: To override the kernel configuration option, add a cpufreq.default_governor=performance parameter to the kernel command-line in the KERNEL_CMDLINE_EXTRA of the kas base configuration file ci/base.yml in the SoC-specific machine inclusion file. For example, meta-qcom/conf/machine/include/qcom-<SoC>.inc file to set the appropriate CPU frequency governor.

Configure SchedUtil governor

The SchedUtil CPU frequency governor is the default governor in the Qualcomm Linux kernel. SchedUtil predicts optimal operating points (OPPs) based on the CPU utilization, maintaining coherence between frequency requests and energy predictions. The SchedUtil CPU frequency governor is tied to EAS, and it tries to predict at which OPP all the CPUs run next to estimate their energy consumption. For more information, see SchedUtil.

Configure utilization clamping (UCLAMP)

Utilization clamping is a scheduler feature that allows user space to manage the performance requirements of tasks. To perform clamping, configure the following parameters:

UCLAMP_MIN: If it’s set to any value > 0, the task demand is always greater than or equal to this value. If the actual task demand is more than this value, then the actual demand signal is used, but if the actual task demand is lower than this value, then UCLAMP_MIN is reported as the task demand.
UCLAMP_MAX: If it’s set to any value > 0, the task demand is always less than or equal to this value. If the actual task demand is less than this value, then the actual demand signal is used, but if actual task demand is more than this value, then UCLAMP_MAX is reported as task demand.

UCLAMP_MAX > UCLAMP_MIN

Use the UCLAMP to influence the scheduler’s task placement decisions. On a heterogeneous system, the scheduler uses the task demand or utilization signal (PELT signal) to classify a task as small or big. Based on the input from the task classification, the scheduler selects small (less computing power) or big (more computing power) CPU cores for task placement with a probable impact on power consumption. For example, clamp non-important (trivial/background/housekeeping) tasks to lower values (lower UCLAMP_MAX) to influence the scheduler to place the tasks on the little cluster. Similarly, clamp the important/foreground/active tasks to higher values (higher UCLAMP_MIN) to influence the scheduler to place the tasks on the big cluster. UCLAMP also allows the scheduler to control frequency guidance. To meet the quick frequency ramp for a task, clamp the task to a higher demand (UCLAMP_MIN) value. The higher clamping helps to ramp up the frequency of the cluster to meet the performance requirements of the task. The UCLAMP operation allows performance management of tasks/task groups from the user space. UCLAMP allows the placement of tasks or task groups and allows performance hints or constraints. This mechanism is used to influence the scheduler placement decisions or to influence the frequency guidance of the cluster. For more information, see UCLAMP and about the interface and configuration, see Utilization Clamping.

Run a user-space task at real-time priority

Tasks that must run ahead of every SCHED_OTHER (CFS) task use one of the POSIX real-time policies. Both policies run at a fixed priority in the range 1–99, where 99 is the highest, and both preempt all CFS tasks: Table: Real-time scheduling policies

Policy	Behaviour
`SCHED_FIFO`	Runs until it blocks, yields, or is preempted by a higher-priority RT task. No time slice.
`SCHED_RR`	Same as `SCHED_FIFO`, except that tasks of equal priority share the CPU in round-robin time slices.

Real-time policies are available on the standard Qualcomm Linux kernel. The PREEMPT_RT kernel does not add the policies; it reduces the latency at which an RT task is scheduled after it becomes runnable. For more information, see RT kernel fundamentals.

Set the policy from the command line

Use chrt from the util-linux package to start a new task or to change the policy of a running task.

Start a command as SCHED_FIFO at priority 80:

chrt --fifo 80 /usr/bin/my_control_loop

Change the policy of a running thread. Use the thread ID (TID), not the process ID, when the target is one thread of a multithreaded process:
chrt --fifo --pid 80 <TID>

Confirm the policy and priority that were applied:

chrt --pid <TID>
# Expected output:
# pid <TID>'s current scheduling policy: SCHED_FIFO
# pid <TID>'s current scheduling priority: 80

To pin the task to a dedicated CPU at the same time, combine chrt with taskset:

taskset --cpu-list 7 chrt --fifo 80 /usr/bin/my_control_loop

For information about reserving CPU 7 from the general-purpose scheduler, see Runtime performance tuning.

Set the policy from the application

Call sched_setscheduler() on the calling thread so that the priority is applied before the time-critical work begins:

#include <sched.h>
#include <sys/mman.h>
#include <stdio.h>

int main(void)
{
    struct sched_param param = { .sched_priority = 80 };

    /* Raise the calling thread to SCHED_FIFO priority 80. */
    if (sched_setscheduler(0, SCHED_FIFO, &param) == -1) {
        perror("sched_setscheduler");
        return 1;
    }

    /* Lock the address space in RAM so that a page fault cannot
     * introduce latency inside the real-time loop.
     */
    if (mlockall(MCL_CURRENT | MCL_FUTURE) == -1) {
        perror("mlockall");
        return 1;
    }

    /* Time-critical loop runs here. */
    return 0;
}

For a pthread-based application, set the attributes before creating the thread so that it starts at the correct priority:

#include <pthread.h>
#include <sched.h>

pthread_attr_t attr;
struct sched_param param = { .sched_priority = 80 };

pthread_attr_init(&attr);
pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED);
pthread_attr_setschedpolicy(&attr, SCHED_FIFO);
pthread_attr_setschedparam(&attr, &param);
pthread_create(&thread, &attr, rt_thread_fn, NULL);

Changing the policy requires the CAP_SYS_NICE capability. Either run the process as root or grant the capability to the binary:

setcap cap_sys_nice+ep /usr/bin/my_control_loop

Configure the real-time throttle

The kernel throttles real-time tasks so that a runaway RT loop cannot starve the system. By default, RT tasks are limited to 950 ms of every 1000 ms period:

cat /proc/sys/kernel/sched_rt_period_us    # 1000000
cat /proc/sys/kernel/sched_rt_runtime_us   # 950000

To let an RT task consume a full CPU, disable the throttle:

echo -1 > /proc/sys/kernel/sched_rt_runtime_us

Disabling the RT throttle allows a non-yielding SCHED_FIFO task to hold a CPU indefinitely. If that task runs on a CPU that services the watchdog petting thread, the SoC watchdog resets the device. Reserve a dedicated CPU for the RT task and keep housekeeping work on the remaining CPUs. For more information, see RT validation and known limitations.

For more information about the real-time policies and their interfaces, see sched(7) and Real-Time group scheduling.

​Understand CPU topology and EAS

​Change the default CPU frequency governor

​Configure SchedUtil governor

​Configure utilization clamping (UCLAMP)

​Run a user-space task at real-time priority

​Set the policy from the command line

​Set the policy from the application

​Configure the real-time throttle

Understand CPU topology and EAS

Change the default CPU frequency governor

Configure SchedUtil governor

Configure utilization clamping (UCLAMP)

Run a user-space task at real-time priority

Set the policy from the command line

Set the policy from the application

Configure the real-time throttle