Skip to main content
To troubleshoot kernel issues, use the following methods:

​
Boot failure due to incorrect or no DTB picked

When troubleshooting boot failure due to DTB load issues, verify authentication and DTB availability, and follow the steps to fix the problem. The following scenarios lead to boot failure:
  • Authentication failure The following example shows the logs for failure due to incorrect authentication: 
    Platform Subtype : 0
DtPlatformLoadDtb qcs6490-rb3gen2.dtb is loaded
Platform Subtype : -2090817768
DtPlatformLoadSign qcs6490-rb3gen2.sgn is loaded
failed to authenticate image !
  • Failure due to unavailability of DTB The following example shows the logs for failure due to the missing DTB: 
    DtPlatformLoadDtb qcs6490-rb3gen2.dtb is loading failed with Status = E
DtPlatformDxeEntryPoint: no DTB blob could be loaded, defaulting to ACPI (Status == Not Found)
    To fix the unavailability of DTB issues, do the following:
    • Verify if the corresponding DTB file is a part of a packaged dtb.bin file.
    • Mount the dtb.bin file locally on the host development machine to verify the following: 
      mount -t vfat dtb.bin /mnt/
fdtdump /mnt/combined-dtb.bin | grep -i model
  • To verify loaded DTB from the kernel log Use the following command to verify DTB loaded during boot from the kernel log: 
    dmesg | grep -i model

​
Serial console not working

To troubleshoot serial console log failure, enable specific drivers in the kernel configuration and add relevant parameters to the kernel boot arguments.
  • Enable the following driver in the kernel configuration file:
    • CONFIG_SERIAL_QCOM_GENI=y
    • CONFIG_SERIAL_QCOM_GENI_CONSOLE=y
  • Add the following parameter to the kernel boot arguments: 
    console=ttyMSM0,115200n8
  • To get early boot messages from the kernel, add the following parameter to the kernel boot arguments: 
    earlycon

​
Debug remote processor failure

To troubleshoot remoteproc failure, capture kernel logs, use matching firmware signature files, and verify firmware locations in the device.
  • Ensure that all the matching subsystem images are flashed without any errors.
  • Verify if there are any errors in the kernel log. The following example shows a sample kernel log: 
    0x000000000A27652C |   5198.790423:   qcom_q6v5_pas 3000000.remoteproc: fatal error received: err_inject_crash.c:413:Crash injected via Diag
0x000000000A276689 |   5198.801061:   remoteproc remoteproc2: crash detected in 3000000.remoteproc: type fatal error
0x000000000A2767A1 |   5198.809602:   remoteproc remoteproc2: handling crash #1 in 3000000.remoteproc
0x000000000A27688E |   5198.816837:   remoteproc remoteproc2: recovering 3000000.remoteproc
0x000000000A276971 |   5198.823784:   qcom_q6v5_pas 8a00000.remoteproc: subsystem event rejected
  • Disable the following subsystem restart feature to ensure that the remoteproc crash signature is visible in the kernel log: 
    echo disabled > /sys/kernel/debug/remoteproc/remoteprocN/recovery
  • Confirm if all the necessary firmware files are present in the /lib/firmware/qcom/<SoC> in rootfs file system.

​
Debug configurations or symbols not reflecting in images

To debug kernel configuration change issues where configurations or symbols are not reflecting in the images, follow these steps:
  1. Add the debug configuration driver to the kernel/configs/debug.config file.
  2. Export the DEBUG_BUILD=1 macro before running the bitbake command.

​
System memory is too low

To debug the cases where a system runs out of free memory, see Debugging linux kernel.

​
Identify DTB during boot

When the device boots, the boot loader verifies the following IDs and loads the corresponding DTB file: 
qcom,msm-id = <x z>;
qcom,board-id = <y y'>;
  • x = ID for SoC
  • z = ID for SoC revision (reserved field)
  • y = ID for CDP, MTP (hardware variants), and platform
  • y’ = ID for subtype (assumed 0 if absent)

​
Debug RT kernel

The following methods are used to troubleshooting issues in the RT kernel:
  • How to verify that the underlying kernel is real time? After the boot is complete, run the following commands to verify the kernel type: 
    uname -r
6.6-rt15
uname -v
SMP PREEMPT_RT
  • How to make drivers RT compliant? Ensure that synchronization primitives do not break RT assumptions. The following are examples of the scenarios that can break RT assumptions and cause unexpected system behavior: 
    /* Acquiring a preemptible lock in non preemptible context */
preempt_disable( )
……
spin_lock( )
/* Acquiring a preemptible lock in non preemptible context /
raw_spin_lock( )
……
spin_lock( )
/* Acquiring a non preemptible lock in preemptible context /
local_lock_irq( )
   …..
raw_spin_lock( )
  • Debug RT kernel configurations The following are the debug configurations for the RT kernel:
    • CONFIG_DEBUG_ATOMIC_SLEEP - Verify for sleep inside an atomic section.
    • CONFIG_PROVE_RAW_LOCK_NESTING - Allows the raw_spinlock vs. spinlock nesting. Ensures that the lock nesting rules for PREEMPT_RT kernels are not violated. While debugging, you can switch from an RT kernel to a non-RT kernel. To switch, make changes to the meta-qcom/ci/linux-qcom-rt-6.18.yml file to change virtual/kernel settings in linux-qcom-rt and compile again. To make the changes, run the following commands: 
      - PREFERRED_PROVIDER_virtual/kernel = "linux-qcom-rt"
+ PREFERRED_PROVIDER_virtual/kernel = "linux-qcom"
For more information about locking primitives with the RT kernel, see Lock types and their rules—The Linux Kernel documentation.

​
Troubleshooting virtio issues

Use the following methods to troubleshoot virtio issues:
  • Ensure that the kernel is compiled with the required configurations.
  • Verify the QEMU command-line options or libvirt XML configuration.
  • Verify the system logs for errors.
  • Enable traces in the QEMU command-line for virtio using the available trace back ends. For more information about KVM traces, see KVM traces.