Skip to main content
Linux uses a virtual memory system. Therefore, the addresses that the user program accesses don’t correspond to the physical addresses that the hardware uses directly. Virtual memory introduces a layer of indirection, allowing programs to assign extra memory beyond the physically available memory. Memory management implementation covers the following areas:
  • Management of physical pages in the memory
  • Buddy system to assign memory in large chunks
  • Slab, slub, and slob allocators to assign smaller chunks of memory
  • vmalloc mechanism to assign noncontiguous blocks of memory
  • Address space of the processes

​
/proc file system

The /proc file system provides the following files:

​
/proc/meminfo

This file provides information about distribution and usage of memory.
  • To view the contents of this file, run the following command: 
    cat /proc/meminfo
    Sample output: 
    MemTotal:        3813532 kB
MemFree:          624836 kB
MemAvailable:    2098008 kB
Buffers:           40416 kB
Cached:          1484320 kB
SwapCached:            0 kB
Active:          1334816 kB
.
.
.
.
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     2431048 kB
Committed_AS:   99995284 kB
VmallocTotal:  258867136 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
CmaTotal:         163840 kB
CmaFree:            1368 kB
For more information about the parameters available in this file, see /proc filesystem.

​
/proc/vmstat

This file shows detailed virtual memory statistics from the kernel. Most of the statistics are available only if you enable the CONFIG_VM_EVENT_COUNTERS option in the init/Kconfig file.
  • To view the contents of this file, run the following command: 
    cat /proc/vmstat
    Sample output: 
    nr_free_pages 156290
nr_alloc_batch 132
nr_inactive_anon 108
nr_active_anon 165006
nr_inactive_file 212275
nr_active_file 168709
nr_unevictable 64
nr_mlock 64
nr_anon_pages 164982
nr_mapped 90366
nr_file_pages 381184:
:
unevictable_pgs_mlocked 0
unevictable_pgs_munlocked 0
unevictable_pgs_cleared 0
unevictable_pgs_stranded 0
unevictable_pgs_mlockfreed 0
For more information about the parameters available in this file, see Linux manual page.

​
/proc/iomem

This file shows the memory map of the system for its various device drivers.
  • To view the contents of this file, run the following command: 
    cat /proc/iomem
    Sample output: 
    007781b8-007791b7 : vmpm
010aa000-010abfff : tsens_physical
010ac000-010ac003 : pshold-base
010ad000-010aefff : tsens_physical
01680000-0168ffff : /soc/arm,smmu-anoc1@1680000
016c0000-016fffff : /soc/arm,smmu-anoc2@16c0000
01d0101c-01d0101f : sp2soc_irq_status
01d01024-01d01027 : sp2soc_irq_clr
01d01028-01d0102b : sp2soc_irq_mask
.
.
.
0caa0000-0caa3fff : jpeg_hw
0caa4000-0caa47ff : fd_core
0caa5000-0caa53ff : fd_misc
0cd00000-0cd3ffff : /soc/arm,smmu-mmss@cd00000
17817000-17817fff : msm-watchdog
17900000-1790dfff : msm-gladiator-erp
80000000-857fffff : System RAM
80080000-817fffff : Kernel code
82330000-82945fff : Kernel data
88f00000-8aafffff : System RAM
95300000-17e3bffff : System RAM

​
/proc/vmallocinfo

This file shows detailed information about virtual address allocation through vmalloc or ioremap.
  • To view the contents of this file, run the following command: 
    cat /proc/vmallocinfo
    Sample output: 
    0xbf000000-0xbf002000     8192            module_alloc_update_bounds+0xc/0x5c     pages=1 vmalloc
0xbf004000-0xbf008000     16384          module_alloc_update_bounds+0xc/0x5c     pages=3 vmalloc
0xee800000-0xef800000    16777216   iotable_init+0x0/0xb0   phys=36800000   ioremap
0xf0000000-0xf0002000     8192            of_iomap+0x30/0x38   ioremap
0xf0002000-0xf0004000     8192            of_iomap+0x30/0x38   ioremap
0xf0004000-0xf000c000     32768          gen_pool_add_virt+0x48/0xb8   pages= 7    vmalloc
0xf000c000-0xf000e000     8192            msm_pm_setup_saved_state+0xcc/0x1bc   ioremap
………………………
0xf0174000-0xf0176000    8192            msm_cpu_status_probe+0xd8/0x20c   ioremap
0xf0f24000-0xf0f28000      16384         _kgsl_sharedmem_page_alloc+0xa0/0x41c   pages=3   vmalloc
0xf0f39000-0xf0f3e000      20480         _kgsl_sharedmem_page_alloc+0xa0/0x41c   pages=4   vmalloc
0xf0f61000-0xf0f66000      20480         _kgsl_sharedmem_page_alloc+0xa0/0x41c   pages=4   vmalloc
……….
0xfa400000-0xfa600000    2097152     iotable_init+0x0/0xb0 phys=fa00000    ioremap
0xfa71e000-0xfa71f000     4096            iotable_init+0x0/0xb0 phys=f991e000     ioremap
0xfefd8000-0xff000000     163840        pcpu_get_vm_areas+0x0/0x56c  vmalloc

​
memblock interface

The memblock interface on the debugfs file system provides details about the available and reserved memory regions in the system. This interface provides the following files:

​
/sys/kernel/debug/memblock/memory

This file provides information about all the available memory regions (HLOS and non-HLOS) visible to the Linux kernel. To determine the overall memory accessible to the kernel, calculate the difference between the start and end addresses of each of the region and sum these values to get the total occupied RAM. The remaining memory, which is the difference between the RAM size of the device and the occupied RAM, is the non-HLOS memory or the memory occupied by other subsystems.
  • To view the contents of this file, run the following command: 
    cat /sys/kernel/debug/memblock/memory
    Sample output: 
    
0: 0x0000000080000000..0x00000000857fffff

1: 0x0000000088f00000..0x000000008aafffff

2: 0x0000000095300000..0x000000017e3bffff

​
/sys/kernel/debug/memblock/reserved

This file provides information about all the reserved memory regions in the system.
  • To view the contents of this file, run the following command: 
    cat /sys/kernel/debug/memblock/reserved
    Sample output: 
    0: 0x0000000080080000..0x0000000082944fff
1: 0x0000000083200000..0x0000000083259bb4
2: 0x0000000083400000..0x00000000839506c9
3: 0x00000000f5800000..0x00000000ffbfffff
4: 0x00000000ffff7000..0x00000000ffffefff
5: 0x00000000ffffff40..0x00000000ffffff77
6: 0x00000000ffffff80..0x00000000ffffffb7
7: 0x00000000ffffffc0..0x00000000fffffff7
8: 0x0000000179258000..0x000000017d9fffff
9: 0x000000017da17000..0x000000017da1ffff
10: 0x000000017da20e00..0x000000017da26fff
11: 0x000000017da27300..0x000000017da2735f
12: 0x000000017da27380..0x000000017da273df
13: 0x000000017da27400..0x000000017da2755f
14: 0x000000017da27580..0x000000017da27587
15: 0x000000017da275c0..0x000000017da275c7
16: 0x000000017da29600..0x000000017da29924
17: 0x000000017da29940..0x000000017da29c64
18: 0x000000017da29c80..0x000000017da29fa4
19: 0x000000017da29fac..0x000000017da2a3f8
20: 0x000000017da2a3fc..0x000000017da2a42e
21: 0x000000017da2a430..0x000000017da2a45e
22: 0x000000017da2a460..0x000000017e3bffff

​
Debug memory leak issues

To debug kernel memory leak issues, enable the following configuration options:
  • CONFIG_DEBUG_KMEMLEAK=y
  • CONFIG_DEBUG_KMEMLEAK_MEM_POOL_SIZE=4000
  • CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF=y
By default, a kernel thread scans the memory every 10 minutes and prints the number of new unreferenced objects found. For example, 
unreferenced object 0xec26f000 (size 4096):

comm "Binder_2", pid 4592, jiffies 8848 (age 336.710s)

hex dump (first 32 bytes):

ec 4d f8 c0 02 00 00 00 00 00 00 00 00 00 00 00 .M..............

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

backtrace:

[<c0126f70>] kmem_cache_alloc_trace+0x17c/0x238

[<c03059c4>] ddl_client_transact+0xd0/0x158

[<c0314a3c>] ddl_open+0x4c/0x194

[<c0302288>] vcd_init_client_context+0x14/0x9c

[<c02ffd10>] vcd_open_in_ready+0x3c/0x94

[<c02fd31c>] vcd_open+0x214/0x274

[<c031b244>] vid_dec_open_client+0x1d0/0x288

[<c031b3ac>] vid_dec_open+0x30/0x7c

[<c012ff70>] chrdev_open+0x10c/0x134

[<c012aa7c>] __dentry_open.isra.12+0x190/0x29c

[<c0138a78>] do_last.isra.29+0x690/0x6c0

[<c0138c70>] path_openat+0xb8/0x35c

[<c0138ff4>] do_filp_open+0x2c/0x78

[<c012b7a0>] do_sys_open+0xd8/0x170

[<c000df20>] ret_fast_syscall+0x0/0x30

[<ffffffff>] 0xffffffff
Disable the KMEMLEAK option at boot time by passing KMEMLEAK=off on the kernel command line. For more information about the kmemleak.txt file, see kernel documentation. The following extra kernel configuration options are available to track the allocator of each page of memory:
  • CONFIG_PAGE_OWNER
  • CONFIG_PAGE_OWNER_ENABLE_DEFAULT
  • CONFIG_PAGE_EXTENSION
Enabling these options can help identify several allocations, which may indicate a memory leak issue.

​
Identify memory corruption issues

Enable the following kernel configuration options to identify memory corruption issues:
  • CONFIG_PAGE_POISONING
  • CONFIG_SLUB_DEBUG_ON
  • CONFIG_DEBUG_LIST
  • CONFIG_SLUB_DEBUG
Sample log: 
BUG <slab cache affected>: <What went wrong>
-----------------------------------------------
INFO: <corruption start>-<corruption_end> <more info>
INFO: Slab <address> <slab information>
INFO: Object <address> <object information>
INFO: Allocated in <kernel function> age=<jiffies since alloc> cpu=<allocated by cpu> pid=<pid of the process>
INFO: Freed in <kernel function> age=<jiffies since free> cpu=<freed by cpu> pid=<pid of the process>
For more information about slub debugging, see the Documentation/vm/slub.txt file available in the kernel documentation.

​
Out of memory

When the system fails to assign a page, the kernel logs display a message such as the following: 
<4>[12146.861355] Thread-430: page allocation failure: order:0, mode:0x10d2

<CALL STACK>

<4>[12146.951687] Mem-info:

<4>[12146.953909] Normal per-cpu:

<4>[12146.956686] CPU 0: hi: 186, btch: 31 usd: 61

<4>[12146.961489] CPU 1: hi: 186, btch: 31 usd: 0

<4>[12146.966235] HighMem per-cpu:

<4>[12146.969122] CPU 0: hi: 186, btch: 31 usd: 54

<4>[12146.973877] CPU 1: hi: 186, btch: 31 usd: 0

…….

<4>[12147.010770] Normal free:53192kB min:3508kB low:4384kB high:5260kB ……..

<4>[12147.050805] lowmem_reserve[]: 0 9022 9022

<4>[12147.054610] HighMem free:153360kB min:512kB low:1824kB high:3140kB ………

<4>[12147.095453] lowmem_reserve[]: 0 0 0

<4>[12147.098617] Normal: 118*4kB 232*8kB 161*16kB 110*32kB 34*64kB 9*128kB 10*256kB 6*512kB 7*1024kB 6*2048kB 4*4096kB = 53224kB

<4>[12147.115455] HighMem: 2774*4kB 11769*8kB 3005*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 153360kB

………………
These messages indicate that the system couldn’t assign the requested page. The top line of the log message provides several details about the out-of-memory issue. For example, in the following log: 
<4>[1214.855361] Thread-4: page allocation failure: order:2, mode:0x10d2
  • order: Indicates the size of the page. In this example, 2 2 x PAGE_SIZE (4 K) = 16 K
    • Linux uses a buddy allocator that allocates pages in powers of 2.
      • The maximum size of the buddy allocator is the order of 10 = 2 10 x 4 kB = 4 MB.
      • For allocation > 4 MB, use an alternate allocation method such as the contiguous memory allocator (CMA).
    • For failure of higher-order allocations, examine whether the memory can be virtually contiguous, instead of being physically contiguous.
  • mode: Indicates the type of page requested
    • mode provides information about get free pages (GFP) flags. In this example, mode is 0x10d2.
    • mode is the result of OR operation on all the GFP flags in the allocation.
  • Pages available in the system A page allocation failure message prints details about the size of pages that were available in the system. 
    Normal: 118*4kB 232*8kB 161*16kB 110*32kB 34*64kB 9*128kB 10*256kB 6*512kB 7*1024kB 6*2048kB 4*4096kB = 53224kB
HighMem: 2774*4kB 11769*8kB 3005*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 153360kB

​
IOMMU page fault

Input-output memory management unit (IOMMU), also known as the system memory management unit (SMMU), performs memory management functions on behalf of subsystems that don’t have their own MMU. The IOMMU hardware block allows physically noncontiguous pages to support virtually contiguous memory. The memory translation logic in the IOMMU is the same as the logic in the CPU MMU. The IOMMU page fault is the most common IOMMU issue. The fault occurs when the requested page is mapped in the page table but isn’t found in the memory. The fault handler receives the context bank instance of the IOMMU and dumps the registers for this context. The following log indicates an IOMMU page fault. 
[ 47.228992] msm_iommu_v1: Unexpected IOMMU page fault!

[ 47.233115] msm_iommu_v1: name = mdp_iommu

[ 47.237238] msm_iommu_v1: context = mdp_0 (0)

[ 47.241507] msm_iommu_v1: Interesting registers:

[ 47.246149] msm_iommu_v1: FAR = 0000000000000000

[ 47.250970] msm_iommu_v1: PAR = 0000000000000000

[ 47.255834] msm_iommu_v1: FSR = 00000002 [TF ]

[ 47.260540] msm_iommu_v1: FSYNR0 = 000005a1 FSYNR1 = 00030005

[ 47.266528] msm_iommu_v1: TTBR0 = 0000000071a28000

[ 47.271370] msm_iommu_v1: TTBR1 = 0000000000000000

[ 47.276248] msm_iommu_v1: SCTLR = 00001043 ACTLR = 70000000

[ 47.282221] msm_iommu_v1: CBAR = 00000000 CBFRSYNRA = 00000000

[ 47.288521] msm_iommu_v1: PRRR = ff0a81a8 NMRR = 40e040e0

[ 47.294461] msm_iommu_v1: NOTE: Value actually unknown for CBAR

[ 47.300394] msm_iommu_v1: NOTE: Value actually unknown for CBFRSYNRA

[ 47.306717] msm_iommu_v1: Page table in DDR shows PA = 0
The following table describes the fields captured in the log message. Table: Information in IOMMU page fault log
ItemDescription
nameName of the hardware block that caused the fault.
FARFault address register (FAR) indicates the address at which the fault occurred.
FSRFault status register (FSR) indicates the following:
  • Translation fault (TF)
  • Access permission fault (APF)
  • Stalled status (SS)
The FSR is one of the most important registers in IOMMU debugging. This register has read/write-clear access. The read operation on this register reads the value in the register while the write operation clears the bits corresponding to 1s in the written data and leaves the bits corresponding to 0s unchanged. This process prevents inadvertent clearing of new faults when writing the register to clear an old fault. Some useful bits in this register are: Table: Bits in fault status register
BitDescription
[Bit 1]: TFTranslation fault (invalid page table entry)
[Bit 2]: AFFAccess fault
[Bit 3]: APFPermission fault (write to read only region)
[Bit 4]: TLBMFTLB miss fault
[Bit 5]: HTWDEEFHardware table walk decode error external fault
[Bit 6]: HTWSEEFHardware table walk subordinate error external fault
[Bit 7]: MHFMany hits in TLB
[Bit 16]: SLSecond-level fault (fault occurred in second level of page table)
[Bit 30]: SSStalled status
[Bit 31]: MULTIMultiple faults
The TF, APF, and SL flags indicate normal operation, whereas the TLBMF, HTWDEEF, HTWSEEF, and MHF flags indicate that there is an issue.

​
IOMMU page table

The IOMMU page table dump provides a faulting address from the FAR and the register dump in the kernel log. This faulting address represents the virtual address, and you can acquire the corresponding physical address from the page table. From the page table dump, you can identify whether the requested address is mapped or not mapped. Each IOMMU domain has a page table, and the dump includes page tables for each of the domains. Currently, there are six domains. The following is the sample dump of the Domain: 2 page table. 
Domain: 2 [L2 cache redirect for page tables is OFF]

0x00000000--0x0001ffff [0x00020000] [UNMAPPED]

0x00020000--0x01807fff [0x017e8000] A:0x82a8e000--0x84275fff [0x017e8000] [R/W][4K]

0x01808000--0x01939fff [0x00132000] A:0xf0c24000--0xf0d55fff [0x00132000] [R/W][4K]

0x0193a000--0x0199ffff [0x00066000] A:0xf13fa000--0xf145ffff [0x00066000] [R/W][4K]

0x019a0000--0x01e85fff [0x004e6000] A:0xf966e000--0xf9b53fff [0x004e6000] [R/W][4K]

0x01e86000--0x01ffffff [0x0017a000] [UNMAPPED]

0x02000000--0x02feffff [0x00ff0000] A:0xf5a22000--0xf6a11fff [0x00ff0000] [R/W][4K]
In this example, the first column represents the virtual address, the second column represents the number of bytes in the corresponding region of contiguous physical addresses, and the third column represents the physical addresses. The permissions are also mentioned for each of these regions.

​
Memory map

For more information about the memory map, see the latest Release Notes.

​
Stack corruption

Wrong coding logic accesses memory locations in the stack, leading to changes in values at those memory locations, causing stack corruption. Stack corruption can occur in the following ways:
  • Due to bad code logic, the program consumes all the stack memory and writes memory beyond the stack boundaries, resulting in a stack overflow.
  • Accessing an array that’s out of bounds.
  • An undefined or freed pointer that points at a stack address.
  • Corrupted return address of a caller function.
To identify the stack corruption issues, enable the following kernel configuration options:
  • CONFIG_STACKPROTECTOR
  • CONFIG_STACKPROTECTOR_STRONG
The Kernel address sanitizer (KASAN) utility also helps in identifying some stack corruption issues.