chmem: add support for dynamic (de)configuration of hotplug memory
Extend chmem to use the new s390 kernel interface for configuring and
deconfiguring hotpluggable memory blocks, with memmap-on-memory support.
Background:
On s390, memmap-on-memory was introduced to ensure that the struct page
array (metadata) for hotpluggable standby memory is allocated from the
memory block itself. This allowed hot-add operations even under memory
pressure, particularly in cases with a strong imbalance between
boot-time online memory and standby memory.
The original design, however, had few limitations:
* All hotpluggable standby memory was added at boot.
* The use of memmap-on-memory was global and static, decided at boot
time. Either all standby blocks used it, or none of them did.
* memmap-on-memory choice could not be changed at runtime, limiting
flexibility. For example, when continuous physical memory was required
later across memory blocks.
The s390 kernel
ff18dcb19aab ("s390/sclp: Add support for dynamic
(de)configuration of memory") no longer pre-adds all standby memory at
boot. Instead, users must explicitly configure a block before it can be
used for online/offline actions. At configuration time, users can
dynamically decide whether to use optional memmap-on-memory for each
memory block, where value of 1 allocates metadata (such as struct pages
array) from the hotplug memory itself, enabling hot-add operations even
under memory pressure. A value of 0 stores metadata in regular system
memory and enables continuous physical memory across memory blocks.
s390 kernel sysfs interface to configure/deconfigure memory with
memmap-on-memory support looks as shown below:
1. Configure memory
echo 1 > /sys/firmware/memory/memoryX/config
2. Deconfigure memory
echo 0 > /sys/firmware/memory/memoryX/config
3. Enable memmap-on-memory
echo 1 > /sys/firmware/memory/memoryX/memmap_on_memory
4. Disable memmap-on-memory
echo 0 > /sys/firmware/memory/memoryX/memmap_on_memory
* Initial memory layout:
lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP-ON-MEMORY
RANGE SIZE STATE BLOCK CONFIGURED MEMMAP-ON-MEMORY
0x00000000-0x7fffffff 2G online 0-15 yes no
0x80000000-0xffffffff 2G offline 16-31 no yes
Memory block size: 128M
Total online memory: 2G
Total offline memory: 2G
Memmap on memory parameter: yes
* Configure memory with memmap-on-memory.
chmem -c 128M -m 1
lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP-ON-MEMORY
RANGE SIZE STATE BLOCK CONFIGURED MEMMAP-ON-MEMORY
0x00000000-0x7fffffff 2G online 0-15 yes no
0x80000000-0x87ffffff 128M offline 16 yes yes
0x88000000-0xffffffff 1.9G offline 17-31 no yes
Memory block size: 128M
Total online memory: 2G
Total offline memory: 2G
Memmap on memory parameter: yes
* Deconfigure memory
chmem -g 128M
lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP-ON-MEMORY
RANGE SIZE STATE BLOCK CONFIGURED MEMMAP-ON-MEMORY
0x00000000-0x7fffffff 2G online 0-15 yes no
0x80000000-0xffffffff 2G offline 16-31 no yes
Memory block size: 128M
Total online memory: 2G
Total offline memory: 2G
Memmap on memory parameter: yes
* Online memory.
If the memory is in deconfigured state, configure and online it.
chmem -e 128M -v
Memory Block 16 (0x0000000080000000-0x0000000087ffffff) configured
Memory Block 16 (0x0000000080000000-0x0000000087ffffff) enabled
lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP-ON-MEMORY
RANGE SIZE STATE BLOCK CONFIGURED MEMMAP-ON-MEMORY
0x00000000-0x7fffffff 2G online 0-15 yes no
0x80000000-0x87ffffff 128M online 16 yes yes
0x88000000-0xffffffff 1.9G offline 17-31 no yes
Memory block size: 128M
Total online memory: 2.1G
Total offline memory: 1.9G
Memmap on memory parameter: yes
* Offline memory
If the memory is in online state, then offline it and deconfigure it.
chmem -d 128M -v
Memory Block 16 (0x0000000080000000-0x0000000087ffffff) disabled
Memory Block 16 (0x0000000080000000-0x0000000087ffffff) deconfigured
lsmem -o RANGE,SIZE,STATE,BLOCK,CONFIGURED,MEMMAP-ON-MEMORY
RANGE SIZE STATE BLOCK CONFIGURED MEMMAP-ON-MEMORY
0x00000000-0x7fffffff 2G online 0-15 yes no
0x80000000-0xffffffff 2G offline 16-31 no yes
Memory block size: 128M
Total online memory: 2G
Total offline memory: 2G
Memmap on memory parameter: yes
Just like online and offline actions, memory configuration and
deconfiguration can be controlled through similar options. Also,
memmap-on-memory setting can be changed, only when the memory block is
in deconfigured state. This means, it is usable only via --configure
option.
Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>