Steps to configure hugepages in RHEL/CentOS 7/8

In this article I will share the steps to configure hugepages and also create hugepages pool to allot pages to the application based on the request in Red Hat and CentOS 7 Linux. We will configure hugepages using hugeadm tool

Configure hugepages in Red Hat/CentOS 7/8We can configure hugepages by using the kernel tunable vm.nr_hugepages

Step 1: Check huge pages status

Check the existing value of vm.nr_hugepages

[root@centos8 ~]# sysctl -a | grep vm.nr_hugepages
vm.nr_hugepages = 0

So looks like vm.nr_hugepages is disabled in my CentOS 8 server.

• Do you know the difference between AnonHugePages and vm.nr_hugepages.
• Here AnonHugePages is referred as Transparent HugePages and vm.nr_hugepages is referred as explicit transparent HugePages.

Anyhow these topics are not the agenda here, we know our hugepages are disabled so we will enable and configure hugepages for 2048 so we will reserve 2MB of hugepages memory in the kernel

 

Step 2: Update vm.nr_hugepages in /etc/sysctl.conf

Edit /etc/sysctl.conf file to configure hugepages and specify the number of hugepages in the vm.nr_hugepages :

vm.nr_hugepages = 2048

 

Step 3: Refresh kernel parameters

Next we must refresh the kernel parameters using sysctl -p to activate the new hugepages value
[root@centos8 ~]# sysctl -p
Verify the new hugepage reservation
[root@centos8 ~]# sysctl -a | grep vm.nr_hugepagesvm.nr_hugepages = 2048

Step 4: Reboot

• In one way we can say that a reboot is not mandatory but strongly recommended. Because we know that vm.nr_hugepages is now showing us the new reservation.
• But since the provided amount of memory must be reserved by the kernel we have to make sure that there are enough contiguous memory available for reservation
• You can read more at : Beginners guide to Linux memory Management in Linux
• This is only possible at early boot up stage when none of the applications or system process would be using the system memory and kernel can easily allot provided value for hugepages
• Hence a reboot is strongly recommended in production environment after you configure hugepages

[root@centos8 ~]# systemctl reboot

 

Configure hugepages pool in Red Hat/CentOS 7

• The hugeadm utility is provided by package libhugetlbfs-utils and can be used to displays and configures the systems huge page pools.
• The size of the pools is set as a minimum and maximum threshold.
• The minimum value is allocated up front by the kernel and guaranteed to remain as hugepages until the pool is shrunk.
• If a maximum is set, the system will dynamically allocate pages if applications request more hugepages than the minimum size of the pool.
• There is no guarantee that more pages than this minimum pool size can be allocated.

 

Step 1: Install libhugetlbfs-utils

The hugeadm utility is provided by package libhugetlbfs-utils and can be used to displays and configures the systems huge page pools. The size of the pools is set as a minimum and maximum threshold.

Execute the following command to install libhugetlbfs-utils package.

NOTE:

On RHEL system you must have an active subscription to RHN or you can configure a local offline repository using which "yum" package manager can install the provided rpm and it's dependencies.

# yum -y install libhugetlbfs libhugetlbfs-utils

 

Step 2: Check supported HugePage Size

Before we configure hugepages pool, list all page sizes supported by the system, even if no pool is available

# hugeadm --page-sizes-all
2097152
1073741824

So currently our Red Hat and CentOS 7 server supportshugepage size of 2MB and 1GB

This command displays every page size supported by the system and has a pool configured

# hugeadm --page-sizes
2097152

So currently our server is configured to use 2MB hugepage size.

You can also get this information from /proc/cpuinfo. You should also know how to check if your CPU supports hugepages# grep Hugepagesize /proc/meminfo
Hugepagesize: 
   2048 kB

Below command displays all active mount points for hugetlbfs

# hugeadm --list-all-mounts
Mount Point          Options
/dev/hugepages       rw,relatime

This displays the Minimum, Current and Maximum number of huge pages in the pool for each pagesize supported by the system.

• The minimum value is allocated up front by the kernel and guaranteed to remain as hugepages until the pool is shrunk.
• If a maximum is set, the system will dynamically allocate pages if applications request more hugepages than the minimum size of the pool.
• There is no guarantee that more pages than this minimum pool size can be allocated.
• Since our hugepage size is 2MB you can see the hugepage pool for 2097152 while for the other hugepage size, no pools are configured

# hugeadm --pool-list
      Size  Minimum  Current  Maximum  Default
   2097152    12850    12850    12850        *
1073741824        0        0        0

Here,

Minimum -	This is the size of the static pool and there will always be at 
			least this number of hugepages in use by the system, either by 
			applications or kept by the kernel in a reserved  pool.  
			
Current -	This value is the number of hugepages currently in use, either 
			by applications or stored on the kernels free list. 
			
Maximum -	This value is the largest number of hugepages that can be in 
			use at any given time.

 

Step 2: Configure HugePages using hugeadm

The (--pool-pages-min) option sets or adjusts the Minimum number of hugepages in the pool for pagesize size. size may be specified in bytes or in kilobytes, megabytes, or gigabytes by appending K, M, or G respectively, or as DEFAULT, which uses the system's default huge page size for size.

Here we are setting 4096 as the minimum number of pages for 2MB HugePage.

# hugeadm --pool-pages-min 2MB:4096

 

The (--pool-pages-max) option sets or adjusts the Maximum number of hugepages.
NOTE: While the Minimum number of pages are guaranteed to be available to applications, there is not guarantee that the system can allocate the pages on demand when the number of huge pages requested by applications is between the Minimum and Maximum pool sizes.

# hugeadm --pool-pages-max 2MB:12850

 

Step 3: Verify the new hugepages pool

Verify the pool list again

# hugeadm --pool-list
      Size  Minimum  Current  Maximum  Default
   2097152     4096     4096    12850        *
1073741824        0        0        0