Fine tune zfs for Solaris application and database servers
Memory Management Between ZFS and Applications in Oracle Solaris 11.x (Doc ID 1663862.1)
APPLIES TO: Solaris Operating System - Version 11.1 and later
The user_reserve_hint_pct Parameter
Solaris 11.2 and Solaris 11.1 SRU 20.5 or newer includes a new user_reserve_hint_pct tunable parameter to provide a hint to the system about application memory usage. This hint is used to limit growth of the ZFS ARC cache so that more memory stays available for applications.
if user_reserve_hint_pct is tuned appropriately, memory that is returned to the freemem pool is less likely to be reused by the kernel. This, in turn, allows administrators to keep a reserve of free memory for future applications demands by restricting growth of the ZFS ARC cache. While the motivation for using this parameter might include faster application startup and dynamic reconfiguration of memory boards, its primary use is to ensure that large memory pages stay available for demanding applications, such as databases.
It's very important to calculate a suitable value for user_reserve_hint_pct to avoid this situation. See 'How to calculate a suitable value' heading below.
A scenario where user_reserve_hint_pct is set but the application has consumed more memory than this value. This is permitted because user_reserve_hint_pct is only a 'hint' for user-land applications and not a hard limit. If applications use more than the pre-defined value this will usually lead to system performance and hang issues.
Description:
Informs the system about how much memory is reserved for application use, and therefore limits how much memory can be used by the ZFS ARC cache as the cache increases over time.
By means of this parameter, administrators can maintain a large reserve of available free memory for future application demands. The user_reserve_hint_pct parameter is intended to be used in place of the zfs_arc_max parameter to restrict the growth of the ZFS ARC cache.
Data Type
Unsigned Integer (64-bit)
Default
0 (unset)
Range
0 - 99%of physical memory
The minimum size of the ZFS ARC is 64MB on systems with physmem <= 16GB of RAM or 0.5% of physmem for system >16GB of RAM.
Units
percent. Values should be positive whole decimal integers. Negative numbers or floating points are not permitted.
Dynamic?
yes
Validation
Yes, the range is validated.
When to Change :
For upward adjustments, increase the value if the initial value is determined to be insufficient over time for application requirements, or if application demand increases on the system. Perform this adjustment only within a scheduled system maintenance window. After you have changed the value, reboot the system.
For downward adjustments, decrease the value if allowed by application requirements. Make sure to use decrease the value only by small amounts, no greater than 5% at a time.
How to calculate a suitable value :
Calculations can be performed one of two ways.
1) If the size of the ARC should be capped; ie if we're converting a previous zfs_arc_max value to user_reserve_hint_pct, use:
user_reserve_hint_pct = USER_RESERVE_HINT_MAX - (((Kernel + Defdump prealloc + ZFS Metadata + desired zfs_arc_max) / Total (physmem))*100)
USER_RESERVE_HINT_MAX = 99. For this example zfs_arc_max = 1GB. All values are specified in Megabytes.
99-(((3276 + 925 + 109 + 1024)/16384)*100) = 66.44 (66 rounded down)
2) If the amount of memory needed by applications is known, calculate it this way:
user_reserve_hint_pct = Application Demand
eg: The following assigns 8GB to apps/db leaving the remaining for the ARC and Kernel:
user_reserve_hint_pct = (8192/16384)*100) = 50
Note: Memory must remain for the Kernel, Deferred Dump, etc to avoid system performance issues.
If the value had not been previously set, the default is zero so the -f flag must be used for the initial setting on a live system.
Down Load set_user_reserve.sh
# ./set_user_reserve.sh -p 60
./set_user_reserve.sh: 60 greater that 0; use -f to force upward adjustment
The following can take several minutes to complete depending on the current size of the ARC.
# ./set_user_reserve.sh -fp 60
Adjusting user_reserve_hint_pct from 0 to 60
Monday, March 30, 2015 04:59:47 PM BST : waiting for current value : 45 to grow to target : 60
Adjustment of user_reserve_hint_pct to 60 successful.
Make the setting persistent across reboot by adding to /etc/system
* Tuning based on MOS note 1663861.1, script version 1.0
* added Monday, March 30, 2015 05:09:53 PM BST by system administrator : <me>
set user_reserve_hint_pct=60
# vi /etc/system
(Add the 4 lines. Save and quit vi)
# tail /etc/system
*
* To set a variable named 'debug' in the module named 'test_module'
*
* set test_module:debug = 0x13
* Tuning based on MOS note 1663861.1, script version 1.0
* added Monday, March 30, 2015 05:09:53 PM BST by system administrator : <me>
set user_reserve_hint_pct=60
~Judi~
~
zfs arc memory usage
zfs memory consumption
zfs memory usage solaris
zfs arc cache tuning
zfs arc tuning
zfs memory tuning
zfs memory management
manage zfs memory usage
fine tune zfs memory usage
zfs performance
zpool
zfs share
zfs set sharenfs
zfs share nfs
~
Memory Management Between ZFS and Applications in Oracle Solaris 11.x (Doc ID 1663862.1)
APPLIES TO: Solaris Operating System - Version 11.1 and later
The user_reserve_hint_pct Parameter
Solaris 11.2 and Solaris 11.1 SRU 20.5 or newer includes a new user_reserve_hint_pct tunable parameter to provide a hint to the system about application memory usage. This hint is used to limit growth of the ZFS ARC cache so that more memory stays available for applications.
if user_reserve_hint_pct is tuned appropriately, memory that is returned to the freemem pool is less likely to be reused by the kernel. This, in turn, allows administrators to keep a reserve of free memory for future applications demands by restricting growth of the ZFS ARC cache. While the motivation for using this parameter might include faster application startup and dynamic reconfiguration of memory boards, its primary use is to ensure that large memory pages stay available for demanding applications, such as databases.
It's very important to calculate a suitable value for user_reserve_hint_pct to avoid this situation. See 'How to calculate a suitable value' heading below.
A scenario where user_reserve_hint_pct is set but the application has consumed more memory than this value. This is permitted because user_reserve_hint_pct is only a 'hint' for user-land applications and not a hard limit. If applications use more than the pre-defined value this will usually lead to system performance and hang issues.
Description:
Informs the system about how much memory is reserved for application use, and therefore limits how much memory can be used by the ZFS ARC cache as the cache increases over time.
By means of this parameter, administrators can maintain a large reserve of available free memory for future application demands. The user_reserve_hint_pct parameter is intended to be used in place of the zfs_arc_max parameter to restrict the growth of the ZFS ARC cache.
Data Type
Unsigned Integer (64-bit)
Default
0 (unset)
Range
0 - 99%of physical memory
The minimum size of the ZFS ARC is 64MB on systems with physmem <= 16GB of RAM or 0.5% of physmem for system >16GB of RAM.
Units
percent. Values should be positive whole decimal integers. Negative numbers or floating points are not permitted.
Dynamic?
yes
Validation
Yes, the range is validated.
When to Change :
For upward adjustments, increase the value if the initial value is determined to be insufficient over time for application requirements, or if application demand increases on the system. Perform this adjustment only within a scheduled system maintenance window. After you have changed the value, reboot the system.
For downward adjustments, decrease the value if allowed by application requirements. Make sure to use decrease the value only by small amounts, no greater than 5% at a time.
How to calculate a suitable value :
Calculations can be performed one of two ways.
1) If the size of the ARC should be capped; ie if we're converting a previous zfs_arc_max value to user_reserve_hint_pct, use:
user_reserve_hint_pct = USER_RESERVE_HINT_MAX - (((Kernel + Defdump prealloc + ZFS Metadata + desired zfs_arc_max) / Total (physmem))*100)
USER_RESERVE_HINT_MAX = 99. For this example zfs_arc_max = 1GB. All values are specified in Megabytes.
99-(((3276 + 925 + 109 + 1024)/16384)*100) = 66.44 (66 rounded down)
2) If the amount of memory needed by applications is known, calculate it this way:
user_reserve_hint_pct = Application Demand
eg: The following assigns 8GB to apps/db leaving the remaining for the ARC and Kernel:
user_reserve_hint_pct = (8192/16384)*100) = 50
Note: Memory must remain for the Kernel, Deferred Dump, etc to avoid system performance issues.
If the value had not been previously set, the default is zero so the -f flag must be used for the initial setting on a live system.
Down Load set_user_reserve.sh
# ./set_user_reserve.sh -p 60
./set_user_reserve.sh: 60 greater that 0; use -f to force upward adjustment
The following can take several minutes to complete depending on the current size of the ARC.
# ./set_user_reserve.sh -fp 60
Adjusting user_reserve_hint_pct from 0 to 60
Monday, March 30, 2015 04:59:47 PM BST : waiting for current value : 45 to grow to target : 60
Adjustment of user_reserve_hint_pct to 60 successful.
Make the setting persistent across reboot by adding to /etc/system
* Tuning based on MOS note 1663861.1, script version 1.0
* added Monday, March 30, 2015 05:09:53 PM BST by system administrator : <me>
set user_reserve_hint_pct=60
# vi /etc/system
(Add the 4 lines. Save and quit vi)
# tail /etc/system
*
* To set a variable named 'debug' in the module named 'test_module'
*
* set test_module:debug = 0x13
* Tuning based on MOS note 1663861.1, script version 1.0
* added Monday, March 30, 2015 05:09:53 PM BST by system administrator : <me>
set user_reserve_hint_pct=60
~Judi~
~
zfs arc memory usage
zfs memory consumption
zfs memory usage solaris
zfs arc cache tuning
zfs arc tuning
zfs memory tuning
zfs memory management
manage zfs memory usage
fine tune zfs memory usage
zfs performance
zpool
zfs share
zfs set sharenfs
zfs share nfs
~
