This is a depressing – and all-too-common – scenario: a user runs the free
command and opens a support case because he sees zero “free” memory on the server.
For years I’ve been trying to explain to users – developers, application engineers – how memory allocation works in Linux. It’s an utterly futile endeavor. Sometimes I just want to soft-link the free
command to exit
.
Starting with RHEL 7, we’ve been blessed with an updated version of free
that now adds the Available
column to the output. This helps sometimes, but users are still confused as they don’t understand the difference between “free” and “available”.
Here’s output of free
on RHEL 6:
total used free shared buffers cached
Mem: 62G 61G 1.1G 677M 951M 39G
-/+ buffers/cache: 21G 41G
root@hecuba# free -h
total used free shared buffers cached
Mem: 62G 61G 1.1G 677M 951M 39G
-/+ buffers/cache: 21G 41G
Swap: 8.0G 804M 7.2G
root@hecuba# free -h
total used free shared buffers cached
Mem: 62G 61G 1.1G 677M 951M 39G
-/+ buffers/cache: 21G 41G
Swap: 8.0G 804M 7.2G
And here’s the output of free
on RHEL 7:
[root@kassandra~]# free -h
total used free shared buff/cache available
Mem: 7.6G 478M 400M 9.5M 6.7G 6.8G
[root@kassandra~]# free -h
total used free shared buff/cache available
Mem: 7.6G 478M 400M 9.5M 6.7G 6.8G
Swap: 2.0G 0B 2.0G
[root@kassandra~]# free -h
total used free shared buff/cache available
Mem: 7.6G 478M 400M 9.5M 6.7G 6.8G
Swap: 2.0G 0B 2.0G
I thought this was a problem of presentation and so I set out to write a quick script that would detail current system memory utilization in a way that your average developer can understand.
It also occurred to me that instead of relying on local system tools to gather this information, I should use snmpwalk
. This would allow for wider access and more standardized presentation. And so I picked a major monitoring software vendor – Solarwinds – and decided to stick with its standards.
According to Solarwinds documentation, the percentage of “used memory” is calculated like so:
Used Memory = (Total RAM in machine - Total RAM Available - Cached Memory - RAM Buffered) / Total RAM * 100
Used Memory = (Total RAM in machine - Total RAM Available - Cached Memory - RAM Buffered) / Total RAM * 100
Used Memory = (Total RAM in machine - Total RAM Available - Cached Memory - RAM Buffered) / Total RAM * 100
Here’s a practical example showing how to calculate the percentage of used memory using SNMP data:
s=<snmp_community_string>
total_ram='1.3.6.1.4.1.2021.4.5'
total_ram_available='1.3.6.1.4.1.2021.4.6'
total_ram_buffered='1.3.6.1.4.1.2021.4.14'
total_cached_memory='1.3.6.1.4.1.2021.4.15'
# Configure snmpwalk function
snmpwalk -v 2c -c $s $h $@
# Configure output formatting
awk -F: '{print $NF}' | grep -oP '[0-9]{1,}'
echo "Used Memory: $(echo "scale=0;($(walk ${total_ram} | chomp)\
> -$(walk ${total_cached_memory} | chomp)\
> -$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)%"
h=<target_host>
s=<snmp_community_string>
# Define relevant OIDs
total_ram='1.3.6.1.4.1.2021.4.5'
total_ram_available='1.3.6.1.4.1.2021.4.6'
total_ram_buffered='1.3.6.1.4.1.2021.4.14'
total_cached_memory='1.3.6.1.4.1.2021.4.15'
# Configure snmpwalk function
walk() {
snmpwalk -v 2c -c $s $h $@
}
# Configure output formatting
chomp() {
awk -F: '{print $NF}' | grep -oP '[0-9]{1,}'
}
[root@devsw01 ~]#
echo "Used Memory: $(echo "scale=0;($(walk ${total_ram} | chomp)\
> -$(walk ${total_cached_memory} | chomp)\
> -$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)%"
Used Memory: 32%
h=<target_host>
s=<snmp_community_string>
# Define relevant OIDs
total_ram='1.3.6.1.4.1.2021.4.5'
total_ram_available='1.3.6.1.4.1.2021.4.6'
total_ram_buffered='1.3.6.1.4.1.2021.4.14'
total_cached_memory='1.3.6.1.4.1.2021.4.15'
# Configure snmpwalk function
walk() {
snmpwalk -v 2c -c $s $h $@
}
# Configure output formatting
chomp() {
awk -F: '{print $NF}' | grep -oP '[0-9]{1,}'
}
[root@devsw01 ~]#
echo "Used Memory: $(echo "scale=0;($(walk ${total_ram} | chomp)\
> -$(walk ${total_cached_memory} | chomp)\
> -$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)%"
Used Memory: 32%
The script below (also on GitHub) shows reasonably detailed memory utilization information that can be used to calm a panicked mind. If this doesn’t help, check with the management to see if the mind in question can be better utilized elsewhere.
NOTE: This script can be made a lot more efficient if you just run snmpwalk once and assign the relevant data to variables that can be reused. Feel free to do so.
# __________/ o \__________
# igor@comradegeneral.com
# ----------------------------------------------------------------------------
# Retrieve a server's memory utilization information via SNMP
# ----------------------------------------------------------------------------
# ----------------------------------------------------------------------------
# 2020-04-13 igor wrote this script
echo "You need to supply this command with a valid hostname. Exiting..."
ping_status=$(timeout 6 ping -c 2 -W 1 ${h} >/dev/null 2>&1 ; echo $?)
if [ -z "${ping_status}" ] || [ ${ping_status} -ne 0 ]
echo "Unable to ping ${h}. Exiting..."
total_ram='1.3.6.1.4.1.2021.4.5'
total_ram_available='1.3.6.1.4.1.2021.4.6'
total_ram_buffered='1.3.6.1.4.1.2021.4.14'
total_cached_memory='1.3.6.1.4.1.2021.4.15'
# Update the SNMP community string. Make sure the script has appropriate permissions
s="*********************"
hfqdn="$(nslookup $h 2>/dev/null | grep -m1 '^Name:' | awk '{print $NF}')"
snmpwalk -v 2c -c $s $h $@
awk -F: '{print $NF}' | grep -oP '[0-9]{1,}'
echo "-------------------------------------"
echo "-------------------------------------"
echo -e "Total RAM installed:\t ~$(echo "scale=0;($(walk ${total_ram} | chomp))/1024/1024"|bc -l)GB"
echo -e "Allocated memory:\t ~$(echo "scale=0;($(walk ${total_ram} | chomp)\
-$(walk ${total_ram_available} | chomp))/1024/1024"|bc -l)GB"
echo -e "Unallocated memory:\t ~$(echo "scale=0;($(walk ${total_ram_available} | chomp))/1024/1024"|bc -l)GB"
echo -e "Utilized memory:\t ~$(echo "scale=0;($(walk ${total_ram} | chomp)\
-$(walk ${total_cached_memory} | chomp)\
-$(walk ${total_ram_buffered} | chomp))/1024/1024"|bc -l)GB ($(echo "scale=0;($(walk ${total_ram} | chomp)\
-$(walk ${total_cached_memory} | chomp)\
-$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)%)"
available_pct="$(echo "scale=0;($(walk ${total_ram_available} | chomp)\
+$(walk ${total_cached_memory} | chomp)\
+$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)"
if [ ${available_pct} -gt 40 ]
elif [ ${available_pct} -le 40 ] && [ ${available_pct} -gt 15 ]
elif [ ${available_pct} -le 15 ]
echo -e "${highlight}Available memory:\t ~$(echo "scale=0;($(walk ${total_ram_available} | chomp)\
+$(walk ${total_cached_memory} | chomp)\
+$(walk ${total_ram_buffered} | chomp))/1024/1024"|bc -l)GB ($(echo "scale=0;($(walk ${total_ram_available} | chomp)\
+$(walk ${total_cached_memory} | chomp)\
+$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)%)${NC}"
echo "Available memory composition:"
echo -e "\tCached: \t ~$(echo "scale=0;($(walk ${total_cached_memory} | chomp))/1024/1024"|bc -l)GB"
echo -e "\tBuffered:\t ~$(echo "scale=0;($(walk ${total_ram_buffered} | chomp))/1024/1024"|bc -l)GB"
echo -e "\tUnallocated:\t ~$(echo "scale=0;($(walk ${total_ram_available} | chomp))/1024/1024"|bc -l)GB"
echo "-------------------------------------"
# ----------------------------------------------------------------------------
# ----------------------------------------------------------------------------
#!/bin/bash
#
# |
# ___/"\___
# __________/ o \__________
# (I) (G) \___/ (O) (R)
# Igor Os
# igor@comradegeneral.com
# 2020-04-13
# ----------------------------------------------------------------------------
# Retrieve a server's memory utilization information via SNMP
# ----------------------------------------------------------------------------
# CHANGE CONTROL
# ----------------------------------------------------------------------------
# 2020-04-13 igor wrote this script
h=""
if [ -z "${h}" ]
then
echo "You need to supply this command with a valid hostname. Exiting..."
exit 300
else
ping_status=$(timeout 6 ping -c 2 -W 1 ${h} >/dev/null 2>&1 ; echo $?)
if [ -z "${ping_status}" ] || [ ${ping_status} -ne 0 ]
then
echo "Unable to ping ${h}. Exiting..."
exit 310
fi
fi
configure() {
total_ram='1.3.6.1.4.1.2021.4.5'
total_ram_available='1.3.6.1.4.1.2021.4.6'
total_ram_buffered='1.3.6.1.4.1.2021.4.14'
total_cached_memory='1.3.6.1.4.1.2021.4.15'
# Update the SNMP community string. Make sure the script has appropriate permissions
s="*********************"
RED='3[1;31m'
GREEN='3[1;32m'
YELLOW='3[1;33m'
NC='3[0m'
hfqdn="$(nslookup $h 2>/dev/null | grep -m1 '^Name:' | awk '{print $NF}')"
if [ -z "${hfqdn}" ]
then
hfqdn="${h}"
fi
}
walk() {
snmpwalk -v 2c -c $s $h $@
}
chomp() {
awk -F: '{print $NF}' | grep -oP '[0-9]{1,}'
}
mem_check() {
echo "-------------------------------------"
echo "${hfqdn}"
echo "-------------------------------------"
echo -e "Total RAM installed:\t ~$(echo "scale=0;($(walk ${total_ram} | chomp))/1024/1024"|bc -l)GB"
echo -e "Allocated memory:\t ~$(echo "scale=0;($(walk ${total_ram} | chomp)\
-$(walk ${total_ram_available} | chomp))/1024/1024"|bc -l)GB"
echo -e "Unallocated memory:\t ~$(echo "scale=0;($(walk ${total_ram_available} | chomp))/1024/1024"|bc -l)GB"
echo -e "Utilized memory:\t ~$(echo "scale=0;($(walk ${total_ram} | chomp)\
-$(walk ${total_cached_memory} | chomp)\
-$(walk ${total_ram_buffered} | chomp))/1024/1024"|bc -l)GB ($(echo "scale=0;($(walk ${total_ram} | chomp)\
-$(walk ${total_cached_memory} | chomp)\
-$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)%)"
available_pct="$(echo "scale=0;($(walk ${total_ram_available} | chomp)\
+$(walk ${total_cached_memory} | chomp)\
+$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)"
if [ ${available_pct} -gt 40 ]
then
highlight="${GREEN}"
elif [ ${available_pct} -le 40 ] && [ ${available_pct} -gt 15 ]
then
highlight="${YELLOW}"
elif [ ${available_pct} -le 15 ]
then
highlight="${RED}"
fi
echo ""
echo -e "${highlight}Available memory:\t ~$(echo "scale=0;($(walk ${total_ram_available} | chomp)\
+$(walk ${total_cached_memory} | chomp)\
+$(walk ${total_ram_buffered} | chomp))/1024/1024"|bc -l)GB ($(echo "scale=0;($(walk ${total_ram_available} | chomp)\
+$(walk ${total_cached_memory} | chomp)\
+$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)%)${NC}"
echo ""
echo "Available memory composition:"
echo -e "\tCached: \t ~$(echo "scale=0;($(walk ${total_cached_memory} | chomp))/1024/1024"|bc -l)GB"
echo -e "\tBuffered:\t ~$(echo "scale=0;($(walk ${total_ram_buffered} | chomp))/1024/1024"|bc -l)GB"
echo -e "\tUnallocated:\t ~$(echo "scale=0;($(walk ${total_ram_available} | chomp))/1024/1024"|bc -l)GB"
echo "-------------------------------------"
}
# ----------------------------------------------------------------------------
# RUNTIME
# \(^_^)/ __|__
# __|__ *---o0o---*
# __|__ *---o0o---*
# *---o0o---*
# ----------------------------------------------------------------------------
configure
mem_check
#!/bin/bash
#
# |
# ___/"\___
# __________/ o \__________
# (I) (G) \___/ (O) (R)
# Igor Os
# igor@comradegeneral.com
# 2020-04-13
# ----------------------------------------------------------------------------
# Retrieve a server's memory utilization information via SNMP
# ----------------------------------------------------------------------------
# CHANGE CONTROL
# ----------------------------------------------------------------------------
# 2020-04-13 igor wrote this script
h=""
if [ -z "${h}" ]
then
echo "You need to supply this command with a valid hostname. Exiting..."
exit 300
else
ping_status=$(timeout 6 ping -c 2 -W 1 ${h} >/dev/null 2>&1 ; echo $?)
if [ -z "${ping_status}" ] || [ ${ping_status} -ne 0 ]
then
echo "Unable to ping ${h}. Exiting..."
exit 310
fi
fi
configure() {
total_ram='1.3.6.1.4.1.2021.4.5'
total_ram_available='1.3.6.1.4.1.2021.4.6'
total_ram_buffered='1.3.6.1.4.1.2021.4.14'
total_cached_memory='1.3.6.1.4.1.2021.4.15'
# Update the SNMP community string. Make sure the script has appropriate permissions
s="*********************"
RED='3[1;31m'
GREEN='3[1;32m'
YELLOW='3[1;33m'
NC='3[0m'
hfqdn="$(nslookup $h 2>/dev/null | grep -m1 '^Name:' | awk '{print $NF}')"
if [ -z "${hfqdn}" ]
then
hfqdn="${h}"
fi
}
walk() {
snmpwalk -v 2c -c $s $h $@
}
chomp() {
awk -F: '{print $NF}' | grep -oP '[0-9]{1,}'
}
mem_check() {
echo "-------------------------------------"
echo "${hfqdn}"
echo "-------------------------------------"
echo -e "Total RAM installed:\t ~$(echo "scale=0;($(walk ${total_ram} | chomp))/1024/1024"|bc -l)GB"
echo -e "Allocated memory:\t ~$(echo "scale=0;($(walk ${total_ram} | chomp)\
-$(walk ${total_ram_available} | chomp))/1024/1024"|bc -l)GB"
echo -e "Unallocated memory:\t ~$(echo "scale=0;($(walk ${total_ram_available} | chomp))/1024/1024"|bc -l)GB"
echo -e "Utilized memory:\t ~$(echo "scale=0;($(walk ${total_ram} | chomp)\
-$(walk ${total_cached_memory} | chomp)\
-$(walk ${total_ram_buffered} | chomp))/1024/1024"|bc -l)GB ($(echo "scale=0;($(walk ${total_ram} | chomp)\
-$(walk ${total_cached_memory} | chomp)\
-$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)%)"
available_pct="$(echo "scale=0;($(walk ${total_ram_available} | chomp)\
+$(walk ${total_cached_memory} | chomp)\
+$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)"
if [ ${available_pct} -gt 40 ]
then
highlight="${GREEN}"
elif [ ${available_pct} -le 40 ] && [ ${available_pct} -gt 15 ]
then
highlight="${YELLOW}"
elif [ ${available_pct} -le 15 ]
then
highlight="${RED}"
fi
echo ""
echo -e "${highlight}Available memory:\t ~$(echo "scale=0;($(walk ${total_ram_available} | chomp)\
+$(walk ${total_cached_memory} | chomp)\
+$(walk ${total_ram_buffered} | chomp))/1024/1024"|bc -l)GB ($(echo "scale=0;($(walk ${total_ram_available} | chomp)\
+$(walk ${total_cached_memory} | chomp)\
+$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)%)${NC}"
echo ""
echo "Available memory composition:"
echo -e "\tCached: \t ~$(echo "scale=0;($(walk ${total_cached_memory} | chomp))/1024/1024"|bc -l)GB"
echo -e "\tBuffered:\t ~$(echo "scale=0;($(walk ${total_ram_buffered} | chomp))/1024/1024"|bc -l)GB"
echo -e "\tUnallocated:\t ~$(echo "scale=0;($(walk ${total_ram_available} | chomp))/1024/1024"|bc -l)GB"
echo "-------------------------------------"
}
# ----------------------------------------------------------------------------
# RUNTIME
# \(^_^)/ __|__
# __|__ *---o0o---*
# __|__ *---o0o---*
# *---o0o---*
# ----------------------------------------------------------------------------
configure
mem_check
And sample output:
[root@devsw01~]# memwalk hecuba
-------------------------------------
-------------------------------------
Total RAM installed: ~62GB
Utilized memory: ~22GB (35%)
Available memory: ~41GB (65%)
Available memory composition:
-------------------------------------
[root@devsw01~]# memwalk hecuba
-------------------------------------
hecuba.kc16.local
-------------------------------------
Total RAM installed: ~62GB
Allocated memory: ~62GB
Unallocated memory: ~0GB
Utilized memory: ~22GB (35%)
Available memory: ~41GB (65%)
Available memory composition:
Cached: ~40GB
Buffered: ~0GB
Unallocated: ~0GB
-------------------------------------
[root@devsw01~]# memwalk hecuba
-------------------------------------
hecuba.kc16.local
-------------------------------------
Total RAM installed: ~62GB
Allocated memory: ~62GB
Unallocated memory: ~0GB
Utilized memory: ~22GB (35%)
Available memory: ~41GB (65%)
Available memory composition:
Cached: ~40GB
Buffered: ~0GB
Unallocated: ~0GB
-------------------------------------