Peak memory usage of a linux/unix process
LinuxCommand LineMemory ManagementLinux Problem Overview
Is there a tool that will run a command-line and report the peak RAM usage total?
I'm imagining something analogous to /usr/bin/time
Linux Solutions
Solution 1 - Linux
[Edit: Works on Ubuntu 14.04: /usr/bin/time -v command
Make sure to use the full path.]
Looks like /usr/bin/time
does give you that info, if you pass -v
(this is on Ubuntu 8.10). See, e.g., Maximum resident set size
below:
$ /usr/bin/time -v ls / .... Command being timed: "ls /" User time (seconds): 0.00 System time (seconds): 0.01 Percent of CPU this job got: 250% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 0 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 315 Voluntary context switches: 2 Involuntary context switches: 0 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
Solution 2 - Linux
(This is an already answered, old question.. but just for the record :)
I was inspired by Yang's script, and came up with this small tool, named memusg. I simply increased the sampling rate to 0.1 to handle much short living processes. Instead of monitoring a single process, I made it measure rss sum of the process group. (Yeah, I write lots of separate programs that work together) It currently works on Mac OS X and Linux. The usage had to be similar to that of time
:
memusg ls -alR / >/dev/null
It only shows the peak for the moment, but I'm interested in slight extensions for recording other (rough) statistics.
It's good to have such simple tool for just taking a look before we start any serious profiling.
Solution 3 - Linux
Valgrind one-liner:
valgrind --tool=massif --pages-as-heap=yes --massif-out-file=massif.out ./test.sh; grep mem_heap_B massif.out | sed -e 's/mem_heap_B=\(.*\)/\1/' | sort -g | tail -n 1
Note use of --pages-as-heap to measure all memory in a process. More info here: http://valgrind.org/docs/manual/ms-manual.html
This will slow down your command significantly.
Solution 4 - Linux
On Linux:
Use /usr/bin/time -v <program> <args>
and look for "Maximum resident set size".
(Not to be confused with the Bash time
built-in command! So use the full path, /usr/bin/time
)
For example:
> /usr/bin/time -v ./myapp
User time (seconds): 0.00
. . .
Maximum resident set size (kbytes): 2792
. . .
On BSD, MacOS:
Use /usr/bin/time -l <program> <args>
, looking for "maximum resident set size":
>/usr/bin/time -l ./myapp
0.01 real 0.00 user 0.00 sys
1440 maximum resident set size
. . .
Solution 5 - Linux
Here's a one-liner that doesn't require any external scripts or utilities and doesn't require you to start the process via another program like Valgrind or time, so you can use it for any process that's already running:
grep ^VmPeak /proc/$PID/status
(replace $PID
with the PID of the process you're interested in)
Solution 6 - Linux
Perhaps (gnu) time(1) already does what you want. For instance:
$ /usr/bin/time -f "%P %M" command
43% 821248
But other profiling tools may give more accurate results depending on what you are looking for.
Solution 7 - Linux
On MacOS Sierra use:
/usr/bin/time -l commandToMeasure
You can use grep
to take what you want maybe.
Solution 8 - Linux
/usr/bin/time maybe does what you want, actually. Something like.
/usr/bin/time --format='(%Xtext+%Ddata %Mmax)'
See time(1) for details...
Solution 9 - Linux
If the process runs for at least a couple seconds, then you can use the following bash script, which will run the given command line then print to stderr the peak RSS (substitute for rss
any other attribute you're interested in). It's somewhat lightweight, and it works for me with the ps
included in Ubuntu 9.04 (which I can't say for time
).
#!/usr/bin/env bash
"$@" & # Run the given command line in the background.
pid=$! peak=0
while true; do
sleep 1
sample="$(ps -o rss= $pid 2> /dev/null)" || break
let peak='sample > peak ? sample : peak'
done
echo "Peak: $peak" 1>&2
Solution 10 - Linux
time -f '%M' <run_program>
Solution 11 - Linux
Well, if you really want to show the memory peak and some more in-depth statistics i recommend using a profiler such as valgrind. A nice valgrind front-end is alleyoop.
Solution 12 - Linux
Because /usr/bin/time
is not present in many modern distributions (Bash built-in time instead), you can use Busybox time implementation with -v
argument:
busybox time -v uname -r
It's output is similar to GNU time output. Busybox is pre-installed in most Linux distros (Debian, Ubuntu, etc.). If you using Arch Linux, you can install it with:
sudo pacman -S busybox
Solution 13 - Linux
You can use a tool like Valgrind to do this.
Solution 14 - Linux
Here is (based on the other answers) a very simple script that watches an already running process. You just run it with the pid of the process you want to watch as the argument:
#!/usr/bin/env bash
pid=$1
while ps $pid >/dev/null
do
ps -o vsz= ${pid}
sleep 1
done | sort -n | tail -n1
Example usage:
max_mem_usage.sh 23423
Solution 15 - Linux
Heaptrack is KDE tool that has a GUI and text interface. I find it more suitable than valgrind to understand the memory usage of a process because it provides more details and flamegraphs. It's also faster because it does less checking that valgrind. And it gives you the peak memory usage.
Anyway, tracking rss and vss is misleading because pages could be shared, that's why that memusg
. What you should really do is track the sum of Pss
in /proc/[pid]/smaps
or use pmap
. GNOME system-monitor used to do so but it was too expensive.
Solution 16 - Linux
Use Massif: http://valgrind.org/docs/manual/ms-manual.html
Solution 17 - Linux
Re-inventing the wheel, with hand made bash script. Quick and clean.
My use case: I wanted to monitor a linux machine which has less RAM and wanted to take a snapshot of per container usage when it runs under heavy usage.
#!/usr/bin/env bash
threshold=$1
echo "$(date '+%Y-%m-%d %H:%M:%S'): Running free memory monitor with threshold $threshold%.."
while(true)
freePercent=`free -m | grep Mem: | awk '{print ($7/$2)*100}'`
do
if (( $(awk 'BEGIN {print ("'$freePercent'" < "'$threshold'")}') ))
then
echo "$(date '+%Y-%m-%d %H:%M:%S'): Free memory $freePercent% is less than $threshold%"
free -m
docker stats --no-stream
sleep 60
echo ""
else
echo "$(date '+%Y-%m-%d %H:%M:%S'): Sufficient free memory available: $freePercent%"
fi
sleep 30
done
Sample output:
> 2017-10-12 13:29:33: Running free memory monitor with threshold 30%..
> 2017-10-12 13:29:33: Sufficient free memory available: 69.4567%
> 2017-10-12 13:30:03: Sufficient free memory available: 69.4567% > > 2017-10-12 16:47:02: Free memory 18.9387% is less than 30% > your custom command output
Solution 18 - Linux
On macOS, you can use DTrace instead. The "Instruments" app is a nice GUI for that, it comes with XCode afaik.
Solution 19 - Linux
'htop' is best command for see which process is using how much RAM.....
for more detail http://manpages.ubuntu.com/manpages/precise/man1/htop.1.html
Solution 20 - Linux
>> Please be sure to answer the question. Provide details and share your research!
Sorry, I am first time here and can only ask questions…
Used suggested:
valgrind --tool=massif --pages-as-heap=yes --massif-out-file=massif.out ./test.sh; grep mem_heap_B massif.out | sed -e 's/mem_heap_B=\(.*\)/\1/' | sort -g | tail -n 1
then:
grep mem_heap_B massif.out
...
mem_heap_B=1150976
mem_heap_B=1150976
...
this is very different from what top
command shows at similar moment:
14673 gu27mox 20 0 3280404 468380 19176 R 100.0 2.9 6:08.84 pwanew_3pic_com
what are measured units from Valgrind??
The /usr/bin/time -v ./test.sh
never answered — you must directly feed executable to /usr/bin/time
like:
/usr/bin/time -v pwanew_3pic_compass_2008florian3_dfunc.static card_0.100-0.141_31212_resubmit1.dat_1.140_1.180 1.140 1.180 31212
Command being timed: "pwanew_3pic_compass_2008florian3_dfunc.static card_0.100-0.141_31212_resubmit1.dat_1.140_1.180 1.140 1.180 31212"
User time (seconds): 1468.44
System time (seconds): 7.37
Percent of CPU this job got: 99%
Elapsed (wall clock) time (h:mm:ss or m:ss): 24:37.14
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 574844
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 74
Minor (reclaiming a frame) page faults: 468880
Voluntary context switches: 1190
Involuntary context switches: 20534
Swaps: 0
File system inputs: 81128
File system outputs: 1264
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0