Custom format for time command
BashShellTimeFormatBash Problem Overview
I'd like to use the time command in a bash script to calculate the elapsed time of the script and write that to a log file. I only need the real time, not the user and sys. Also need it in a decent format. e.g 00:00:00:00 (not like the standard output). I appreciate any advice.
The expected format supposed to be 00:00:00.0000 (milliseconds) [hours]:[minutes]:[seconds].[milliseconds]
I've already 3 scripts. I saw an example like this:
{ time { # section code goes here } } 2> timing.log
But I only need the real time, not the user and sys. Also need it in a decent format. e.g 00:00:00:00 (not like the standard output).
In other words, I'd like to know how to turn the time output into something easier to process.
Bash Solutions
Solution 1 - Bash
You could use the date command to get the current time before and after performing the work to be timed and calculate the difference like this:
#!/bin/bash
# Get time as a UNIX timestamp (seconds elapsed since Jan 1, 1970 0:00 UTC)
T="$(date +%s)"
# Do some work here
sleep 2
T="$(($(date +%s)-T))"
echo "Time in seconds: ${T}"
printf "Pretty format: %02d:%02d:%02d:%02d\n" "$((T/86400))" "$((T/3600%24))" "$((T/60%60))" "$((T%60))""
Notes: $((...)) can be used for basic arithmetic in bash – caution: do not put spaces before a minus - as this might be interpreted as a command-line option.
See also: http://tldp.org/LDP/abs/html/arithexp.html
EDIT:
Additionally, you may want to take a look at sed to search and extract substrings from the output generated by time.
EDIT:
Example for timing with milliseconds (actually nanoseconds but truncated to milliseconds here). Your version of date has to support the %N format and bash should support large numbers.
# UNIX timestamp concatenated with nanoseconds
T="$(date +%s%N)"
# Do some work here
sleep 2
# Time interval in nanoseconds
T="$(($(date +%s%N)-T))"
# Seconds
S="$((T/1000000000))"
# Milliseconds
M="$((T/1000000))"
echo "Time in nanoseconds: ${T}"
printf "Pretty format: %02d:%02d:%02d:%02d.%03d\n" "$((S/86400))" "$((S/3600%24))" "$((S/60%60))" "$((S%60))" "${M}"
DISCLAIMER:
My original version said
M="$((T%1000000000/1000000))"
but this was edited out because it apparently did not work for some people whereas the new version reportedly did. I did not approve of this because I think that you have to use the remainder only but was outvoted.
Choose whatever fits you.
Solution 2 - Bash
To use the Bash builtin time rather than /bin/time you can set this variable:
TIMEFORMAT='%3R'
which will output the real time that looks like this:
5.009
or
65.233
The number specifies the precision and can range from 0 to 3 (the default).
You can use:
TIMEFORMAT='%3lR'
to get output that looks like:
3m10.022s
The l (ell) gives a long format.
Solution 3 - Bash
From the man page for time:
-
There may be a shell built-in called time, avoid this by specifying
/usr/bin/time -
You can provide a format string and one of the format options is elapsed time - e.g.
%E/usr/bin/time -f'%E' $CMD
Example:
$ /usr/bin/time -f'%E' ls /tmp/mako/
res.py res.pyc
0:00.01
Solution 4 - Bash
Use the bash built-in variable SECONDS. Each time you reference the variable it will return the elapsed time since the script invocation.
Example:
echo "Start $SECONDS"
sleep 10
echo "Middle $SECONDS"
sleep 10
echo "End $SECONDS"
Output:
Start 0
Middle 10
End 20
Solution 5 - Bash
Not quite sure what you are asking, have you tried:
time yourscript | tail -n1 >log
Edit: ok, so you know how to get the times out and you just want to change the format. It would help if you described what format you want, but here are some things to try:
time -p script
This changes the output to one time per line in seconds with decimals. You only want the real time, not the other two so to get the number of seconds use:
time -p script | tail -n 3 | head -n 1
Solution 6 - Bash
The accepted answer gives me this output
# bash date.sh
Time in seconds: 51
date.sh: line 12: unexpected EOF while looking for matching `"'
date.sh: line 21: syntax error: unexpected end of file
This is how I solved the issue
#!/bin/bash
date1=$(date --date 'now' +%s) #date since epoch in seconds at the start of script
somecommand
date2=$(date --date 'now' +%s) #date since epoch in seconds at the end of script
difference=$(echo "$((date2-$date1))") # difference between two values
date3=$(echo "scale=2 ; $difference/3600" | bc) # difference/3600 = seconds in hours
echo SCRIPT TOOK $date3 HRS TO COMPLETE # 3rd variable for a pretty output.
Solution 7 - Bash
Try man time (if your output does not look like the content below, try to install a modern version of this utility):
TIME(1) General Commands Manual TIME(1)
NAME
time - run programs and summarize system resource usage
SYNOPSIS
time [ -apqvV ] [ -f FORMAT ] [ -o FILE ]
[ --append ] [ --verbose ] [ --quiet ] [ --portability ]
[ --format=FORMAT ] [ --output=FILE ] [ --version ]
[ --help ] COMMAND [ ARGS ]
DESCRIPTION
...
OPTIONS
...
-f FORMAT, --format FORMAT
Use FORMAT as the format string that controls the output of time. See the below more information.
...
FORMATTING THE OUTPUT
...
And read the FORMATTING THE OUTPUT section:
FORMATTING THE OUTPUT
The format string FORMAT controls the contents of the time output. The format string can be set using the `-f' or `--format', `-v' or `--verbose', or `-p' or `--portability' options. If they are not given, but the TIME environment variable is set, its
value is used as the format string. Otherwise, a built-in default format is used. The default format is:
%Uuser %Ssystem %Eelapsed %PCPU (%Xtext+%Ddata %Mmax)k
%Iinputs+%Ooutputs (%Fmajor+%Rminor)pagefaults %Wswaps
The format string usually consists of `resource specifiers' interspersed with plain text. A percent sign (`%') in the format string causes the following character to be interpreted as a resource specifier, which is similar to the formatting characters in
the printf(3) function.
A backslash (`\') introduces a `backslash escape', which is translated into a single printing character upon output. `\t' outputs a tab character, `\n' outputs a newline, and `\\' outputs a backslash. A backslash followed by any other character outputs a
question mark (`?') followed by a backslash, to indicate that an invalid backslash escape was given.
Other text in the format string is copied verbatim to the output. time always prints a newline after printing the resource use information, so normally format strings do not end with a newline character (or `\n').
There are many resource specifications. Not all resources are measured by all versions of Unix, so some of the values might be reported as zero. Any character following a percent sign that is not listed in the table below causes a question mark (`?') to
be output, followed by that character, to indicate that an invalid resource specifier was given.
The resource specifiers, which are a superset of those recognized by the tcsh(1) builtin `time' command, are:
% A literal `%'.
C Name and command line arguments of the command being timed.
D Average size of the process's unshared data area, in Kilobytes.
E Elapsed real (wall clock) time used by the process, in [hours:]minutes:seconds.
F Number of major, or I/O-requiring, page faults that occurred while the process was running. These are faults where the page has actually migrated out of primary memory.
I Number of file system inputs by the process.
K Average total (data+stack+text) memory use of the process, in Kilobytes.
M Maximum resident set size of the process during its lifetime, in Kilobytes.
O Number of file system outputs by the process.
P Percentage of the CPU that this job got. This is just user + system times divided by the total running time. It also prints a percentage sign.
R Number of minor, or recoverable, page faults. These are pages that are not valid (so they fault) but which have not yet been claimed by other virtual pages. Thus the data in the page is still valid but the system tables must be updated.
S Total number of CPU-seconds used by the system on behalf of the process (in kernel mode), in seconds.
U Total number of CPU-seconds that the process used directly (in user mode), in seconds.
W Number of times the process was swapped out of main memory.
X Average amount of shared text in the process, in Kilobytes.
Z System's page size, in bytes. This is a per-system constant, but varies between systems.
c Number of times the process was context-switched involuntarily (because the time slice expired).
e Elapsed real (wall clock) time used by the process, in seconds.
k Number of signals delivered to the process.
p Average unshared stack size of the process, in Kilobytes.
r Number of socket messages received by the process.
s Number of socket messages sent by the process.
t Average resident set size of the process, in Kilobytes.
w Number of times that the program was context-switched voluntarily, for instance while waiting for an I/O operation to complete.
x Exit status of the command.
With this information, you can assemble a format string. Note that the native time utility for MacOSX is a stripped down, old binary that does not have these features. But, modern linux distributions will be using this modern version of time:
time -f "dt=%E" <your command>
...
dt=0:22.84
Docker images, in general, use Linux.
docker run --cpus 1 time -f "runtime-stats: dt=%E" <your command>
And a useful benchmark procedure might be:
for image in foo bar; do
for cpus in `seq 1 8`; do
docker run --cpus $cpus -v `pwd`:/foo -t $image time -f 'runtime-stats: dt=%E' <your-command> | grep -E '^runtime-stats:' > measured-$image.runtime
done
done