Hidden features of Bash
BashShellScriptingHidden FeaturesBash Problem Overview
Shell scripts are often used as glue, for automation and simple one-off tasks. What are some of your favorite "hidden" features of the Bash shell/scripting language?
- One feature per answer
- Give an example and short description of the feature, not just a link to documentation
- Label the feature using bold title as the first line
See also:
Bash Solutions
Solution 1 - Bash
insert preceding line's final parameter
alt-. the most useful key combination ever, try it and see, for some reason no one knows about this one.
press it again and again to select older last parameters.
great when you want to do something else to something you used just a moment ago.
Solution 2 - Bash
If you want to keep a process running after you log out:
disown -h <pid>
is a useful bash built-in. Unlike nohup
, you can run disown
on an already-running process.
First, stop your job with control-Z, get the pid from ps
(or use echo $!
), use bg
to send it to the background, then use disown
with the -h flag.
Don't forget to background your job or it will be killed when you logout.
Solution 3 - Bash
Almost everything listed under EXPANSION section in the manual
In particular, parameter expansion:
$ I=foobar
$ echo ${I/oo/aa} #replacement
faabar
$ echo ${I:1:2} #substring
oo
$ echo ${I%bar} #trailing substitution
foo
$ echo ${I#foo} #leading substitution
bar
Solution 4 - Bash
My favorite:
sudo !!
Rerun the previous command with sudo.
Solution 5 - Bash
More magic key combinations:
-
Ctrl + r begins a “reverse incremental search” through your command history. As you continue to type, it retrieves the most recent command that contains all the text you enter.
-
Tab completes the word you've typed so far if it's unambiguous.
-
Tab Tab lists all completions for the word you've typed so far.
-
Alt + * inserts all possible completions, which is particularly helpful, say, if you've just entered a potentially destructive command with wildcards:
rm -r source/d*.c
Alt + *
rm -r source/delete_me.c source/do_not_delete_me.c
-
Ctrl + Alt + e performs alias, history, and shell expansion on the current line. In other words, the current line is redisplayed as it will be processed by the shell:
ls $HOME/tmp
Ctrl Alt + e
ls -N --color=tty -T 0 /home/cramey
Solution 6 - Bash
Get back history commands and arguments
It's possible to selectively access previous commands and arguments using the !
operator. It's very useful when you are working with long paths.
You can check your last commands with history
.
You can use previous commands with !<n>
being n
the index of the command in history
, negative numbers count backwards from the last command in history.
ls -l foo bar
touch foo bar
!-2
You can use previous arguments with !:<n>
, zero is the command, >= 1 are the arguments.
ls -l foo
touch !:2
cp !:1 bar
And you can combine both with !<n>:<m>
touch foo bar
ls -l !:1 !:2
rm !-2:1 !-2:2
!-2
You can also use argument ranges !<n>:<x>-<y>
touch boo far
ls -l !:1-2
Other !
special modifiers are:
-
*
for all the argumentsls -l foo bar ls !*
-
^
for the first argument (!:1
==!^
) -
$
for the last argumentls -l foo bar cat !$ > /dev/null
Solution 7 - Bash
I like the -x feature, allowing to see what's going on in your script.
bash -x script.sh
Solution 8 - Bash
SECONDS=0; sleep 5 ; echo "that took approximately $SECONDS seconds"
> SECONDS > > Each time this parameter is > referenced, the number of seconds > since shell invocation is returned. > If a value is assigned to SECONDS, > the value returned upon subsequent > references is the number of seconds > since the assignment plus the value > assigned. If SECONDS is unset, it > loses its special properties, even if > it is subsequently reset.
Solution 9 - Bash
Here is one of my favorites. This sets tab completion to not be case sensitive. It's really great for quickly typing directory paths, especially on a Mac where the file system is not case sensitive by default. I put this in .inputrc
in my home folder.
set completion-ignore-case on
Solution 10 - Bash
The special variable random:
if [[ $(($RANDOM % 6)) = 0 ]]
then echo "BANG"
else
echo "Try again"
fi
Solution 11 - Bash
Regular expression handling
Recent bash releases feature regular expression matching, so you can do:
if [[ "mystring" =~ REGEX ]] ; then
echo match
fi
where REGEX is a raw regular expression in the format described by man re_format.
Matches from any bracketed parts are stored in the BASH_REMATCH array, starting at element 1 (element 0 is the matched string in its entirety), so you can use this to do regex-powered parsing too.
Solution 12 - Bash
Quick & Dirty correction of typos (especially useful for long commands over slow connections where using the command history and scrolling through it would be horrible):
$ cat /proc/cupinfo
cat: /proc/cupinfo: No such file or directory
$ ^cup^cpu
Also try !:s/old/new
which substitutes old with new in the previous command once.
If you want to substitute many occurrences you can do a global substitution with !:gs/old/new
.
You can use the gs
and s
commands with any history event, e.g.
!-2:s/old/new
To substitute old
with new
(once) in the second to last command.
Solution 13 - Bash
Ctrlx Ctrle
This will load the current command into the editor defined in the variable VISUAL. This is really useful for long commands like some of those listed here.
To use vi as your editor:
export VISUAL=vi
Solution 14 - Bash
Here two of my favorites:
To check the syntax w/o really executing the script use:
bash -n script.sh
Go back to the last directory (yes I know pushd and popd, but this is quicker)
cd -
Solution 15 - Bash
Using Infix Boolean Operators
Consider the simple if:
if [ 2 -lt 3 ]
then echo "Numbers are still good!"
fi
That -lt looks kinda ugly. Not very modern. If you use double brackets around your boolean expression you can the normal boolean operators!
if [[ 2 < 3 ]]
then echo "Numbers are still good!"
fi
Solution 16 - Bash
Arrays:
#!/bin/bash
array[0]="a string"
array[1]="a string with spaces and \"quotation\" marks in it"
array[2]="a string with spaces, \"quotation marks\" and (parenthesis) in it"
echo "There are ${#array[*]} elements in the array."
for n in "${array[@]}"; do
echo "element = >>${n}<<"
done
More details on arrays (and other advanced bash scripting stuff) can be found in the Advanced Bash-Scripting Guide.
Solution 17 - Bash
Running a command before displaying the bash prompt
Set a command in the "PROMPT_COMMAND" env variable and it will be run automatically before each prompt. Example:
[lsc@home]$ export PROMPT_COMMAND="date"
Fri Jun 5 15:19:18 BST 2009
[lsc@home]$ ls
file_a file_b file_c
Fri Jun 5 15:19:19 BST 2009
[lsc@home]$ ls
For the next april fools, add "export PROMPT_COMMAND=cd" to someone's .bashrc then sit back and watch the confusion unfold.
Solution 18 - Bash
man
pages:
Magic key combinations from the bash -
Ctrl + a and Ctrl + e move the cursor to the beginning and end of the current line, respectively.
-
Ctrl + t and Alt + t transpose the character and word before the cursor with the current one, then move the cursor forward.
-
Alt + u and Alt + l convert the current word (from the cursor to the end) to uppercase and lowercase.
Hint: Press Alt + – followed by either of these commands to convert the beginning of the current word.
Bonus man
tips:
-
While viewing
man
pages, use / to search for text within the pages. Use n to jump ahead to the next match or N for the previous match. -
Speed your search for a particular command or sub-section within the
man
pages by taking advantage of their formatting:
o Instead of typing /history expansion to find that section, try /^history, using the caret (^
) to find only lines that begin with "history."
o Try / read, with a few leading spaces, to search for that builtin command. Builtins are always indented in the man
pages.
Solution 19 - Bash
export TMOUT=$((15*60))
Terminate bash after 15 minutes of idle time, set to 0 to disable. I usually put this to ~/.bashrc on my root accounts. It's handy when administrating your boxes and you may forget to logout before walking away from the terminal.
Solution 20 - Bash
Undo
C-S-- Control Shift Minus Undo-es typing actions.
Kill / Yank
Any delete operation C-w (delete previous word), C-k (delete to end of line), C-u (delete to start of line) etc... copies it's deleted text to the kill ring, you can paste the last kill with: C-y and cycle through (and paste from) the ring of deleted items with Alt-y
Solution 21 - Bash
You can ignore certain files while tab completing by setting th FIGNORE
variable.
For example, if you have a subverion repo and you want to navigate more easily do
export FIGNORE=".svn"
now you can cd
without being blocked by .svn
directories.
Solution 22 - Bash
Using arithmetic:
if [[ $((2+1)) = $((1+2)) ]]
then echo "still ok"
fi
Solution 23 - Bash
Brace expansion
Standard expansion with {x,y,z}:
$ echo foo{bar,baz,blam}
foobar foobaz fooblam
$ cp program.py{,.bak} # very useful with cp and mv
Sequence expansion with {x..y}:
$ echo {a..z}
a b c d e f g h i j k l m n o p q r s t u v w x y z
$ echo {a..f}{0..3}
a0 a1 a2 a3 b0 b1 b2 b3 c0 c1 c2 c3 d0 d1 d2 d3 e0 e1 e2 e3 f0 f1 f2 f3
Solution 24 - Bash
I recently read Csh Programming Considered Harmful which contained this astounding gem:
>Consider the pipeline:
A | B | C
>You want to know the status of C, well, that's easy: it's in $?, or >$status in csh. But if you want it from A, you're out of luck -- if >you're in the csh, that is. In the Bourne shell, you can get it, although >doing so is a bit tricky. >Here's something I had to do where I ran dd's >stderr into a grep -v pipe to get rid of the records in/out noise, but had >to return the dd's exit status, not the grep's:
device=/dev/rmt8
dd_noise='^[0-9]+\+[0-9]+ records (in|out)$'
exec 3>&1
status=`((dd if=$device ibs=64k 2>&1 1>&3 3>&- 4>&-; echo $? >&4) |
egrep -v "$dd_noise" 1>&2 3>&- 4>&-) 4>&1`
exit $status;
Solution 25 - Bash
Truncate content of a file (zeroing file)
> file
Specifically, this is very good for truncating log files, when the file is open by another process, which still may write to the file.
Solution 26 - Bash
Not really a feature but rather a direction: I found many "hidden features", secrets and various bash usefulness at commandlinefu.com. Many of the highest rated answers to this answers, I learned them on that site :)
Solution 27 - Bash
Another small one: Alt+#
comments out the current line and moves it into the history buffer.
So when you're assembling a command line and you need to issue an interim command to e.g. find a file, you just hit alt+#, issue the other command, go up in the history, uncomment and proceed.
Solution 28 - Bash
Braces in lieu of do
and done
in for loop
For
loop body are usually in do...done
(just an example):
for f in *;
do
ls "$f";
done
But we can use a C style using braces:
for f in *; {
ls "$f";
}
I think this looks better than do...done
and I prefer this one. I have not yet found this in any Bash documentation, so this is really a hidden feature.
Solution 29 - Bash
C style numeric expressions:
let x="RANDOM%2**8"
echo -n "$x = 0b"
for ((i=8; i>=0; i--)); do
let n="2**i"
if (( (x&n) == n )); then echo -n "1"
else echo -n "0"
fi
done
echo ""
Solution 30 - Bash
These properties are another one of my favorites.
export HISTCONTROL=erasedups
export HISTSIZE=1000
The first one makes sure bash doesn't log commands more than once, will really improves history
's usefulness. The other expands the history size to 1000 from the default of 100. I actually set this to 10000 on my machines.
Solution 31 - Bash
Easily move around between multiple directories
Not a hidden feature, but much more flexible than pushd which requires stack-like navigation.
a() { alias $1=cd\ $PWD; }
cd
somewhere and type a 1
. Later on just typing 1
will return to that directory.
Solution 32 - Bash
One I use a lot is !$ to refer to the last word of the last command:
$ less foobar.txt
...
# I dont want that file any more
$ rm !$
Solution 33 - Bash
set -o vi
in order to have vi-like editing of the command history as well as of the currently typed command.
Solution 34 - Bash
As others have mentioned, Ctrl-r is great for stepping back through your command history. But what if you want to go forward after you've taken one or a few steps too many? That's where Ctrl-s comes in handy. However, it's normally mapped to XOFF (interrupt data flow). Since that's not too useful any more because we're not using slow serial terminals, you can turn off that mapping with:
stty -ixon
in your ~/.bashrc
file.
This also makes Ctrl-q available which is normally a duplicate of Ctrl-v (quoted-insert which allows you to insert a literal control character). I have Ctrl-q mapped to menu-complete which steps through completions when pressed repeatedly. I like to leave Tab set to regular complete.
You can set Ctrl-q to menu-complete by adding this line to your ~/.inputrc
file:
"\C-q": menu-complete
Solution 35 - Bash
Bash has variable indirection:
$ foo=bar
$ baz=foo
$ echo ${!baz}
bar
Solution 36 - Bash
I have an alias r='fc-s'
, and I find it very useful in some limited cases. To run the last command, just type r
and hit enter, and that's it. Of course, that itself is not very useful because up arrow does the same thing. But you can use r
to run the previous command with substitutions. Let's say your last command was a long command compiling some file:
$ gcc -c <file_name>.c <lots of options> -o <file_name>.o
Now you want to compile another file with the same options and have a corresponding .o
file:
$ r <file_name>=<new_file>
will do it. You don't have to use up arrow, navigate to the right places and then replace them each manually. This can be repeated multiple times, so you can do this next:
$ r <new_file>=<other_file>
Of course, for such a thing you have makefiles, but I hope I have shown that the alias is useful.
I haven't needed the use of this alias a lot, but there have been times that I have been glad that I have this alias!
Solution 37 - Bash
Here Strings (<<<
). The Bash manual gives this description:
> The word is expanded and supplied to the command on its standard input.
Example:
$ cat<<<"$(( 10*3+1 )) nice isn't it?"
31 nice isn't it?
Solution 38 - Bash
Using 'let' built-in bash command for basic arithmetic
A=10
let B="A * 10 + 1" # B=101
let B="B / 8" # B=12, let does not do floating point
let B="(RANDOM % 6) + 1" # B is now a random number between 1 and 6
To do floating point evaluations, you can use the "bc" command (no part of bash).
FP=`echo "scale=4; 10 / 3" | bc` # FP="3.3333"
Solution 39 - Bash
Process substitution with <(cmd ...) or >(cmd ...)
In each form, the cmd is executed with its input or output hooked up to a FIFO, and the path to that FIFO is substituted on the command line:
$ echo A file to read: <(cat), a file to write to: >(cat)
A file to read: /dev/fd/63, a file to write to: /dev/fd/62
For example, to compare two website without saving intermediate files:
$ diff <(curl -s http://tldp.org/LDP/abs/html/) <(curl -s http://www.redhat.com/mirrors/LDP/LDP/abs/html/)
If you have a command that takes a file name as input, but doesn't accept '-' to mean stdout, you can trick it:
$ do_thingee --log -
error: can't open log file: '-'
$ do_thingee --log >(cat)
do_thingee v0.2
initializing things
processing 4 things
done
Solution 40 - Bash
Special socket filenames: /dev/tcp/HOST/PORT and /dev/udp/HOST/PORT
Read from a daytime server (port 13):
$ cat < /dev/tcp/utcnist.colorado.edu/13
55786 11-08-13 03:34:21 50 0 0 172.3 UTC(NIST) *
This can be quite useful in conjunction with tcpserver.
A more advanced example from <http://thesmithfam.org/blog/2006/05/23/bash-socket-programming-with-devtcp-2/> if you don't have access to wget or curl:
$ exec 3<>/dev/tcp/www.google.com/80 # hook up to file desc 3
$ echo -e "GET / HTTP/1.1\n\n" >&3 # send the HTTP request
$ cat <&3 # read the HTTP response
Solution 41 - Bash
Embedded Command substitution:
> hostname && dig +short $(hostname) && dig +short -x $(dig +short $(hostname))
This command is good for checking RDNS on your mail server. :P
Solution 42 - Bash
Quick History Search
Following gives a tcsh like history search which is handy and easier.
Add the following lines to ~/.inputrc
or /etc/inputrc
.
$ cat ~/.inputrc
"\e[A": history-search-backward
"\e[B": history-search-forward
You may want to use a less accidental key combination such as Esc
+ p
. If that's the case, use
"\ep": history-search-backward
"\en": history-search-forward
Then, simply type the first few letters and press UpArrow key. It'll show the most recent command that start with the given letters.
ex:
type grep
, UpArrow.
It'll show something like grep -ri myText .
Solution 43 - Bash
Get more info about Key combinations in Bash in http://linuxconfig.net/manual-howto/key-combinations-in-bash.html