What do double-asterisk (**) wildcards mean?

BashUnixWildcardGlobWildcard Expansion

Bash Problem Overview


I've tried the following command but I am having trouble interpreting the results:

ls **

I'm not sure exactly what it is outputting and why.

Bash Solutions


Solution 1 - Bash

You're most likely seeing a special feature of some shells that allow wildcard filename patterns to match across directory boundaries, as opposed to a single *, which is a wildcard that matches only within a directory.

If you do not have such a shell, ** will likely be equivalent to *, because "matching zero or more characters followed by zero or more characters" is the same as just "matching zero or more characters".

But if you do have such a shell, ** will match all files and directories in the current directory and subdirectories, whereas * only matches files and directories in the current directory. (In both cases "dot files", those with names starting with ., are not matched).

**'s real power comes when you use it in more specific patterns. For example, you can specify all .txt files no matter what subdirectory they are in with **/*.txt, whereas *.txt only matches those in the current directory.

You should look at the wildcard matching rules for your shell to know for sure what your shell is doing. For example, the bash manual says:

> *
> Matches any string, including the null string. When the > 'globstar' shell option is enabled, and '*' is used in a filename > expansion context, two adjacent '*'s used as a single pattern will > match all files and zero or more directories and subdirectories. > If followed by a '/', two adjacent '*'s will match only > directories and subdirectories.

In recent versions of bash the 'globstar' shell option is disabled by default. Enabled via:

shopt -s globstar

I believe zsh also supports this syntax.

It's important to keep in mind that wildcards are expanded by the shell, not by the ls command. If you type ls **, or ls *.txt, the ls command itself never sees the * characters; it only sees an expanded list of files matching the pattern, just as if you had typed the entire list on the command line.

Solution 2 - Bash

Globbing

By using the double asterisk (**), you are using a glob to list files on a filesystem. A glob is a string of literal or wildcard characters used for matching the file paths. Using one or more globs for locating files on a filesystem is called globbing.

Apart from Linux shells, globbing is also used in various configuration files to specify the list of files to locate. For example: files and folders to ignore in the .gitignore file, files and include options in tsconfig.json file in Typescript projects etc.

Following are some of the most important aspects of the globbing and double asterisk (**) is one of them:


Segments and Separators (/)

The separator is always the / character. A segment is everything that comes between the two separators.

Example: Tests/HelloWorld.js

Here, Tests and HelloWorld.js are the segments and / is the separator.


Single Asterisk (*)

Single Asterisk (*) matches zero or more characters within one segment. It is used for globbing the files within one directory.

Example: *.js

This glob will match files such as HelloWorld.js but not files like Tests/HelloWorld.js or Tests/UI/HelloWorld.js


Double Asterisk (**)

Double Asterisk (**) matches zero or more characters across multiple segments. It is used for globbing files that are in nested directories.

Example: Tests/**/*.js

Here, the file selecting will be restricted to the Tests directory. The glob will match the files such as Tests/HelloWorld.js, Tests/UI/HelloWorld.js, Tests/UI/Feature1/HelloWorld.js.


Question Mark(?)

Question mark(?) matches a single character within one segment. When some files or directories differ in their name by just one character, you can use the ?.

Example: tests/?at.js

This will match files such as tests/cat.js, test/Cat.js, test/bat.js etc.


Square Brackets ([abc])

Square Brackets ([...]) globs the files with a single character mentioned in the square brackets.

Example: tests/[CB]at.js

This glob will match files like tests/Cat.js or tests/Bat.js


Square Brackets Range ([a-z])

Square Brackets Range ([a-z]), matches one character specified in the range.

Example: tests/feature[1-9]/HelloWorld.js

This glob will match files like tests/feature1/HelloWorld.js, test/feature2/HelloWorld.js and so on... upto 9.


Negation (!)

Negation (!) can be used for excluding some files.

Example 1: tests/[!C]at.js

This will exclude the file tests/Cat.js and will match files like tests/Bat.js, tests/bat.js, tests/cat.js.

Negation is also used in configuration files inside an array to negate or exclude some files.

Example 2: ['Tests/**/*.js', '!Tests/UI/**']

This will exclude all files and folders from Tests/UI directory.


That's it! Hope that helps!

Solution 3 - Bash

While one or more of the other answers may be correct, they are a bit hard to grok, especially if you're a visual person like me.

So I decided to provide a visual answer, fully confirmed by tests.

I created the following directory structure, with two files "f" at each level, one with and one without a file extension, and then tested all of the patterns in the table headers below using the following command in Bash on Darwin with globstar enabled: stat -f "%N" <pattern>.

If you are a visual person, looking at the tables will give you a deeper understanding of what ** means than by just reading the definitions in the other good answers.

.
├── f
├── f.md
└── x
    ├── f
    ├── f.md
    └── y
        ├── f
        ├── f.md
        └── z
            ├── f
            └── f.md
Comparing * to **
* ** */ **/ **/*.* x/*.*
or
*/*.*
x/*
or
*/*
x/**
or
*/**
f
f.md
x
x/f
x/f.md
x/y
x/y/f
x/y/f.md
x/y/z
x/y/z/f
x/y/z/f.md
Matching or finding files in a tree
*.md


**.md
**f.md
f**.md
*/*.md
x/*.md
**/x/*.md



*/*/*.md
x/*/*.md
**/y/*.md



*/*/*/*.md
x/*/*/*.md
**/z/*.md




x/**/*.md




**/*.md





f
f.md
x
x/f
x/f.md
x/y
x/y/f
x/y/f.md
x/y/z
x/y/z/f
x/y/z/f.md

Notice how prepending or appending ** with anything other than a / results in it acting the same as *, as shown above by **.md and **f.md, neither of which matched x/y/f.md for example.

Solution 4 - Bash

The exact behavior of this particular wildcard has been well covered by the other answers, but information on the general case may be useful.

This behavior is not limited to ls, and is referred to as "globbing", which is the expansion of patterns based on matches with existing filenames. It is important to note that these patterns do not use regular expression syntax.

The shell pre-processes the arguments before they are sent to the program. There are generally multiple levels of expansion, some of these involve globbing.

A great resource for more information on the other wildcards available in a file glob pattern is the unix manpage. A online version for glob can be found here.

Finally, a simple example of what this can do for you, especially when combined with other shell expansion goodies, in this case those provided by the bash shell. Information about the expansions used in this example can be found in the Bash Guide for Beginners - which is my goto resource, despite the title.

ls *{01..04}.{txt,csv} becomes ls *01.txt *01.csv *02.txt *02.csv *03.txt *03.csv *04.txt *04.csv

Which could output something like this:

input_01.txt input_02.txt input_03.txt input_04.txt output_01.csv output_02.csv output_03.csv output_04.csv

While skipping these:

input_05.txt input_06.txt input_07.txt input_08.txt input_09.txt input_10.txt output_05.csv output_06.csv output_07.csv output_08.csv output_09.csv output_10.csv

A trivial example, but if you know that this behavior is not specific to ls, then you can imagine the utility when coupled with mv, cp, rsync, etc.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionMike BlairView Question on Stackoverflow
Solution 1 - BashKeith ThompsonView Answer on Stackoverflow
Solution 2 - BashYogesh Umesh VaityView Answer on Stackoverflow
Solution 3 - BashInigoView Answer on Stackoverflow
Solution 4 - BashMorgenView Answer on Stackoverflow