Concatenating multiple text files into a single file in Bash

BashShell

Bash Problem Overview


What is the quickest and most pragmatic way to combine all *.txt file in a directory into one large text file?

Currently I'm using windows with cygwin so I have access to BASH.

Windows shell command would be nice too but I doubt there is one.

Bash Solutions


Solution 1 - Bash

This appends the output to all.txt

cat *.txt >> all.txt

This overwrites all.txt

cat *.txt > all.txt

Solution 2 - Bash

Just remember, for all the solutions given so far, the shell decides the order in which the files are concatenated. For Bash, IIRC, that's alphabetical order. If the order is important, you should either name the files appropriately (01file.txt, 02file.txt, etc...) or specify each file in the order you want it concatenated.

$ cat file1 file2 file3 file4 file5 file6 > out.txt

Solution 3 - Bash

You can use Windows shell copy to concatenate files.

C:\> copy *.txt outputfile

From the help:

> To append files, specify a single file for destination, but multiple files for source (using wildcards or file1+file2+file3 format).

Solution 4 - Bash

The Windows shell command type can do this:

type *.txt > outputfile.txt

Type type command also writes file names to stderr, which are not captured by the > redirect operator (but will show up on the console).

Solution 5 - Bash

Be careful, because none of these methods work with a large number of files. Personally, I used this line:

for i in $(ls | grep ".txt");do cat $i >> output.txt;done

EDIT: As someone said in the comments, you can replace $(ls | grep ".txt") with $(ls *.txt)

EDIT: thanks to @gnourf_gnourf expertise, the use of glob is the correct way to iterate over files in a directory. Consequently, blasphemous expressions like $(ls | grep ".txt") must be replaced by *.txt (see the article here).

Good Solution

for i in *.txt;do cat $i >> output.txt;done

Solution 6 - Bash

How about this approach?

find . -type f -name '*.txt' -exec cat {} + >> output.txt

Solution 7 - Bash

the most pragmatic way with the shell is the cat command. other ways include,

awk '1' *.txt > all.txt
perl -ne 'print;' *.txt > all.txt

Solution 8 - Bash

type [source folder]\*.[File extension] > [destination folder]\[file name].[File extension]

For Example:

type C:\*.txt > C:\1\all.txt

That will Take all the txt files in the C:\ Folder and save it in C:\1 Folder by the name of all.txt

Or

type [source folder]\* > [destination folder]\[file name].[File extension]

For Example:

type C:\* > C:\1\all.txt

That will take all the files that are present in the folder and put there Content in C:\1\all.txt

Solution 9 - Bash

The most upvoted answers will fail if the file list is too long.

A more portable solution would be using fd

fd -e txt -d 1 -X awk 1 > combined.txt

-d 1 limits the search to the current directory. If you omit this option then it will recursively find all .txt files from the current directory.
-X (otherwise known as --exec-batch) executes a command (awk 1 in this case) for all the search results at once.

> Note, fd is not a "standard" Unix program, so you will likely need to install it

Solution 10 - Bash

You can do like this: cat [directory_path]/**/*.[h,m] > test.txt

if you use {} to include the extension of the files you want to find, there is a sequencing problem.

Solution 11 - Bash

When you run into a problem where it cats all.txt into all.txt, You can try check all.txt is existing or not, if exists, remove

Like this:

[ -e $"all.txt" ] && rm $"all.txt"

Solution 12 - Bash

all of that is nasty....

ls | grep *.txt | while read file; do cat $file >> ./output.txt; done;

easy stuff.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionYadaView Question on Stackoverflow
Solution 1 - BashRobert GreinerView Answer on Stackoverflow
Solution 2 - BashChinmay KanchiView Answer on Stackoverflow
Solution 3 - BashCarl NorumView Answer on Stackoverflow
Solution 4 - BashGreg HewgillView Answer on Stackoverflow
Solution 5 - BashjackfizeeView Answer on Stackoverflow
Solution 6 - BashGPrathapView Answer on Stackoverflow
Solution 7 - Bashghostdog74View Answer on Stackoverflow
Solution 8 - BashOriView Answer on Stackoverflow
Solution 9 - BashMichael HallView Answer on Stackoverflow
Solution 10 - BashMichael-zhangView Answer on Stackoverflow
Solution 11 - BashleoView Answer on Stackoverflow
Solution 12 - BashkSiRView Answer on Stackoverflow