Moving large number of files
BashBash Problem Overview
If I run the command mv folder2/*.* folder
, I get "argument list too long" error.
I find some example of ls
and rm
, dealing with this error, using find folder2 -name "*.*"
. But I have trouble applying them to mv
.
Bash Solutions
Solution 1 - Bash
find folder2 -name '*.*' -exec mv {} folder \;
-exec
runs any command, {}
inserts the filename found, \;
marks the end of the exec command.
Solution 2 - Bash
The other find
answers work, but are horribly slow for a large number of files, since they execute one command for each file. A much more efficient approach is either to use +
at the end of find
, or use xargs
:
# Using find ... -exec +
find folder2 -name '*.*' -exec mv --target-directory=folder '{}' +
# Using xargs
find folder2 -name '*.*' | xargs mv --target-directory=folder
Solution 3 - Bash
find folder2 -name '*.*' -exec mv \{\} /dest/directory/ \;
Solution 4 - Bash
First, thanks to Karl's answer. I have only minor correction to this.
My scenario:
Millions of folders inside /source/directory, containing subfolders and files inside. Goal is to copy it keeping the same directory structure.
To do that I use such command:
find /source/directory -mindepth 1 -maxdepth 1 -name '*' -exec mv {} /target/directory \;
Here:
- -mindepth 1 : makes sure you don't move root folder
- -maxdepth 1 : makes sure you search only for first level children. So all it's content is going to be moved too, but you don't need to search for it.
Commands suggested in answers above made result directory structure flat - and it was not what I looked for, so decided to share my approach.
Solution 5 - Bash
This one-liner command should work for you. Yes, it is quite slow, but works even with millions of files.
for i in /folder1/*; do mv "$i" /folder2; done
It will move all the files from folder /folder1
to /folder2
.
Solution 6 - Bash
find doesn't work with really long lists of files, it will give you the same error "Argument list too long". Using a combination of ls, grep and xargs worked for me:
$ ls|grep RadF|xargs mv -t ../fd/
It did the trick moving about 50,000 files where mv and find alone failed.