Copy folder recursively, excluding some folders

I am trying to write a simple bash script that will copy the entire contents of a folder including hidden files and folders into another folder, but I want to exclude certain specific folders. How could I achieve this?

Use rsync:

rsync -av --exclude='path1/to/exclude' --exclude='path2/to/exclude' source destination

Note that using source and source/ are different. A trailing slash means to copy the contents of the folder source into destination. Without the trailing slash, it means copy the folder source into destination.

Alternatively, if you have lots of directories (or files) to exclude, you can use --exclude-from=FILE, where FILE is the name of a file containing files or directories to exclude.

--exclude may also contain wildcards, such as --exclude=*/.svn*

Use tar along with a pipe.

cd /source_directory
tar cf - --exclude=dir_to_exclude . | (cd /destination && tar xvf - )

You can even use this technique across ssh.

You can use find with the -prune option.

An example from man find:

cd /source-dir
find . -name .snapshot -prune -o ( ! -name *~ -print0 )|
cpio -pmd0 /dest-dir

   This command copies the contents of /source-dir to /dest-dir, but omits
   files  and directories named .snapshot (and anything in them).  It also
   omits files or directories whose name ends in ~,  but  not  their  con‐
   tents.  The construct -prune -o \( ... -print0 \) is quite common.  The
   idea here is that the expression before -prune matches things which are
   to  be  pruned.  However, the -prune action itself returns true, so the
   following -o ensures that the right hand side  is  evaluated  only  for
   those  directories  which didn't get pruned (the contents of the pruned
   directories are not even visited, so their  contents  are  irrelevant).
   The  expression on the right hand side of the -o is in parentheses only
   for clarity.  It emphasises that the -print0 action  takes  place  only
   for  things  that  didn't  have  -prune  applied  to them.  Because the
   default `and' condition between tests binds more tightly than -o,  this
   is  the  default anyway, but the parentheses help to show what is going
   on.

you can use tar, with --exclude option , and then untar it in destination. eg

cd /source_directory
tar cvf test.tar --exclude=dir_to_exclude *
mv test.tar /destination 
cd /destination  
tar xvf test.tar

see the man page of tar for more info

Similar to Jeff's idea (untested):

find . -name * -print0 | grep -v "exclude" | xargs -0 -I {} cp -a {} destination/

 Quick Start

Run:

rsync -av --exclude='path1/in/source' --exclude='path2/in/source' [source]/ [destination]

---

 Notes

• -avr will create a new directory named [destination].
• source and source/ create different results:
  • source — copy the contents of source into destination.
  • source/ — copy the folder source into destination.
• To exclude many files:
  • --exclude-from=FILE — FILE is the name of a file containing other files or directories to exclude.
• --exclude may also contain wildcards:
  • e.g. --exclude=*/.svn*

Modified from: https://stackoverflow.com/a/2194500/749232

---

 Example

Starting folder structure:

.
├── destination
└── source
    ├── fileToCopy.rtf
    └── fileToExclude.rtf

Run:

rsync -av --exclude='fileToCopy.rtf' source/ destination

Ending folder structure:

.
├── destination
│   └── fileToExclude.rtf
└── source
    ├── fileToCopy.rtf
    └── fileToExclude.rtf

Simple solution (but I would still prefer the bash pattern matching from the top comments):

touch /path/to/target/.git
cp -n -ax * /path/to/target/
rm /path/to/target/.git

This exploits the -n option of cp, which forces cp to not overwrite existing targets.

Drawback: Works with GNU cp. If you don't have GNU cp, then the cp operation might return an error code (1), which is annoying because then you can't tell if it was a real failure.

EXCLUDE="foo bar blah jah"                                                                             
DEST=$1

for i in *
do
    for x in $EXCLUDE
    do  
        if [ $x != $i ]; then
            cp -a $i $DEST
        fi  
    done
done

Untested...

inspired by @SteveLazaridis's answer, which would fail, here is a POSIX shell function - just copy and paste into a file named cpx in yout $PATH and make it executible (chmod a+x cpr). [Source is now maintained in my GitLab.

#!/bin/sh

# usage: cpx [-n|--dry-run] "from_path" "to_path" "newline_separated_exclude_list"
# limitations: only excludes from "from_path", not it's subdirectories

cpx() {
# run in subshell to avoid collisions
  (_CopyWithExclude "$@")
}

_CopyWithExclude() {
  case "$1" in
    -n|--dry-run) { DryRun='echo'; shift; } ;;
  esac

  from="$1"
  to="$2"
  exclude="$3"

  $DryRun mkdir -p "$to"

  if [ -z "$exclude" ]; then
	  cp "$from" "$to"
	  return
  fi

  ls -A1 "$from" \
	| while IFS= read -r f; do
	    unset excluded
	    if [ -n "$exclude" ]; then
  	      for x in $(printf "$exclude"); do
          if [ "$f" = "$x" ]; then
              excluded=1
        	  break
          fi
  	      done
	    fi
	    f="${f#$from/}"
	    if [ -z "$excluded" ]; then
  	      $DryRun cp -R "$f" "$to"
	    else
  	      [ -n "$DryRun" ] && echo "skip '$f'"
	    fi
      done
}

# Do not execute if being sourced
[ "${0#*cpx}" != "$0" ] && cpx "$@"

Example usage

EXCLUDE="
.git
my_secret_stuff
"
cpr "$HOME/my_stuff" "/media/usb" "$EXCLUDE"