How do I mirror a directory with wget without creating parent directories?

LinuxFtpWgetMirror

Linux Problem Overview


I want to mirror a folder via FTP, like this:

wget --mirror --user=x --password=x ftp://ftp.site.com/folder/subfolder/evendeeper

But I do not want to create a directory structure like this:

>ftp.site.com -> folder -> subfolder -> evendeeper

I just want:

>evendeeper

And anything below it to be the resulting structure. It would also be acceptable for the contents of evendeeper to wind up in the current directory as long as subdirectories are created for subdirectories of evendeeper on the server.

I am aware of the -np option, according to the documentation that just keeps it from following links to parent pages (a non-issue for the binary files I'm mirroring via FTP). I am also aware of the -nd option, but this prevents creating any directory structure at all, even for subdirectories of evendeeper.

I would consider alternatives as long as they are command-line-based, readily available as Ubuntu packages and easily automated like wget.

Linux Solutions


Solution 1 - Linux

For a path like: ftp.site.com/a/b/c/d

-nH would download all files to the directory a/b/c/d in the current directory, and -nH --cut-dirs=3 would download all files to the directory d in the current directory.

Solution 2 - Linux

-np (no parent) option will probably do what you want, tied in with -L 1 (I think, don't have a wget install before me), which limits the recursion to one level.

EDIT. ok. gah... maybe I should wait until I've had coffee.. There is a --cut or similar option, which allows you to "cut" a specified number of directories from the output path, so for /a/b/c/d, a cut of 2 would force wget to create c/d on your local machine

Solution 3 - Linux

Instead of using:

-nH --cut-dirs=1

use:

-nH --cut-dirs=100

This will cut more directories and no folders will be created.

Note: 100 = the number of folders to skip creating. You can change 100 to any number.

Solution 4 - Linux

I had a similar requirement and the following combination seems to be the perfect choice:

In the below example, all the files in http://url/dir1/dir2 (alone) are downloaded to local directory /dest/dir

wget  -nd -np -P /dest/dir --recursive http://url/dir1/dir2

Thanks @ffledgling for the hint on "-nd"

For the above example:

wget -nd -np --mirror --user=x --password=x ftp://ftp.site.com/folder/subfolder/evendeeper

Snippets from manual:

   -nd
   --no-directories
       Do not create a hierarchy of directories when retrieving recursively.  With this option turned on, all files will get saved to the current directory, without clobbering (if a name shows up more than once, the
       filenames will get extensions .n).


   -np
   --no-parent
       Do not ever ascend to the parent directory when retrieving recursively.  This is a useful option, since it guarantees that only the files below a certain hierarchy will be downloaded.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionTom BoutellView Question on Stackoverflow
Solution 1 - Linuxvs_infView Answer on Stackoverflow
Solution 2 - LinuxMarc BView Answer on Stackoverflow
Solution 3 - LinuxTony TCGView Answer on Stackoverflow
Solution 4 - Linuxpr-palView Answer on Stackoverflow