Curl to grab remote filename after following location

UnixCommand LineCurl

Unix Problem Overview


When downloading a file using curl, how would I follow a link location and use that for the output filename (without knowing the remote filename in advance)?

For example, if one clicks on the link below, you would download a filenamed "pythoncomplete.vim." However using curl's -O and -L options, the filename is simply the original remote-name, a clumsy "download_script.php?src_id=10872."

curl -O -L http://www.vim.org/scripts/download_script.php?src_id=10872

In order to download the file with the correct filename you would have to know the name of the file in advance:

curl -o pythoncomplete.vim -L http://www.vim.org/scripts/download_script.php?src_id=10872

It would be excellent if you could download the file without knowing the name in advance, and if not, is there another way to quickly pull down a redirected file via command line?

Unix Solutions


Solution 1 - Unix

The remote side sends the filename using the Content-Disposition header.

curl 7.21.2 or newer does this automatically if you specify --remote-header-name / -J.

curl -O -J -L $url

Solution 2 - Unix

If you have a recent version of curl (7.21.2 or later), see @jmanning2k's answer.

I you have an older version of curl (like 7.19.7 which came with Snow Leopard), do two requests: a HEAD to get the file name from response header, then a GET:

url="http://www.vim.org/scripts/download_script.php?src_id=10872"
filename=$(curl -sI  $url | grep -o -E 'filename=.*$' | sed -e 's/filename=//')
curl -o $filename -L $url

Solution 3 - Unix

If you can use wget instead of curl:

wget --content-disposition $url

Solution 4 - Unix

I wanted to comment to jmanning2k's answer but as a new user I can't, so I tried to edit his post which is allowed but the edit was rejected saying it was supposed to be a comment. sigh

Anyway, see this as a comment to his answer thanks.

This seems to only work if the header looks like filename=pythoncomplete.vim as in the example, but some sites send a header that looks like filename*=UTF-8' 'filename.zip' that one isn't recognized by curl 7.28.0

Solution 5 - Unix

I wanted a solution that worked on both older and newer Macs, and the legacy code David provided for Snow Leopard did not behave well under Mavericks. Here's a function I created based on David's code:

function getUriFilename() {
    header="$(curl -sI "$1" | tr -d '\r')"

    filename="$(echo "$header" | grep -o -E 'filename=.*$')"
    if [[ -n "$filename" ]]; then
        echo "${filename#filename=}"
        return
    fi

    filename="$(echo "$header" | grep -o -E 'Location:.*$')"
    if [[ -n "$filename" ]]; then
        basename "${filename#Location\:}"
        return
    fi

    return 1
}

With this defined, you can run:

url="http://www.vim.org/scripts/download_script.php?src_id=10872"
filename="$(getUriFilename $url)"
curl -L $url -o "$filename"

Solution 6 - Unix

Please note that certain malconfigured webservers will serve the name using "Filename" as key, where RFC2183 specifies it should be "filename". curl only handles the latter case.

Solution 7 - Unix

An example using the answer above for Apache Archiva artifact repository to pull latest version. The curl returns the Location line and the filename is at the end of the line. Need to remove the CR at end of file name.

url="http://archiva:8080/restServices/archivaServices/searchService/artifact?g=com.imgur.backup&a=snapshot-s3-util&v=LATEST"
filename=$(curl --silent -sI -u user:password $url | grep Location | awk -F\/ '{print $NF}' | sed 's/\r$//')
curl --silent -o $filename -L -u user:password $url

Solution 8 - Unix

I had the same Problem like John Cooper. I got no filename but a Location File name back. His answer also worked but are 2 commands. This oneliner worked for me....

url="https://download.mozilla.org/?product=firefox-latest-ssl&os=linux64&lang=de";url=$(curl -L --head -w '%{url_effective}' $url 2>/dev/null | tail -n1) ; curl -O $url

Stolen and added some stuff from https://unix.stackexchange.com/questions/126252/resolve-filename-from-a-remote-url-without-downloading-a-file

Solution 9 - Unix

Using the solution proposed above, I wrote this helper function curl2file

function curl2file() {
    url=$1
    url=$(curl -L --head -w '%{url_effective}' $url 2>/dev/null | tail -n1) ; curl -O $url
}

Usage:

curl2file https://cloud.tsinghua.edu.cn/f/4666d28af98a4e63afb5/?dl=1

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionNick S.View Question on Stackoverflow
Solution 1 - Unixjmanning2kView Answer on Stackoverflow
Solution 2 - UnixDavid J. LiszewskiView Answer on Stackoverflow
Solution 3 - UnixJacekMView Answer on Stackoverflow
Solution 4 - UnixDiskutantView Answer on Stackoverflow
Solution 5 - UnixChaim Leib HalbertView Answer on Stackoverflow
Solution 6 - UnixdraterView Answer on Stackoverflow
Solution 7 - UnixJohn CooperView Answer on Stackoverflow
Solution 8 - UnixJackfrittView Answer on Stackoverflow
Solution 9 - UnixloretoparisiView Answer on Stackoverflow