How to download GitHub Release from private repo using command line
ShellGithubCommand LineShell Problem Overview
GitHub guide explains 2 way to authorize but looks neither of those works with the Release files.
as a result of:
curl -u 'username' -L -o a.tgz https://github.com/company/repository/releases/download/TAG-NAME/A.tgz
There always is something like
<!DOCTYPE html>
<!--
Hello future GitHubber! ...
Shell Solutions
Solution 1 - Shell
To download release file from private repo, you can use Personal access token which can be generated at settings/tokens with Full control of private repositories scope.
Then download the asset with curl
command (change with appropriate values):
curl -vLJO -H 'Authorization: token my_access_token' 'https://api.github.com/repos/:owner/:repo/releases/assets/:id'
or if you're using an OAuth app, use:
curl -u my_client_id:my_client_secret https://api.github.com/repos/:owner/:repo/releases/assets/:id
where:
-
:owner
is your user or organisation username; -
:repo
is your repository name; -
:id
is your asset id, can be found in tag release URL, like:https://api.github.com/repos/:owner/:repo/releases/tags/:tag
-
:token
is your personal access token (can be created at/settings/tokens
;
Note: Using access_token
as a query param is deprecated.
See: Repositories API v3 at GitHub
Here is the Bash script which can download asset file given specific name of file:
#!/usr/bin/env bash
# Script to download asset file from tag release using GitHub API v3.
# See: http://stackoverflow.com/a/35688093/55075
CWD="$(cd -P -- "$(dirname -- "$0")" && pwd -P)"
# Check dependencies.
set -e
type curl grep sed tr >&2
xargs=$(which gxargs || which xargs)
# Validate settings.
[ -f ~/.secrets ] && source ~/.secrets
[ "$GITHUB_API_TOKEN" ] || { echo "Error: Please define GITHUB_API_TOKEN variable." >&2; exit 1; }
[ $# -ne 4 ] && { echo "Usage: $0 [owner] [repo] [tag] [name]"; exit 1; }
[ "$TRACE" ] && set -x
read owner repo tag name <<<$@
# Define variables.
GH_API="https://api.github.com"
GH_REPO="$GH_API/repos/$owner/$repo"
GH_TAGS="$GH_REPO/releases/tags/$tag"
AUTH="Authorization: token $GITHUB_API_TOKEN"
WGET_ARGS="--content-disposition --auth-no-challenge --no-cookie"
CURL_ARGS="-LJO#"
# Validate token.
curl -o /dev/null -sH "$AUTH" $GH_REPO || { echo "Error: Invalid repo, token or network issue!"; exit 1; }
# Read asset tags.
response=$(curl -sH "$AUTH" $GH_TAGS)
# Get ID of the asset based on given name.
eval $(echo "$response" | grep -C3 "name.:.\+$name" | grep -w id | tr : = | tr -cd '[[:alnum:]]=')
#id=$(echo "$response" | jq --arg name "$name" '.assets[] | select(.name == $name).id') # If jq is installed, this can be used instead.
[ "$id" ] || { echo "Error: Failed to get asset id, response: $response" | awk 'length($0)<100' >&2; exit 1; }
GH_ASSET="$GH_REPO/releases/assets/$id"
# Download asset file.
echo "Downloading asset..." >&2
curl $CURL_ARGS -H "Authorization: token $GITHUB_API_TOKEN" -H 'Accept: application/octet-stream' "$GH_ASSET"
echo "$0 done." >&2
Before running, you need to set your GITHUB_API_TOKEN
with your GitHub token (see: /settings/tokens
at GH). This can be placed in your ~/.secrets
file, like:
GITHUB_API_TOKEN=XXX
Example script usage:
./get_gh_asset.sh :owner :repo :tag :name
where name is your filename (or partial of it). Prefix script with TRACE=1
to debug it.
In case you wonder why curl
fails sometimes with (as mentioned in other answer):
> Only one auth mechanism allowed; only the X-Amz-Algorithm
query parameter, Signature query string parameter or the Authorization
header should be specified.
when running like:
curl -vLJ -H 'Authorization: token <token>' -H 'Accept: application/octet-stream' https://api.github.com/repos/:owner/:repo/releases/assets/<id>
this is because you're specifying multiple mechanism at the same time, so S3 server doesn't know which one to use, therefore you have to choose only one, such as:
X-Amz-Algorithm
query parameter- Signature query string parameter (
X-Amz-Signature
) - Authorization header (
Authorization: token <token>
)
and since GitHub redirects you from asset page (when requesting application/octet-stream
), it populates credentials automatically in query string and since curl
is passing over the same credentials in the request header (which you've specified), therefore they're conflicting. So as for workaround you can use access_token
instead.
Solution 2 - Shell
We had to download release assets from private GitHub repos fairly often, so we created fetch, which is an open source, cross-platform tool that makes it easy to download source files and release assets from a git tag, commit, or branch of public and private GitHub repos.
For example, to download the release asset foo.exe
from version 0.1.3
of a private GitHub repo to /tmp
, you would do the following:
GITHUB_OAUTH_TOKEN="your token"
fetch --repo="https://github.com/foo/bar" --tag="0.1.3" --release-asset="foo.exe" /tmp
Solution 3 - Shell
Seems both authentication methods only work for the API endpoints. There's an API endpoint for downloading release assets (docu):
GET /repos/:owner/:repo/releases/assets/:id
But one will need to know the asset's numerical ID for that. I asked their support whether they could add an API endpoint to download release assets by name (like they have for tarballs).
Update: Response from Github support: > We don't offer any API like you're suggesting. We try to avoid brittle URLs that would change if the tag or filename of the release asset change. I'll take your feedback to the team to discuss further.
Solution 4 - Shell
Here is curl
& jq
one;)liner:
CURL="curl -H 'Authorization: token <auth_token>' \
https://api.github.com/repos/<owner>/<repo>/releases"; \
ASSET_ID=$(eval "$CURL/tags/<tag>" | jq .assets[0].id); \
eval "$CURL/assets/$ASSET_ID -LJOH 'Accept: application/octet-stream'"
Change parts surrounded with <>
with your data.
To generate auth_token
go to github.com/settings/tokens
If you like to login with password use this (note it will ask for password twice):
CURL="curl -u <github_user> https://api.github.com/repos/<owner>/<repo>/releases"; \
ASSET_ID=$(eval "$CURL/tags/<tag>" | jq .assets[0].id); \
eval "$CURL/assets/$ASSET_ID -LJOH 'Accept: application/octet-stream'"
Solution 5 - Shell
I found out the answer in this comment: https://github.com/request/request/pull/1058#issuecomment-55285276
curl
is forwarding the authentication header in the request to the AmazonS3 bucket, where the Github release assets are stored.
Error response from S3:
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>InvalidArgument</Code>
<Message>
Only one auth mechanism allowed; only the X-Amz-Algorithm query parameter, Signature query string parameter or the Authorization header should be specified
</Message>
<ArgumentName>Authorization</ArgumentName>
<ArgumentValue>token <yourtoken> </ArgumentValue><RequestId>4BEDDBA630688865</RequestId> <HostId>SXLsRKgKM6tPa/K7g7tSOWmQEqowG/4kf6cwOmnpObXrSzUt4bzOFuihmzxK6+gx</HostId>
</Error>
One line wget solution:
wget --auth-no-challenge --header='Accept:application/octet-stream' https://<token>:@api.github.com/repos/:owner/:repo/releases/assets/:id -O app.zip
Try:curl -i -H "Authorization: token <token>" -H "Accept:application/octet-stream" https://<token>:@api.github.com/repos/:owner/:repo/releases/assets/:id
, for some more details. Add -L
to see the S3 error message.
Solution 6 - Shell
Here is a "one-liner" using wget
for making HTTP requests and python
for JSON parsing:
(export AUTH_TOKEN=<oauth-token>; \
export ASSET_ID=$(wget -O - https://api.github.com/repos/<owner>/<repo>/releases/tags/<tag>?access_token=$AUTH_TOKEN | python -c 'import sys, json; print json.load(sys.stdin)["assets"][0]["id"]'); \
wget --header='Accept:application/octet-stream' -O <download-name> https://api.github.com/repos/<owner>/<repo>/releases/assets/$ASSET_ID?access_token=$AUTH_TOKEN)
To use it, just replace <oauth-token>
, <owner>
, <repo>
, <tag>
and <download-name>
with appropriate values.
Explanation:
- The first statement (
export AUTH_TOKEN=<oauth-token>
) sets GitHub OAuth token which is used by the subsequentwget
commands. - The second statement has two parts:
- The
wget -O - https://api.github.com/repos/<owner>/<repo>/releases/tags/<tag>?access_token=$AUTH_TOKEN
part gets GitHub release information from a tag name and prints it on stdout.
- The
- The
python -c 'import sys, json; print json.load(sys.stdin)["assets"][0]["id"]'
part parses JSON from stdin and extracts theid
of the (first) release asset.
- The third statement (
wget --header='Accept:application/octet-stream' -O <tarball-name>.tar.gz https://api.github.com/repos/<owner>/<repo>/releases/assets/$ASSET_ID?access_token=$AUTH_TOKEN)
) gets a single GitHub release asset by id and stores it to a file. - The outer parentheses create a subshell and ensure exported environment variables are discarded afterwards.
Solution 7 - Shell
An easier working solution is to use .netrc to store credentials. This way, curl does not forward credentials to Amazon S3 Bucket
In ~/.netrc
file (should be created with 0600 file permission):
machine api.github.com
login yourusername
password yourpassword
Then use the curl -n option to use .netrc:
curl -L -O -J -n -H "Accept:application/octet-stream" https://api.github.com/repos/:owner/:repo/releases/assets/:id
Solution 8 - Shell
Use gh release download to easily script downloads. gh auth login first to authorise.
So to download the examples URL https://github.com/company/repository/releases/download/TAG-NAME/A.tgz
use:
gh release download --repo company/repository TAG-NAME -p 'A.tgz'
Solution 9 - Shell
Like @dwayne I solved this using the gh CLI which you would also need to to install. As I was inside a docker container I needed to install curl
, dirmngr
, then the gh
CLI and then download the release like below. Also shows how to authenticate using a personal token to the gh cli, and unzip the release files (a tarball in my case).
FROM debian:10.9 as base
RUN apt update \
# Install package dependencies
&& apt install -y \
build-essential \
wget \
dirmngr \
curl
# Install GH CLI - see https://github.com/cli/cli/blob/trunk/docs/install_linux.md
RUN curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | gpg --dearmor -o /usr/share/keyrings/githubcli-archive-keyring.gpg && \
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | tee /etc/apt/sources.list.d/github-cli.list > /dev/null && \
apt update && \
apt install -y gh
# auth to github with PAT and download release
ARG RELEASE_TOKEN
RUN echo $RELEASE_TOKEN | gh auth login --with-token && \
cd /opt && \
gh release download YOUR_TAG_HERE --repo https://github.com/SOME_ORG/SOME-REPO && \
tar -xjf SOME-RELEASE-FILES.tar.bz2 -C /opt && \
rm SOME-RELEASE-FILES.tar.bz2