Github actions share workspace/artifacts between jobs?
GithubContinuous IntegrationGithub ActionsGithub Problem Overview
Trying to use Github's beta actions, I have two jobs, one that builds the code and then one that will deploy code. However, I can't seem to get the build artifact in deploy job.
My latest attempt is to manually set a container image with the same volumes for each job, according to docs this should be solution: https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idcontainervolumes
> Sets an array of volumes for the container to use. You can use volumes to share data between services or other steps in a job. You can specify named Docker volumes, anonymous Docker volumes, or bind mounts on the host.
Workflow
name: CI
on:
push:
branches:
- master
paths:
- .github/workflows/server.yml
- server/*
jobs:
build:
runs-on: ubuntu-latest
container:
image: docker://node:10
volumes:
- /workspace:/github/workspace
steps:
- uses: actions/checkout@master
- run: yarn install
working-directory: server
- run: yarn build
working-directory: server
- run: yarn test
working-directory: server
- run: ls
working-directory: server
deploy:
needs: build
runs-on: ubuntu-latest
container:
image: docker://google/cloud-sdk:latest
volumes:
- /workspace:/github/workspace
steps:
- uses: actions/checkout@master
- run: ls
working-directory: server
- run: gcloud --version
The first job (build) has a build directory, but when the second job (deploy) runs it doesn't and only contains the source code.
This project is a mono repo with code I'm trying to deploy being under path server
hence all the working-directory
flags.
Github Solutions
Solution 1 - Github
You can use the Github Actions upload-artifact and download-artifact to share data between jobs.
In job1:
steps:
- uses: actions/checkout@v1
- run: mkdir -p path/to/artifact
- run: echo hello > path/to/artifact/world.txt
- uses: actions/upload-artifact@master
with:
name: my-artifact
path: path/to/artifact
And job2:
steps:
- uses: actions/checkout@master
- uses: actions/download-artifact@master
with:
name: my-artifact
path: path/to/artifact
- run: cat path/to/artifact/world.txt
https://github.com/actions/upload-artifact
https://github.com/actions/download-artifact
Solution 2 - Github
If you are using the upload/download GitHub Actions, beware of the structure of the artifact.
Starting January 2020, see "GitHub Actions: Changes to artifact download experience":
> We have changed the artifact download experience in GitHub Actions so it no longer adds an extra root directory to the downloaded archive.
> Previously, if you uploaded the following files and folders as an artifact named foo
, the downloaded archive would contain the following structure:
> foo/ |-- file1.txt |-- dir1/ | |-- dir1-file1.txt
> Now, you will get an archive that only contains the files and folders you uploaded:
> file1.txt dir1/ |-- dir1-file1.txt
Solution 3 - Github
For those interested in sharing a Docker image between two jobs, here is how I did it:
jobs:
docker-build:
name: Docker build
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Build Docker image
run: |
docker build -t foo/bar:$GITHUB_SHA
mkdir -p path/to/artifacts
docker save foo/bar:$GITHUB_SHA > path/to/artifacts/docker-image.tar
- name: Temporarily save Docker image
uses: actions/upload-artifact@v2
with:
name: docker-artifact
path: path/to/artifacts
retention-days: 1
docker-deploy:
name: Deploy to Docker Hub
runs-on: ubuntu-latest
needs: docker-build
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Retrieve saved Docker image
uses: actions/download-artifact@v2
with:
name: docker-artifact
path: path/to/artifacts
- name: Docker load
run: |
cd path/to/artifacts
docker load < docker-image.tar
# docker_build_push.sh
Very inspired by https://github.com/unfor19/install-aws-cli-action/actions/runs/400601222/workflow
Merci @unfor19
Solution 4 - Github
Use Cache or Artifacts Upload/Download
Caching is used to re-use data/files between jobs or workflows while Artifacts are used to save files after workflow ended.