Github actions share workspace/artifacts between jobs?

GithubContinuous IntegrationGithub Actions

Github Problem Overview


Trying to use Github's beta actions, I have two jobs, one that builds the code and then one that will deploy code. However, I can't seem to get the build artifact in deploy job.

My latest attempt is to manually set a container image with the same volumes for each job, according to docs this should be solution: https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idcontainervolumes

> Sets an array of volumes for the container to use. You can use volumes to share data between services or other steps in a job. You can specify named Docker volumes, anonymous Docker volumes, or bind mounts on the host.

Workflow

name: CI
on:
  push:
    branches:
    - master
    paths:
    - .github/workflows/server.yml
    - server/*
jobs:
  build:
    runs-on: ubuntu-latest
    container:
      image: docker://node:10
      volumes:
      - /workspace:/github/workspace
    steps:
    - uses: actions/checkout@master
    - run: yarn install
      working-directory: server
    - run: yarn build
      working-directory: server
    - run: yarn test
      working-directory: server
    - run: ls
      working-directory: server
  deploy:
    needs: build
    runs-on: ubuntu-latest
    container:
      image: docker://google/cloud-sdk:latest
      volumes:
      - /workspace:/github/workspace
    steps:
      - uses: actions/checkout@master
      - run: ls
        working-directory: server
      - run: gcloud --version

The first job (build) has a build directory, but when the second job (deploy) runs it doesn't and only contains the source code.

This project is a mono repo with code I'm trying to deploy being under path server hence all the working-directory flags.

Github Solutions


Solution 1 - Github

You can use the Github Actions upload-artifact and download-artifact to share data between jobs.

In job1:

steps:
- uses: actions/checkout@v1

- run: mkdir -p path/to/artifact

- run: echo hello > path/to/artifact/world.txt

- uses: actions/upload-artifact@master
  with:
    name: my-artifact
    path: path/to/artifact

And job2:

steps:
- uses: actions/checkout@master

- uses: actions/download-artifact@master
  with:
    name: my-artifact
    path: path/to/artifact
    
- run: cat path/to/artifact/world.txt

https://github.com/actions/upload-artifact
https://github.com/actions/download-artifact

Solution 2 - Github

If you are using the upload/download GitHub Actions, beware of the structure of the artifact.

Starting January 2020, see "GitHub Actions: Changes to artifact download experience":

> We have changed the artifact download experience in GitHub Actions so it no longer adds an extra root directory to the downloaded archive.

> Previously, if you uploaded the following files and folders as an artifact named foo, the downloaded archive would contain the following structure:

> foo/ |-- file1.txt |-- dir1/ | |-- dir1-file1.txt

> Now, you will get an archive that only contains the files and folders you uploaded:

> file1.txt dir1/ |-- dir1-file1.txt

Solution 3 - Github

For those interested in sharing a Docker image between two jobs, here is how I did it:

jobs:
  docker-build:
    name: Docker build
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Build Docker image
        run: |
          docker build -t foo/bar:$GITHUB_SHA
          mkdir -p path/to/artifacts
          docker save foo/bar:$GITHUB_SHA > path/to/artifacts/docker-image.tar
          
      - name: Temporarily save Docker image
        uses: actions/upload-artifact@v2
        with:
          name: docker-artifact
          path: path/to/artifacts
          retention-days: 1

  docker-deploy:
    name: Deploy to Docker Hub
    runs-on: ubuntu-latest
    needs: docker-build
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Retrieve saved Docker image
        uses: actions/download-artifact@v2
        with:
          name: docker-artifact
          path: path/to/artifacts

      - name: Docker load
        run: |
          cd path/to/artifacts
          docker load < docker-image.tar
          # docker_build_push.sh

Very inspired by https://github.com/unfor19/install-aws-cli-action/actions/runs/400601222/workflow

Merci @unfor19

Solution 4 - Github

Use Cache or Artifacts Upload/Download

Caching is used to re-use data/files between jobs or workflows while Artifacts are used to save files after workflow ended.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionLabithiotisView Question on Stackoverflow
Solution 1 - GithubTyler CarberryView Answer on Stackoverflow
Solution 2 - GithubVonCView Answer on Stackoverflow
Solution 3 - GithubThomasView Answer on Stackoverflow
Solution 4 - GithubPsijicView Answer on Stackoverflow