What is the difference between a node, stage, and step in Jenkins pipelines?

JenkinsJenkins Pipeline

Jenkins Problem Overview


I'm trying to understand how to structure my Jenkins 2.7 pipeline groovy script. I've read through the pipeline tutorial, but feel that it could expand more on these topics.

I can understand that a pipeline can have many stages and each stage can have many steps. But what is the difference between a step(); and a method call inside a stage, say sh([script: "echo hello"]);. Should nodes be inside or outside of stages? Should the overall properties of a job be inside or outside a node?

Here is my current structure on an ubuntu master node:

#!/usr/bin/env groovy

node('master') {
    properties([
        [$class: 'BuildDiscarderProperty', strategy: [$class: 'LogRotator', numToKeepStr: '10']]
    ]);

    stage 'Checkout'
        checkout scm

    stage 'Build'
        sh([script: "make build"]);
    
    archive("bin/*");
}

Jenkins Solutions


Solution 1 - Jenkins

The concepts of node, stage and step are different:

  • node specifies where something shall happen. You give a name or a label, and Jenkins runs the block there.

  • stage structures your script into a high-level sequence. Stages show up as columns in the Pipeline Stage view with average stage times and colours for the stage result.

  • step is one way to specify what shall happen. sh is of a similar quality, it is a different kind of action. (You can also use build for things that are already specified as projects.)

So steps can reside within nodes, (if they don't, they are executed on the master), and nodes and steps can be structured into an overall sequence with stages.

Solution 2 - Jenkins

It depends. Any node declaration allocates an executor (on Jenkins master or slave). This requires that you stash and unstash the workspace, as another executor does not have the checked out sources available.

Several steps of the Pipeline DSL run in a flyweight executor and thus do not require to be inside a node block. This might be helpful for an example such as the following, where you need to allocate multiple nodes anyway:

stage("Checkout") {
  checkout scm
}

stage("Build") {
  node('linux') {
    sh "make"
  }
  node('windows') {
    bat "whatever"
  }
}

stage("Upload") {
  ...

Another (maybe more realistic example) would be to allocate multiple nodes in parallel. Then there's no need to let the stage call be executed in another allocated executor (aka within node).

Your example looks good to me. There's no need to allocate multiple nodes within the single stages, as this would be only additional overhead.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestiontarabyteView Question on Stackoverflow
Solution 1 - Jenkinschris_fView Answer on Stackoverflow
Solution 2 - JenkinsStephenKingView Answer on Stackoverflow