How to lock multiple stages of declarative Jenkins pipeline?

JenkinsJenkins Pipeline

Jenkins Problem Overview


I want to run multiple stages inside a lock within a declarative Jenkins pipeline:

pipeline {
    agent any
    stages {
        lock(resource: 'myResource') {
            stage('Stage 1') {
                steps {
                  echo "my first step"
                }
            }

            stage('Stage 2') {
                steps {
                  echo "my second step"
                }
            }

        }
    }
}

I get the following error:

Started by user anonymous
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 10: Expected a stage @ line 10, column 9.
           lock(resource: 'myResource') {
           ^

WorkflowScript: 10: Stage does not have a name @ line 10, column 9.
           lock(resource: 'myResource') {
           ^

WorkflowScript: 10: Nothing to execute within stage "null" @ line 10, column 9.
           lock(resource: 'myResource') {
           ^

3 errors

	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:116)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:430)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:393)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:257)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:405)
Finished: FAILURE

What's the problem here? The documentation explicitly states:

> lock can be also used to wrap multiple stages into a single > concurrency unit

Jenkins Solutions


Solution 1 - Jenkins

It should be noted that you can lock all stages in a pipeline by using the lock option:

pipeline {
    agent any
    options {
        lock resource: 'shared_resource_lock'
    }
    stages {
        stage('will_already_be_locked') {
            steps {
                echo "I am locked before I enter the stage!"
            }
        }
        stage('will_also_be_locked') {
            steps {
                echo "I am still locked!"
            }
        }
    }
}

Solution 2 - Jenkins

This has since been addressed.

You can lock multiples stages by grouping them in a parent stage, like this :

stage('Parent') {
  options {
    lock('something')
  }
  stages {
    stage('one') {
      ...
    }
    stage('two') {
      ...
    }
  }
}

(Don't forget you need the Lockable Resources Plugin)

Solution 3 - Jenkins

The problem is that, despite the fact that declarative pipelines were technically available in beta in September, 2016, the blog post you reference (from October) is documenting scripted pipelines, not declarative (it doesn't say as much, so I feel your pain). Lockable resources hasn't been baked in as a declarative pipeline step in a way that would enable the feature you're looking for yet.

You can do:

pipeline {
  agent { label 'docker' }
  stages {
    stage('one') {
      steps {
        lock('something') {
          echo 'stage one'
        }
      }
    }
  }
}

But you can't do:

pipeline {
  agent { label 'docker' }
  stages {
    lock('something') {
      stage('one') {
        steps {
          echo 'stage one'
        }
      }
      stage('two') {
        steps {
          echo 'stage two'
        }
      }
    }
  }
}

And you can't do:

pipeline {
  agent { label 'docker' }
  stages {
    stage('one') {
      lock('something') {
        steps {
          echo 'stage one'
        }
      }
    }
  }
}

You could use a scripted pipeline for this use case.

Solution 4 - Jenkins

If the resource is only used by this pipeline you could also disable concurrent builds:

pipeline {
    agent any
    options {
        disableConcurrentBuilds()
    }
    stages {
        stage('will_already_be_locked') {
            steps {
                echo "I am locked before I enter the stage!"
            }
        }
        stage('will_also_be_locked') {
            steps {
                echo "I am still locked!"
            }
        }
   }
}

Solution 5 - Jenkins

I run multiple build and test containers on the same build nodes. The test containers must lock up the node name as db username for the tests.

lock(resource: "${env.NODE_NAME}" as String, variable: 'DBUSER')

Locks in options are computed at load time, but NODE_NAME is unknown that early. In order to lock multiple stages for visual effect, we can create stages inside script block, i.e. 'run test' stage in the snippet. The stage visualization is just as good as other stage blocks.

pipeline {
    agent any
	stages {
		stage('refresh') {
			steps {
				echo "freshing on $NODE_NAME"
				lock(resource: "${env.NODE_NAME}" as String, variable: 'DBUSER') {
					sh '''
						printenv | sort
					'''
					script {
						stage('run test')
						sh '''
							printenv | sort
						'''
					}
				}
			}
		}
	}
}

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionJohnView Question on Stackoverflow
Solution 1 - JenkinsdolphyView Answer on Stackoverflow
Solution 2 - JenkinsKeatsPeeksView Answer on Stackoverflow
Solution 3 - JenkinsburnettkView Answer on Stackoverflow
Solution 4 - JenkinsTorsten WalterView Answer on Stackoverflow
Solution 5 - Jenkinswayne sView Answer on Stackoverflow