How to reuse a previously created workspace at different stages

Question

I ran into a problem when I have two steps defined in my pipeline that both are running on the same node and must be running in the same workspace.

The first of these steps starts with my master node initially, but by the end of the steps you need to expand some files to another node.

The second stage should simply continue on my host and rely on some modules that were installed from the first stage.

Here is my conveyor to better explain:

#!groovy pipeline { agent { label 'master' } stages { stage('Build') { // 1. Running on master in /var/lib/jenkins/workspace/_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ steps { sh ''' npm install bower install gulp set-staging-node-env gulp prepare-staging-files gulp webpack ''' stash includes: 'dist/**/*', name: 'builtSources' stash includes: 'config/**/*', name: 'appConfig' node('Protractor') { // 2. Running on vnccentos7 in /var/jenkins/workspace/_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ dir('/opt/foo/deploy/') { unstash 'builtSources' unstash 'appConfig' } } } } stage('Unit Tests') { agent { label 'master' } // 3. Running on master in /var/lib/jenkins/workspace/_Pipelines_IACT-J enkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ@ 2 steps { parallel ( "Jasmine": { sh 'gulp karma-tests-ci' }, "Mocha": { sh 'gulp mocha-tests' } ) } } } } 

As you can see, I added comments at the beginning of each \ node stage used to display the jenkins output that I see for which workspaces are allocated.

The problem I am facing is that the unit testing phase does not work as it is trying to use some node modules that it cannot find. They are present in the first workspace that is being created, and I want this stage to continue to be used, so I did not use the new suffix space '@ 2'.

Is there a way to tell Jenkins to save previously created workspaces in the pipeline?

EDIT

I guess, since I again indicated the agent {label:'master'} in my next step, is this what creates a new workspace? Should I use the node approach instead? Will this allow you to use the same workspace?

I really tried using node('master'){...} around each of the parallel steps in the Unit Tests step, but they still use the @ 2 suffix space, not the original one.

I saw other threads talking about how you should not reuse the same workspace as file lock problems. Instead, they suggest archiving the \ unarchive workspace between steps.

I also saw approaches in which you can save the workspace path in a variable and use it later, which works well for my case, but I did not find any declarative syntactic patterns, only groovy.

EDIT 2

I tried several approaches related to saving the selected workspace from the first stage to a variable and using the ws(...) directive in the following stages:

 pipeline { agent { label 'master' } stages { stage('Build') { steps { script { def workspace = pwd() } sh ''' npm install bower install gulp set-staging-node-env gulp prepare-staging-files gulp webpack ''' stash includes: 'dist/**/*', name: 'builtSources' stash includes: 'config/**/*', name: 'appConfig' node('Protractor') { dir('/opt/foo/deploy/') { unstash 'builtSources' unstash 'appConfig' } } } } stage('Unit Tests') { steps { parallel ( "Jasmine": { node('master') { ws("${workspace}"){ sh 'gulp karma-tests-ci' } } }, "Mocha": { node('master') { ws("${workspace}"){ sh 'gulp mocha-tests' } } } ) } post { success { sh 'gulp combine-coverage-reports' sh 'gulp clean-lcov' publishHTML(target: [ allowMissing: false, alwaysLinkToLastBuild: false, keepAll: false, reportDir: 'test/coverage', reportFiles: 'index.html', reportName: 'Test Coverage Report' ]) } } } } } 

I tried to simply remove the second agent declaration from the Unit Tests phase, but the stage remained on my Protractor node, which I did not want. Therefore, following the answers \ comments here, I used node blocks around each of my parallel steps and used ws blocks, as you can see,

The stage fails, and the logs show that it does not use the workspace allocated from the first stage (without @ suffixes):

 [Jasmine] Running on master in /var/lib/jenkins/workspace/_Pipelines_IACT-J enkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ@ 2 [Pipeline] [Jasmine] { [Pipeline] [Jasmine] ws [Jasmine] Running in /var/lib/jenkins/workspace/_Pipelines_IACT-J enkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ@ 2@2 [Pipeline] [Jasmine] { [Pipeline] [Jasmine] sh [Jasmine] [_Pipelines_IACT-J enkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ@ 2@2 ] Running shell script [Jasmine] + gulp karma-tests-ci [Jasmine] [08:27:01] No gulpfile found 

Its an even double suffix with @ 2, so I'm not sure what it is doing now.

Answer 1

Not sure if it suits your use case, but this example script shows how to split the same node / workspace between different steps and containers:

In addition, if you use the Docker agent for a specific stage by specifying the {label 'whatever'} agent at the top level, you can make sure that this stage will use the same node and workspace as the rest of the Pipeline:

 pipeline { agent { label 'whatever' } stages { stage('build') { steps { sh "./build-artifact.sh" } } stage('test in docker') { agent { docker { image 'ubuntu:16.04' reuseNode true } } steps { sh "./run-tests-in-docker.sh" } } } } 

https://github.com/jenkinsci/pipeline-model-definition-plugin/wiki/Controlling-your-build-environment#reusing-nodeworkspace-with-per-stage-docker-agents

Answer 2

Use dir instead of ws .

ws automatically changes the "@NUMBER" prefix when the workspace to be used is already taken by another build job.

dir just move the current working directory exactly where you specify.

 pipeline { agent none environment { WIN_WORKSPACE = "" MAC_WORKSPACE = "" } stages { stage("Build") { parallel { stage("Build on Windows") { agent { label "windows" } steps { script { WIN_WORKSPACE = WORKSPACE } // steps... } } stage("Build on macOS") { agent { label "macos" } steps { script { MAC_WORKSPACE = WORKSPACE } // steps... } } } } stage("Deploy") { parallel { stage("Deploy on Windows") { agent { label "windows" } steps { dir(WIN_WORKSPACE) { // steps... } } } stage("Deploy on macOS") { agent { label "macos" } steps { dir(MAC_WORKSPACE) { // steps... } } } } } } } 

Works well as you want.

Answer 3

I am using node syntax to solve it.

to be sure that I would use the dir area to set the workspace myself

Answer 4

Specify a custom workspace. From the Pipeline syntax: "ws: Allocate workspace"

 ws("/usr/local/jenkins/jobs/custom_workspace") { stage . . . stage . . . } 

., it works?

Answer 5

An external workspace manager can solve your problem.

[...] it determines the local path to the workspace and switches to it.

 // Basic usage: def extWorkspace = exwsAllocate diskPoolId: 'diskpool1' node ('linux') { exws (extWorkspace) { scm checkout sh 'mvn clean install -DskipTests' } } node ('test') { exws (extWorkspace) { sh 'mvn test' } }