Jenkins build artifacts, create docker image and deploy over ssh

In this article we will configure Jenkins server to build some java application with ‘maven‘ and upload compiled artifact to the ‘Nexus‘ server.  We will create repository in the GitLab to store Dockerfile which, will download ‘openjdk:8‘ from official repository and add downloaded artifact inside of docker image and start jar file process. After preparation of docker image we will use official https://hub.docker.com as registry server to push image. Then we will deploy this docker image in the ‘WebDeploySRV‘ server. At the end we will create group of Pipeline which will do all jobs by one command.

Java application will be used WebGoat:
https://github.com/WebGoat/WebGoat

Our network topology will be as following:
jfdtopology

Before start to do all stuff we need install needed plugins to the Jenkins. Go to the Jenkins -> Manage Jenkins -> Manage Plugins -> Go to the Available tab and in in the search line type ‘Maven Integration Plugin
maven_integration_plugin

To not configure ‘docker login‘ for each of Jenkins slave nodes we will use ‘CloudBees Docker Build and Publish plugin‘ Jenkins plugin. For that repeat steps which we did before and install this plugin.
docker-build-and-publish-plugin

To remote execute SSH commands in the deployment server we need install ‘SSH plugin‘. For that repeat steps which we did before and install this plugin.
remote-execute-ssh-plugin

To execute all our ‘Jobs‘ with one button we need to install ‘Build Pipeline Plugin‘. For that repeat steps which we did before and install this plugin.
build-pipelin-plugin

Create credentials for docker registry(https://hub.docker.com) server in the Jenkins. Main menu ‘Jenkins‘ -> ‘Credentials‘ -> press ‘global
add-multiple-credentials

Add Credentials‘ -> In the kind select ‘Username with password‘ -> Scope ‘Global‘ -> Enter username, password, ID, Description and press ‘OK‘ button.
username-with-password-credential
username-with-password-credential1

Add one more credential to SSH authentication for deploy server (This credential will be used in the Jenkins project ‘AppWebGoat-Deploy‘ to deploy app in the ‘WebDeploySRV‘ server. Private SSH key is generated before. Please copy content of private key and paste here). In the ‘Kind‘ select ‘SSH Username with private key‘, username ‘root‘ and ‘Private key -> Enter directly‘ then press ‘OK’ button
sshkey-credential
sshkey-credential1

Go to the Jenkins main menu -> ‘Manage Jenkins‘ -> ‘Configure System‘ -> Find ‘SSH remote hosts‘ section and press ‘Add‘ button
sshremotehost-add

Enter IP address of WebDeploySRV server (IP: 192.168.106.240), add port 22,  select credentials ‘root‘ which we have already added for SSH, press ‘Check connection‘ (Result must be: Successful connection) button and ‘Apply‘ then ‘Save
sshremotehost-add1

In the https://hub.docker.com/ create new repository for our docker image.
create-docker-repo
create-docker-repo1

Copy the selected name. This name will be used from our Jenkins to push to the registry server.
copy-selected-reponame

From one of slave nodes copy path of docker socket file (This /var/run/docker.sock  path will be used in the Jenkins to connect to the docker process):

$ vagrant.exe ssh JenkinsSlave1 -c "netstat -na| grep docker.sock"
unix 2 [ACC] STREAM LISTENING 12338 /var/run/docker.sock

Set ‘root’ user password for GitLab server:

$ vagrant.exe ssh GitLab
Last login: Mon Feb  5 13:30:26 2018 from gateway
[vagrant@localhost ~]$ sudo su -
Last login: Mon Feb  5 13:22:32 -02 2018 on pts/0
[root@localhost ~]# echo 'R00tm1n123' | openssl passwd -1 -stdin
[root@localhost ~]# gitlab-rails console production
Loading production environment (Rails 4.2.10)
irb(main):001:0> u = User.where(id: 1).first
=> #
irb(main):002:0> u.password = 'R00tm1n123'
=> "R00tm1n123"
irb(main):003:0> u.password_confirmation = 'R00tm1n123'
=> "R00tm1n123"
irb(main):004:0> u.save!

Generate SSH keypair in the WebDeploySRV (Our compiled artifact will be deployed in this server inside of docker container) server. This key will be used as SSH key in the Jenkins project ‘AppWebGoat-Deploy’.

$ vagrant.exe status
Current machine states:
JenkMaster                running (virtualbox)
JenkinsSlave1             running (virtualbox)
JenkinsSlave2             running (virtualbox)
GitLab                    running (virtualbox)
WebDeploySRV              running (virtualbox)

$ vagrant.exe ssh WebDeploySRV
ubuntu@ubuntu-xenial:~$ sudo su -
root@ubuntu-xenial:~# ssh-keygen -b 2048 -t rsa -f /root/.ssh/id_rsa -q -N ""
root@ubuntu-xenial:~# cat /root/.ssh/id_rsa.pub > /root/.ssh/authorized_keys
root@ubuntu-xenial:~# ls -l ~/.ssh/
total 12
-rw-r--r-- 1 root root  400 Feb  6 15:01 authorized_keys
-rw------- 1 root root 1675 Feb  6 15:00 id_rsa
-rw-r--r-- 1 root root  400 Feb  6 15:00 id_rsa.pub
root@ubuntu-xenial:~# cat ~/.ssh/authorized_keys
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAuy6QnUhRYK1qYln6PCIgXNZ8z52hMon0khfstg7u+eR94bvH
AbGiriECVHY/p1bseRxRrVcODnwUfbOcVD92ty5ryhth87Xbpigu9kQjQu/LW3DN
XdEIA7lcbaMRhKL/fjNgFOd2oWAcJmYW0R7s1IQVSKNXsogguRiZaDS7C9JvaK7b
gmG5iIxUi/XvYvr5XwXCeFYMHeXoSw27aILHcY6yDzFGIJjvrhyKMKPULsEQA9BI
cPUJSmsSFwnrtWp6PLenXWwIGyD/ugV7LUqAmjkFrvr1sJzn6xb+q+CZq67XxV0f
JyMh4xjRPE7IPAZYSmXfQepVGb7bt7LQ848M5wIDAQABAoIBAQCN4Ma2oNkl6Wwo
rkDREaDsHZuvFrDXiN+7WK0QdhNHkJyl3FoEe6tuRxDoClVHNsi5J/D07XZ3Hi+9
88U7ynrmcCY9pDHDs4WQ1QZ7nbwIgY/neIIBaW6ku/xGEgM2QQJdmv1p+uPdf+b2
aA1FZyZmEtO7IixX43zCL9YfuVBJrWN69RvxhdyT9hxxpecamJNt2NP9sM5y7ukx
M/u8iYBHeYgzW3+k/YCpZTIsU1vCRuru20nLOuI9ghjAPyoVAo/AD+RB4hJ+Fyie
JAwDeoMdmPdy4MNvV+R3gIHpqWzk86cHaF5/knREYUvGT1DzXE/RPZZV9ske7TbX
EGKTvd2BAoGBAPByYTiJVQgHjS8FT6SVVCMDp9ediQSoBfCu1lQemJTrmlPPTQK/
Vhaxy/07rnf1E85iHlhYNmvZle5ynnXBd+wDWbeTlV7YsAJysb2ypCdzUc12eMVs
HlKfeYUAbdz8VxxqQUOqY1LmHtfAGQsUSvFyPyUAJCR7YsIvzg5C2IGnAoGBAMdK
KG8cJVozC+TtmWZA4WLWzVeOA8ttvvatRgoq4RZAVO+irjviXFK42IXRC87283Io
en+T4kj7s7+V2JrgeBbN1Kvu+RGzEVTm6XwK3NFSFs6ImmXnlJL6uyrnHRfmX8Qp
tJD5ImKu4fK50Hn50sWdQ+yf1yEH3umDpq7uXgLBAoGACQdyiSn3DHckIRjk9bw5
zZrMibc7tvYMSw51s/D2L9ltESqerLS8qduBWoAMPCry1HQwyYZ+fKxOCgc51hgu
xakWjGM5X7dpXJ4r+ICl02lsL/f77wXxWY7thEYQ2hSLvOBslProAevsSpPHHPhl
HA37Owt3T/tmKP6ckUKHWM8CgYBjiwAdV6K0pjC3vZJUx+2McByLMi1lDEFJd403
gOmDMsj6FXkQAmX5D8lfNea/DwzUpbFiThzGNas39AIIgjyZr2AeLTSNibMLhHd1
dPuNNoKb0RHF+uF0dPbOXDhk+rY1Blvh2f856X6EvhmLA6Uq/f2ykLhlEEhgvVih
ogClQQKBgHfm4WIDz35g5PZVkhHwU9K03JwDrZyIolRT9S2mF/SwEk1OHDJ7r39z
PVTvADZ0AndYiJ0+QOhPG42PeprbDxMRvF9wGxxofsg6O2qxcJ6Vi+9DSu/iqeyh
RcNROIsBemPcX+C4u4pjwaqKGoQPPbSzqZV8+QO9apFgIbkiWGbe
-----END RSA PRIVATE KEY-----

GitLab server credentials:
GitLab URL: http://192.168.106.230
GitLab Login: root
GotLab Password: R00tm1n123

Nexus server credentials:
Nexus URL: http://192.168.106.100:8081
Nexus Login: admin
Nexus password: admin123

In the Jenkins go to the main page -> New Item -> Select ‘Freesytle project‘ -> Enter name ‘AppWebGoat-Build‘ and press ‘OK‘ button.
appwebgoat-build-project

In the opened page go to the section ‘Source Code Management‘, select ‘Git‘, input git repository URL for WebGoat ‘https://github.com/WebGoat/WebGoat.git‘, change branch to ‘develop‘ and press ‘Apply‘ button:
add-github-repo

In the ‘General‘ tab select checkbox ‘Restrict where this project can be run‘ -> then enter ‘swarm‘ in the ‘Label Expression‘ line and press ‘Apply‘ button
restrict-buildproject-from-slaves

Go at the end to Build section -> Press ‘Add build step‘ -> Select ‘Invoke top-level Maven targets‘ -> Enter ‘clean install‘ in the Goals line and press ‘Apply‘ and ‘Save‘ button:
add-build-step-maven-targets

Then press ‘Build Now‘ button. It will fail and then go back and change maven configuration again.
build-fail-console-output
build-configure-again

Enter ‘clean install -Dmaven.test.skip=true‘ in the Goals line and press ‘Apply‘ and ‘Save‘ button:
maven-compile-options

Then press ‘Build Now‘ button:
press-build-now-button

Output must be as following:
build-console-output

Then press ‘Back to Project‘ -> Press ‘Workspace‘ -> Press ‘webgoat-server‘ -> Press ‘target‘ -> and copy full path of ‘webgoat-server-8.0.0.M3.jar‘ file. It will be: webgoat-server/target/webgoat-server-8.0.0.M3.jar
get-artifact-path

In the Nexus page ‘http://192.168.106.100:8081‘ enter login and password:
login-to-the-nexus

Press settings icon:
nexus-settings-icon

Press ‘Repositories‘ button:
nexus-repositories-button

Press ‘Create repository‘ button to create repository to store our ‘artifacts‘ and docker images. For artifacts we will use ‘raw(hosted)‘ type but, for docker we will use ‘docker(hosted)‘ type.
create-repo-for-artifacts
both-artifact-repo-screenshot

First we create Artifacts and then docker repository. The name of artifacts repo will be ‘jenkins-artifacts‘. Configure other settings like as in screenshot then press ‘Create repository‘ button.
create-jartifact-repo

Configure ‘jenkins-docker‘ repository like as in screenshot:
create-jartifact-docker-repo

Then go to the repositories list and copy URL’s for both of them.

http://192.168.106.100:8081/#admin/repository/repositories
copy-nexus-repo-links

http://192.168.106.100:8081/repository/jenkins-artifacts/
jenkins-artifacts-repo-link

http://192.168.106.100:8081/repository/jenkins-docker/
jenkins-docker-repo-link

Prepare URL for our Jenkins (With login: admin and password: admin123 login to the Nexus and upload file with name ‘webgoat-server-8.0.0.M3.jar‘ from local workspace to the remote ‘jenkins-artifacts‘ repository):

curl -v -u admin:admin123 –upload-file webgoat-server/target/webgoat-server-8.0.0.M3.jar http://192.168.106.100:8081/repository/jenkins-artifacts/owasp/webgoat-server-8/8.0/webgoat-server-8.0.0.M3.jar

Go to the ‘AppWebGoat-Build‘ project and press ‘Configure‘ -> At the end of configure in the ‘Build‘ section just press ‘Execute shell
add-execute-shell-for-nexus

Copy curl syntax which prepared before and paste inside of ‘Execute shell‘ section and press ‘Apply‘ -> ‘Save‘ button:
execute-shell-nexus-curl

Build project again and look at the output:
build-to-nexus-output

Check jar file in the nexus repository. Press ‘Cube‘ icon -> ‘Assets‘ -> click ‘jenkins-artifacts‘ -> then click to jar file to copy jar URL
nexus-cube-icon
nexus-copy-link-location

http://192.168.106.100:8081/repository/jenkins-artifacts/owasp/webgoat-server-8/8.0/webgoat-server-8.0.0.M3.jar

Then go to main menu ‘Jenkins‘ -> ‘New Item‘ -> Select ‘Freestyle-Project‘ -> enter name ‘AppWebGoat-Docker-Container‘ and press ‘OK‘ button.
docke-container-project

In the ‘General‘ tab -> select ‘Restrict where this project can be run‘ -> Label Expression ‘swarm‘ -> ‘Apply
restrict-dockerproject-for-slaves

At the end go to the ‘Build‘ section and select ‘Execute shell
execute-shell-docker-project

In the ‘command‘ input copied URL for jar file with curl syntax like as following and press ‘Apply‘ button.

Copy JAR file from Nexus to the Jenkins workspace

curl -O -v http://192.168.106.100:8081/repository/jenkins-artifacts/owasp/webgoat-server-8/8.0/webgoat-server-8.0.0.M3.jar
copy-artifact-to-worspace

Then in the ‘Build‘ section -> Press ‘Add build step‘ -> press ‘Docker Build and Publish
docker-build-and-publish

Enter name of repo, tag, docker process socket file in the Jenkins slave servers, credentials in for hub.docker.com and press save button.
docker-build-and-publish-credentials

Then go to the GitLab URL (Take password which we set before for username ‘root‘) and create new repository. Press ‘Create a project‘ button.
gitlab-create-project

Enter project name ‘webgoat‘ -> Select ‘Public‘ and press to ‘Create project‘ button.
gitlab-repo-public

Copy the link for our repo but, don’t forget change dns name ‘gitlab.hacklab‘ to the IP address (192.168.106.230)of GitLab server
copy-repo-url

Clone this repository to your local folder:

$ git clone http://192.168.106.230/root/webgoat.git && cd webgoat
Cloning into 'webgoat'...
warning: You appear to have cloned an empty repository.
Checking connectivity... done.
User@DESKTOP-SHSRIMD MINGW64 ~/vagrantfiles/Jenkins-GitLab/webgoat (master)
$ ls -la
total 8
drwxr-xr-x 1 User 197121 0 Feb  6 14:55 ./
drwxr-xr-x 1 User 197121 0 Feb  6 14:55 ../
drwxr-xr-x 1 User 197121 0 Feb  6 14:55 .git/

Then prepare ‘Dockerfile‘ with the following content and add this file to our repo. In this dockerfile, from workspace of our Jenkins we add already copied jar file from nexus to the root folder of our container and then execute this jar file in the container.

$ cat  Dockerfile
FROM openjdk:8
ADD webgoat-server-8.0.0.M3.jar /
ENTRYPOINT java -jar /webgoat-server-8.0.0.M3.jar

$ git add Dockerfile
$ git commit -m "Added Dockerfile for WebGoat"
[master (root-commit) 5055544] Added Dockerfile for WebGoat
The file will have its original line endings in your working directory.
 1 file changed, 5 insertions(+)
 create mode 100644 Dockerfile

$ git push
Username for 'http://192.168.106.230/': root
error: unable to read askpass response from 'C:/Program Files/Git/mingw64/libexe                                                                                            c/git-core/git-gui--askpass'
Password for 'http://root@192.168.106.230/': ENTER_PASSWORD
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 294 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To http://192.168.106.230/root/webgoat.git
 * [new branch]      master -> master

Then in the Jenkins for project ‘AppWebGoat-Docker-Container‘ in the section ‘Source Code Management‘ set this repository ‘http://192.168.106.230/root/webgoat.git‘. We will not use credentials because, it is public repo and branch will be ‘master
add-gitlab-repo-url

Go to ‘Build Now‘ project ‘AppWebGoat-Docker-Container‘:
build-docker-project

Look at the result in the Jenkins and in the https://hub.docker.com

Result in the https://hub.docker.com :
result-hub-docker

Now we need to create new ‘Freestyle project‘ to deploy our application. Go to the Jenkins main menu -> ‘New Item‘ -> Select ‘Freestyle project‘ -> Enter name ‘AppWebGoat-Deploy‘ and press ‘OK‘ button.
create-deploy-project

In the ‘Build‘ section ‘Add build step‘ and then click to ‘Execute shell script on remote host using ssh‘ (Automatically will be selected SSH host ‘WebDeploySRV‘ which we have already added before)
build-execute-shell-on-remote

Add ‘docker run -p 8080:8080 -d jamalshahverdiev/webgoat:latest‘ command in the ‘Command‘ section. This command will be executed in the ‘WebDeploySRV‘ server over SSH. Then press ‘Save‘ button.
select-ssh-host-to-execute

Go back to the ‘AppWebGoat-Deploy‘ Jenkins project and press ‘Build Now‘. Output must be as following.
deploy-output
deploy-output1

In the server ‘WebDeploySRV‘ look at the running containers:

root@ubuntu-xenial:~# docker ps
CONTAINER ID        IMAGE                             COMMAND                  CREATED              STATUS              PORTS                    NAMES
b517fe8332ce        jamalshahverdiev/webgoat:latest   "/bin/sh -c 'java -j…"   About a minute ago   Up About a minute   0.0.0.0:8080->8080/tcp   brave_hypatia

At the end we must open the following page:
http://192.168.106.240:8080/WebGoat/login
web-page-result

Actually it is end of the work. But, if we want to automate all three projects, then we can use the ‘Build Pipeline plugin‘.

Before start configuration just remove container from ‘WebDeploySRV‘ server:

root@ubuntu-xenial:~# docker rm -f brave_hypatia
brave_hypatia

Go to the Jenkins main menu -> Select ‘AppWebGoat-Build‘ -> ‘Configure‘ -> At the end of page in the section ‘Post-build Actions‘ press ‘Add post-build action‘ then press ‘Trigger parameterized build on other projects
trigger-parapetrized-docker

Enter name of the second project which must be triggered after ‘AppWebGoat-Build‘ project. In our case the name of project is ‘AppWebGoat-Docker-Container‘. Select checkbox ‘Trigger build without parameters‘ and ‘Save‘ button.
trigger-docker-container

Then repeat same step in project ‘AppWebGoat-Docker-Container‘ to execute trigger for project ‘AppWebGoat-Deploy‘ (It is next step of work). Go to configuration of ‘AppWebGoat-Docker-Container‘ project and add ‘Post-build Actions‘ to ‘Trigger parameterized build on other projects‘. In the ‘Projects to build‘ section just add ‘AppWebGoat-Deploy‘ and then press ‘Save‘ button.
trigger-param-deploy

Go to the main menu of the Jenkins. Press ‘+‘ button in the right side of ‘All
add-pipelined-deploy

Enter name ‘AppWebGoat-Build-Deploy‘, Select ‘Build Pipeline View‘ and press ‘OK‘ button.
add-pipelined-deploy1

In the opened page select Select initial Job: AppWebGoat-Build‘ and the press ‘Apply‘ and ‘OK‘. Because, it first one and it will initiate second one. Second will initiate third one.
select-build-starter

In the opened page press ‘Run‘ button:
look-pipeline-flow

Progress status (If you want to see the output just, press terminal icon ):
look-pipeline-flow1

Result of the work:
look-pipeline-flow2

Just try to open and recheck WEB server.

Advertisements

2 thoughts on “Jenkins build artifacts, create docker image and deploy over ssh

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s