Jenkins build artifacts, create docker image and deploy over ssh

In this article we will configure Jenkins server to build some java application with ‘maven‘ and upload compiled artifact to the ‘Nexus‘ server.  We will create repository in the GitLab to store Dockerfile which, will download ‘openjdk:8‘ from official repository and add downloaded artifact inside of docker image and start jar file process. After preparation of docker image we will use official https://hub.docker.com as registry server to push image. Then we will deploy this docker image in the ‘WebDeploySRV‘ server. At the end we will create group of Pipeline which will do all jobs by one command.

Java application will be used WebGoat:
https://github.com/WebGoat/WebGoat

jfdtopology

Our network topology will be as following:

maven_integration_plugin

Before start to do all stuff we need install needed plugins to the Jenkins. Go to the Jenkins -> Manage Jenkins -> Manage Plugins -> Go to the Available tab and in in the search line type ‘Maven Integration Plugin

docker-build-and-publish-plugin

To not configure ‘docker login‘ for each of Jenkins slave nodes we will use ‘CloudBees Docker Build and Publish plugin‘ Jenkins plugin. For that repeat steps which we did before and install this plugin.

remote-execute-ssh-plugin

To remote execute SSH commands in the deployment server we need install ‘SSH plugin‘. For that repeat steps which we did before and install this plugin.

build-pipelin-plugin

To execute all our ‘Jobs‘ with one button we need to install ‘Build Pipeline Plugin‘. For that repeat steps which we did before and install this plugin.

add-multiple-credentials

Create credentials for docker registry(https://hub.docker.com) server in the Jenkins. Main menu ‘Jenkins‘ -> ‘Credentials‘ -> press ‘global

username-with-password-credential
username-with-password-credential1

Add Credentials‘ -> In the kind select ‘Username with password‘ -> Scope ‘Global‘ -> Enter username, password, ID, Description and press ‘OK‘ button.

sshkey-credential
sshkey-credential1

Add one more credential to SSH authentication for deploy server (This credential will be used in the Jenkins project ‘AppWebGoat-Deploy‘ to deploy app in the ‘WebDeploySRV‘ server. Private SSH key is generated before. Please copy content of private key and paste here). In the ‘Kind‘ select ‘SSH Username with private key‘, username ‘root‘ and ‘Private key -> Enter directly‘ then press ‘OK’ button

sshremotehost-add

Go to the Jenkins main menu -> ‘Manage Jenkins‘ -> ‘Configure System‘ -> Find ‘SSH remote hosts‘ section and press ‘Add‘ button

sshremotehost-add1

Enter IP address of WebDeploySRV server (IP: 192.168.106.240), add port 22,  select credentials ‘root‘ which we have already added for SSH, press ‘Check connection‘ (Result must be: Successful connection) button and ‘Apply‘ then ‘Save

create-docker-repo
create-docker-repo1

In the https://hub.docker.com/ create new repository for our docker image.

copy-selected-reponame

Copy the selected name. This name will be used from our Jenkins to push to the registry server.

From one of slave nodes copy path of docker socket file (This /var/run/docker.sock  path will be used in the Jenkins to connect to the docker process):

<br>
$ vagrant.exe ssh JenkinsSlave1 -c "netstat -na| grep docker.sock"<br>
unix 2 [ACC] STREAM LISTENING 12338 /var/run/docker.sock<br>

Set ‘root’ user password for GitLab server:

<br>
$ vagrant.exe ssh GitLab<br>
Last login: Mon Feb  5 13:30:26 2018 from gateway<br>
[vagrant@localhost ~]$ sudo su -<br>
Last login: Mon Feb  5 13:22:32 -02 2018 on pts/0<br>
[root@localhost ~]# echo 'R00tm1n123' | openssl passwd -1 -stdin<br>
[root@localhost ~]# gitlab-rails console production<br>
Loading production environment (Rails 4.2.10)<br>
irb(main):001:0&gt; u = User.where(id: 1).first<br>
=&gt; #<br>
irb(main):002:0&gt; u.password = 'R00tm1n123'<br>
=&gt; "R00tm1n123"<br>
irb(main):003:0&gt; u.password_confirmation = 'R00tm1n123'<br>
=&gt; "R00tm1n123"<br>
irb(main):004:0&gt; u.save!<br>

Generate SSH keypair in the WebDeploySRV (Our compiled artifact will be deployed in this server inside of docker container) server. This key will be used as SSH key in the Jenkins project ‘AppWebGoat-Deploy’.

<br>
$ vagrant.exe status<br>
Current machine states:<br>
JenkMaster                running (virtualbox)<br>
JenkinsSlave1             running (virtualbox)<br>
JenkinsSlave2             running (virtualbox)<br>
GitLab                    running (virtualbox)<br>
WebDeploySRV              running (virtualbox)</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>$ vagrant.exe ssh WebDeploySRV<br>
ubuntu@ubuntu-xenial:~$ sudo su -<br>
root@ubuntu-xenial:~# ssh-keygen -b 2048 -t rsa -f /root/.ssh/id_rsa -q -N ""<br>
root@ubuntu-xenial:~# cat /root/.ssh/id_rsa.pub &gt; /root/.ssh/authorized_keys<br>
root@ubuntu-xenial:~# ls -l ~/.ssh/<br>
total 12<br>
-rw-r--r-- 1 root root  400 Feb  6 15:01 authorized_keys<br>
-rw------- 1 root root 1675 Feb  6 15:00 id_rsa<br>
-rw-r--r-- 1 root root  400 Feb  6 15:00 id_rsa.pub<br>
root@ubuntu-xenial:~# cat ~/.ssh/authorized_keys<br>
-----BEGIN RSA PRIVATE KEY-----<br>
MIIEowIBAAKCAQEAuy6QnUhRYK1qYln6PCIgXNZ8z52hMon0khfstg7u+eR94bvH<br>
AbGiriECVHY/p1bseRxRrVcODnwUfbOcVD92ty5ryhth87Xbpigu9kQjQu/LW3DN<br>
XdEIA7lcbaMRhKL/fjNgFOd2oWAcJmYW0R7s1IQVSKNXsogguRiZaDS7C9JvaK7b<br>
gmG5iIxUi/XvYvr5XwXCeFYMHeXoSw27aILHcY6yDzFGIJjvrhyKMKPULsEQA9BI<br>
cPUJSmsSFwnrtWp6PLenXWwIGyD/ugV7LUqAmjkFrvr1sJzn6xb+q+CZq67XxV0f<br>
JyMh4xjRPE7IPAZYSmXfQepVGb7bt7LQ848M5wIDAQABAoIBAQCN4Ma2oNkl6Wwo<br>
rkDREaDsHZuvFrDXiN+7WK0QdhNHkJyl3FoEe6tuRxDoClVHNsi5J/D07XZ3Hi+9<br>
88U7ynrmcCY9pDHDs4WQ1QZ7nbwIgY/neIIBaW6ku/xGEgM2QQJdmv1p+uPdf+b2<br>
aA1FZyZmEtO7IixX43zCL9YfuVBJrWN69RvxhdyT9hxxpecamJNt2NP9sM5y7ukx<br>
M/u8iYBHeYgzW3+k/YCpZTIsU1vCRuru20nLOuI9ghjAPyoVAo/AD+RB4hJ+Fyie<br>
JAwDeoMdmPdy4MNvV+R3gIHpqWzk86cHaF5/knREYUvGT1DzXE/RPZZV9ske7TbX<br>
EGKTvd2BAoGBAPByYTiJVQgHjS8FT6SVVCMDp9ediQSoBfCu1lQemJTrmlPPTQK/<br>
Vhaxy/07rnf1E85iHlhYNmvZle5ynnXBd+wDWbeTlV7YsAJysb2ypCdzUc12eMVs<br>
HlKfeYUAbdz8VxxqQUOqY1LmHtfAGQsUSvFyPyUAJCR7YsIvzg5C2IGnAoGBAMdK<br>
KG8cJVozC+TtmWZA4WLWzVeOA8ttvvatRgoq4RZAVO+irjviXFK42IXRC87283Io<br>
en+T4kj7s7+V2JrgeBbN1Kvu+RGzEVTm6XwK3NFSFs6ImmXnlJL6uyrnHRfmX8Qp<br>
tJD5ImKu4fK50Hn50sWdQ+yf1yEH3umDpq7uXgLBAoGACQdyiSn3DHckIRjk9bw5<br>
zZrMibc7tvYMSw51s/D2L9ltESqerLS8qduBWoAMPCry1HQwyYZ+fKxOCgc51hgu<br>
xakWjGM5X7dpXJ4r+ICl02lsL/f77wXxWY7thEYQ2hSLvOBslProAevsSpPHHPhl<br>
HA37Owt3T/tmKP6ckUKHWM8CgYBjiwAdV6K0pjC3vZJUx+2McByLMi1lDEFJd403<br>
gOmDMsj6FXkQAmX5D8lfNea/DwzUpbFiThzGNas39AIIgjyZr2AeLTSNibMLhHd1<br>
dPuNNoKb0RHF+uF0dPbOXDhk+rY1Blvh2f856X6EvhmLA6Uq/f2ykLhlEEhgvVih<br>
ogClQQKBgHfm4WIDz35g5PZVkhHwU9K03JwDrZyIolRT9S2mF/SwEk1OHDJ7r39z<br>
PVTvADZ0AndYiJ0+QOhPG42PeprbDxMRvF9wGxxofsg6O2qxcJ6Vi+9DSu/iqeyh<br>
RcNROIsBemPcX+C4u4pjwaqKGoQPPbSzqZV8+QO9apFgIbkiWGbe<br>
-----END RSA PRIVATE KEY-----<br>

GitLab server credentials:
GitLab URL: http://192.168.106.230
GitLab Login: root
GotLab Password: R00tm1n123

Nexus server credentials:
Nexus URL: http://192.168.106.100:8081
Nexus Login: admin
Nexus password: admin123

appwebgoat-build-project

In the Jenkins go to the main page -> New Item -> Select ‘Freesytle project‘ -> Enter name ‘AppWebGoat-Build‘ and press ‘OK‘ button.

add-github-repo

In the opened page go to the section ‘Source Code Management‘, select ‘Git‘, input git repository URL for WebGoat ‘https://github.com/WebGoat/WebGoat.git‘, change branch to ‘develop‘ and press ‘Apply‘ button:

restrict-buildproject-from-slaves

In the ‘General‘ tab select checkbox ‘Restrict where this project can be run‘ -> then enter ‘swarm‘ in the ‘Label Expression‘ line and press ‘Apply‘ button

add-build-step-maven-targets

Go at the end to Build section -> Press ‘Add build step‘ -> Select ‘Invoke top-level Maven targets‘ -> Enter ‘clean install‘ in the Goals line and press ‘Apply‘ and ‘Save‘ button:

build-fail-console-output
build-configure-again

Then press ‘Build Now‘ button. It will fail and then go back and change maven configuration again.

maven-compile-options

Enter ‘clean install -Dmaven.test.skip=true‘ in the Goals line and press ‘Apply‘ and ‘Save‘ button:

press-build-now-button

Then press ‘Build Now‘ button:

build-console-output

Output must be as following:

get-artifact-path

Then press ‘Back to Project‘ -> Press ‘Workspace‘ -> Press ‘webgoat-server‘ -> Press ‘target‘ -> and copy full path of ‘webgoat-server-8.0.0.M3.jar‘ file. It will be: webgoat-server/target/webgoat-server-8.0.0.M3.jar

login-to-the-nexus

In the Nexus page ‘http://192.168.106.100:8081‘ enter login and password:

nexus-settings-icon

Press settings icon:

nexus-repositories-button

Press ‘Repositories‘ button:

create-repo-for-artifacts
both-artifact-repo-screenshot

Press ‘Create repository‘ button to create repository to store our ‘artifacts‘ and docker images. For artifacts we will use ‘raw(hosted)‘ type but, for docker we will use ‘docker(hosted)‘ type.

create-jartifact-repo

First we create Artifacts and then docker repository. The name of artifacts repo will be ‘jenkins-artifacts‘. Configure other settings like as in screenshot then press ‘Create repository‘ button.

create-jartifact-docker-repo

Configure ‘jenkins-docker‘ repository like as in screenshot:

Then go to the repositories list and copy URL’s for both of them.

copy-nexus-repo-links
jenkins-artifacts-repo-link
jenkins-docker-repo-link

Prepare URL for our Jenkins (With login: admin and password: admin123 login to the Nexus and upload file with name ‘webgoat-server-8.0.0.M3.jar‘ from local workspace to the remote ‘jenkins-artifacts‘ repository):

curl -v -u admin:admin123 –upload-file webgoat-server/target/webgoat-server-8.0.0.M3.jar http://192.168.106.100:8081/repository/jenkins-artifacts/owasp/webgoat-server-8/8.0/webgoat-server-8.0.0.M3.jar

add-execute-shell-for-nexus

Go to the ‘AppWebGoat-Build‘ project and press ‘Configure‘ -> At the end of configure in the ‘Build‘ section just press ‘Execute shell

execute-shell-nexus-curl

Copy curl syntax which prepared before and paste inside of ‘Execute shell‘ section and press ‘Apply‘ -> ‘Save‘ button:

build-to-nexus-output

Build project again and look at the output:

nexus-cube-icon
nexus-copy-link-location

Check jar file in the nexus repository. Press ‘Cube‘ icon -> ‘Assets‘ -> click ‘jenkins-artifacts‘ -> then click to jar file to copy jar URL

docke-container-project

Then go to main menu ‘Jenkins‘ -> ‘New Item‘ -> Select ‘Freestyle-Project‘ -> enter name ‘AppWebGoat-Docker-Container‘ and press ‘OK‘ button.

restrict-dockerproject-for-slaves

In the ‘General‘ tab -> select ‘Restrict where this project can be run‘ -> Label Expression ‘swarm‘ -> ‘Apply

execute-shell-docker-project

At the end go to the ‘Build‘ section and select ‘Execute shell

In the ‘command‘ input copied URL for jar file with curl syntax like as following and press ‘Apply‘ button.

Copy JAR file from Nexus to the Jenkins workspace

copy-artifact-to-worspace

curl -O -v http://192.168.106.100:8081/repository/jenkins-artifacts/owasp/webgoat-server-8/8.0/webgoat-server-8.0.0.M3.jar

docker-build-and-publish

Then in the ‘Build‘ section -> Press ‘Add build step‘ -> press ‘Docker Build and Publish

docker-build-and-publish-credentials

Enter name of repo, tag, docker process socket file in the Jenkins slave servers, credentials in for hub.docker.com and press save button.

gitlab-create-project

Then go to the GitLab URL (Take password which we set before for username ‘root‘) and create new repository. Press ‘Create a project‘ button.

gitlab-repo-public

Enter project name ‘webgoat‘ -> Select ‘Public‘ and press to ‘Create project‘ button.

copy-repo-url

Copy the link for our repo but, don’t forget change dns name ‘gitlab.hacklab‘ to the IP address (192.168.106.230)of GitLab server

Clone this repository to your local folder:

<br>
$ git clone http://192.168.106.230/root/webgoat.git &amp;&amp; cd webgoat<br>
Cloning into 'webgoat'...<br>
warning: You appear to have cloned an empty repository.<br>
Checking connectivity... done.<br>
User@DESKTOP-SHSRIMD MINGW64 ~/vagrantfiles/Jenkins-GitLab/webgoat (master)<br>
$ ls -la<br>
total 8<br>
drwxr-xr-x 1 User 197121 0 Feb&nbsp; 6 14:55 ./<br>
drwxr-xr-x 1 User 197121 0 Feb&nbsp; 6 14:55 ../<br>
drwxr-xr-x 1 User 197121 0 Feb&nbsp; 6 14:55 .git/<br>

Then prepare ‘Dockerfile‘ with the following content and add this file to our repo. In this dockerfile, from workspace of our Jenkins we add already copied jar file from nexus to the root folder of our container and then execute this jar file in the container.

<br>
$ cat  Dockerfile<br>
FROM openjdk:8<br>
ADD webgoat-server-8.0.0.M3.jar /<br>
ENTRYPOINT java -jar /webgoat-server-8.0.0.M3.jar</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>$ git add Dockerfile<br>
$ git commit -m "Added Dockerfile for WebGoat"<br>
[master (root-commit) 5055544] Added Dockerfile for WebGoat<br>
The file will have its original line endings in your working directory.<br>
 1 file changed, 5 insertions(+)<br>
 create mode 100644 Dockerfile</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>$ git push<br>
Username for 'http://192.168.106.230/': root<br>
error: unable to read askpass response from 'C:/Program Files/Git/mingw64/libexe                                                                                            c/git-core/git-gui--askpass'<br>
Password for 'http://root@192.168.106.230/': ENTER_PASSWORD<br>
Counting objects: 3, done.<br>
Delta compression using up to 4 threads.<br>
Compressing objects: 100% (2/2), done.<br>
Writing objects: 100% (3/3), 294 bytes | 0 bytes/s, done.<br>
Total 3 (delta 0), reused 0 (delta 0)<br>
To http://192.168.106.230/root/webgoat.git<br>
 * [new branch]      master -&gt; master<br>

add-gitlab-repo-url

Then in the Jenkins for project ‘AppWebGoat-Docker-Container‘ in the section ‘Source Code Management‘ set this repository ‘http://192.168.106.230/root/webgoat.git‘. We will not use credentials because, it is public repo and branch will be ‘master

build-docker-project

Go to ‘Build Now‘ project ‘AppWebGoat-Docker-Container‘:

Look at the result in the Jenkins and in the https://hub.docker.com

result-hub-docker

Result in the https://hub.docker.com :

create-deploy-project

Now we need to create new ‘Freestyle project‘ to deploy our application. Go to the Jenkins main menu -> ‘New Item‘ -> Select ‘Freestyle project‘ -> Enter name ‘AppWebGoat-Deploy‘ and press ‘OK‘ button.

build-execute-shell-on-remote

In the ‘Build‘ section ‘Add build step‘ and then click to ‘Execute shell script on remote host using ssh‘ (Automatically will be selected SSH host ‘WebDeploySRV‘ which we have already added before)

select-ssh-host-to-execute

Add ‘docker run -p 8080:8080 -d jamalshahverdiev/webgoat:latest‘ command in the ‘Command‘ section. This command will be executed in the ‘WebDeploySRV‘ server over SSH. Then press ‘Save‘ button.

deploy-output
deploy-output1

Go back to the ‘AppWebGoat-Deploy‘ Jenkins project and press ‘Build Now‘. Output must be as following.

In the server ‘WebDeploySRV‘ look at the running containers:

root@ubuntu-xenial:~# docker ps
CONTAINER ID        IMAGE                             COMMAND                  CREATED              STATUS              PORTS                    NAMES
b517fe8332ce        jamalshahverdiev/webgoat:latest   "/bin/sh -c 'java -j…"   About a minute ago   Up About a minute   0.0.0.0:8080->8080/tcp   brave_hypatia
web-page-result

At the end we must open the following page:
http://192.168.106.240:8080/WebGoat/login

Actually it is end of the work. But, if we want to automate all three projects, then we can use the ‘Build Pipeline plugin‘.

Before start configuration just remove container from ‘WebDeploySRV‘ server:

root@ubuntu-xenial:~# docker rm -f brave_hypatia
brave_hypatia
trigger-parapetrized-docker

Go to the Jenkins main menu -> Select ‘AppWebGoat-Build‘ -> ‘Configure‘ -> At the end of page in the section ‘Post-build Actions‘ press ‘Add post-build action‘ then press ‘Trigger parameterized build on other projects

trigger-docker-container

Enter name of the second project which must be triggered after ‘AppWebGoat-Build‘ project. In our case the name of project is ‘AppWebGoat-Docker-Container‘. Select checkbox ‘Trigger build without parameters‘ and ‘Save‘ button.

trigger-param-deploy

Then repeat same step in project ‘AppWebGoat-Docker-Container‘ to execute trigger for project ‘AppWebGoat-Deploy‘ (It is next step of work). Go to configuration of ‘AppWebGoat-Docker-Container‘ project and add ‘Post-build Actions‘ to ‘Trigger parameterized build on other projects‘. In the ‘Projects to build‘ section just add ‘AppWebGoat-Deploy‘ and then press ‘Save‘ button.

add-pipelined-deploy

Go to the main menu of the Jenkins. Press ‘+‘ button in the right side of ‘All

add-pipelined-deploy1

Enter name ‘AppWebGoat-Build-Deploy‘, Select ‘Build Pipeline View‘ and press ‘OK‘ button.

select-build-starter

In the opened page select Select initial Job: AppWebGoat-Build‘ and the press ‘Apply‘ and ‘OK‘. Because, it first one and it will initiate second one. Second will initiate third one.

look-pipeline-flow

In the opened page press ‘Run‘ button:

look-pipeline-flow1

Progress status (If you want to see the output just, press terminal icon ):

look-pipeline-flow2

Result of the work:

Just try to open and recheck WEB server.

Advertisements

3 thoughts on “Jenkins build artifacts, create docker image and deploy over ssh

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s