GitLab-CI Pipeline

12 min readJun 4, 2020

Configuring .gitlab-ci.yml for project

Now that we have learned about all the basic things (for concepts, refer to previous blog), let’s begin with our actual configuration file. Below is the configuration file, you can replace the contents of the .gitlab-ci.yml file with this file.

image: docker:latest
services:
  - docker:dind


variables:
  DOCKER_DRIVER: overlay
  SPRING_PROFILES_ACTIVE: gitlab-ci
  MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode"
  MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"


cache: 
    paths:
    - .m2/repository/
    - target/


stages:
  - build
  - test
  - codecoverage
  - codeQuality
  - package
  - deploy


include:
   - template: SAST.gitlab-ci.yml
   - template: Code-Quality.gitlab-ci.yml


build:
    image: maven:3.5-jdk-8-alpine
    stage: build 
    script:   
        - mvn compile
        - ls target/generated-sources
    artifacts: 
        paths: 
         - target/*.war
        

unittests:
    image: maven:3.5-jdk-8-alpine
    stage: test 
    script: 
        - mvn $MAVEN_CLI_OPTS test 
    artifacts:
        reports:
            junit:
                - target/surefire-reports/TEST-*.xml


code_quality:
   stage: codeQuality
   artifacts:
     reports:
       codequality: gl-code-quality-report.json
   after_script:
     - cat gl-code-quality-report.json


spotbugs-sast:
  dependencies:
    - build
  script:
    - /analyzer run -compile=false
  variables:
    MAVEN_REPO_PATH: ./.m2/repository
  artifacts:
    reports:
      sast: gl-sast-report.json


codecoverage:
    image: maven:3.5-jdk-8-alpine
    stage: codecoverage
    script:
        - mvn clean verify
    artifacts:
        paths:
            - target/site/jacoco/index.html


docker-build:
  stage: package
  script:
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
    - docker build -t registry.gitlab.com/ishitasinghal/messageboard . --build-arg BUILDKIT_INLINE_CACHE=1 
    - docker push registry.gitlab.com/ishitasinghal/messageboard


k8s-deploy:
 image: google/cloud-sdk:latest
 stage: deploy
 script:
 - echo "$GOOGLE_KEY" > key.json
 - gcloud auth activate-service-account ishita-singhal@my-project-ishita.iam.gserviceaccount.com --key-file key.json
 - gcloud container clusters get-credentials cluster-1-ishita --zone us-central1-c --project my-project-ishita
 - kubectl delete secret $DPRK_SECRET_KEY_NAME 2>/dev/null || echo "secret does not exist"
 - kubectl create secret docker-registry $DPRK_SECRET_KEY_NAME --docker-username="$DPRK_DOCKERHUB_INTEGRATION_USERNAME" --docker-password="$DPRK_DOCKERHUB_INTEGRATION_PASSWORD" --docker-email="$DPRK_DOCKERHUB_INTEGRATION_EMAIL" --docker-server="$DPRK_DOCKERHUB_INTEGRATION_URL"/
 - kubectl apply -f deployment.yml

On running the script, the following pipeline is obtained.

Note: The secret variables used in the file are added in the secret variables in the GitLab CI/CD settings. We will be discussing them in the upcoming stages.

Now, we will be discussing the content written in the file.

Understanding the configuration file

Starting from the first three lines written in the file, it pulls an image to run and execute the commands written in the file.

image: docker:latest
services:
 — docker:dind

The image is a keyword, which pulls a Docker image present in the local Docker engine or Docker hub, that will be used to create a container to build and test our application. In GitLab pipelines, each job is run in a separate container which is isolated.
The services is also a keyword, that takes another docker image, and defines what services are to be included during the build time. It can use any type of service that has to be involved.
For example, including a MYSQL service that runs a database container. Using images makes it easier and faster to use an already existing image instead of installing MySQL at each build time.
In this project, the dind service is used, known as the as docker-in-docker image. This service enables dockers to build within the pipeline.
docker:latest image contains all the necessary things that are required to connect to docker daemon and the docker daemon itself, such as docker build, docker run, but the docker daemon is not started as an entrypoint. Whereas, docker:dind is build on the docker:latest image and starts the docker daemon as an entrypoint.

Note: Using both dockers:dind and dind isn't necessary, only one of them can be used, all we need to do is just start dockerd before docker build and run.

Coming to the next few lines, which uses the variables tag.

variables:
 DOCKER_DRIVER: overlay
 SPRING_PROFILES_ACTIVE: gitlab-ci
 MAVEN_CLI_OPTS: “-s .m2/settings.xml — batch-mode”
 MAVEN_OPTS: “-Dmaven.repo.local=.m2/repository”

Variables, similar to their use with any other language to store a specific value. Similar is their use in GitLab. Variables can be passed to the jobs listed in the file. They can be created either locally for each job or globally. In this pipeline, they have been used globally, once at the top.
The first variable is DOCKER_DRIVER: overlay. It is acts as a storage driver. Since, while executing jobs on shared runners, increases traffic which makes the pipeline execution time more. Therefore, overlay value, creates a distributed network among multiple docker daemon hosts, allowing containers to communicate securely over the network.
The next variable is SPRING_PROFILES_ACTIVE. Since, the project we are using is a Spring Boot Application, therefore we need to activate spring profile to separate parts of application to make them work in certain environments only. For example, if we are running the application on the developer machine, localhost and for GitLab CI, mongo can be used. Different URL's can be used for different environments.
MAVEN_CLI_OPTS This launches a Maven command line option, and the value "-s .m2/settings.xml --batch-mode" is passed to take the contents from maven's local repository .m2 the settings.xml file automatically in a batch mode, that is, this command won't stop to accept input from the user.
Next, the MAVEN_OPTS is used for launching the Java Virtual Machine(JVM) that runs Maven. The value "-Dmaven.repo.local=.m2/repository" is passed as an option to the JVM, to have a maven environment and set the local repository as .m2/repository.

The cache part of the yaml file.

cache: 
 paths:
 — .m2/repository/
 — target/

Cache is basically used to store all the project’s dependencies to save time while running jobs.
Here we are keeping Maven’s .m2 repository in cache, as it contains the necessary files, such as settings.xml and the target folder, which is Maven's default output whenever a build command is run and contains the jar and the war files.
The contents of the m2 folder and target folder is required many times in the folder such as testing, packaging, deployment which is reused many time. So saving it in cache saves fetching time.

Now, the one of the main part of the pipeline comes, stages.

stages:
 — build
 — test
 — codecoverage
 — codeQuality
 — package
 — deploy

In this script, six stages have been defined. These stages determines the lifecycle of our build. In total, 8 jobs are written in all the stages.
Few jobs run in parallel with more than one job depending on one stage.
Stages are run sequentially in the order they are defined.
If any of the stage fails, the pipeline is stopped midway.

Now, let’s understand each stage in detail.

The build stage:

build:
    image: maven:3.5-jdk-8-alpine
    stage: build 
    script:   
        - mvn compile
        - ls target/generated-sources
    artifacts: 
        paths: 
         - target/*.war

In this stage, we are simply pulling a Maven image locally for this job, to execute the mvn (maven) commands, such as mvn compile .
By writing the stage: build tag, it defines that this job belongs to the build stage.
Any command we want to run is written under the script tag. For this job, we have specified to run the mvn compile command, which compiles code residing in the repository, and fails in case of any error.
ls command is just listing the contents of the folder on the console.
All the artifacts generated in the process, such as the war files, are stored in the target folder.
We pulled the maven:3.5-jdk-8-alpine as it is a light-weight image, that includes basic commands only that we want to run.

The test stage:

unittests:
 image: maven:3.5-jdk-8-alpine
 stage: test 
 script: 
 — mvn $MAVEN_CLI_OPTS test 
 artifacts:
 reports:
 junit:
 — target/surefire-reports/TEST-*.xml

After the code has been compiled, the test cases written by the developer, which are run in this stage.
This is known as unit testing.
mvn $MAVEN_CLI_OPTS test command is written to test all the unit test cases that has been written.
For unit testing, JUnit framework has been used in the project.
All the JUnit reports generated are stored as an artifact in the target folder.
The surefire-reports/TEST-*.xml implies that a surefire dependency, which is a testing report dependency that generates an xml report, and can be viewed on the tests tab in the pipeline section.
The surefire dependency has been added to the pom.xml file in the repository.

Coming to the codecoverage stage:

Code coverage includes the number of lines of code covered, or the percentage of code covered. More the test coverage, lesser is the chance of the code containing unidentified bugs left in the code.

codecoverage:
 image: maven:3.5-jdk-8-alpine
 stage: codecoverage
 script:
 — mvn clean verify
 artifacts:
 paths:
 — target/site/jacoco/index.html

In this project, we have used Jacoco as a code coverage tool. To include the Jacoco plugin, put the following plugin inside the POM file.

<plugin>
 <groupId>org.jacoco</groupId>
 <artifactId>jacoco-maven-plugin</artifactId>
 <version>0.8.6-SNAPSHOT</version>
</plugin>

To avoid the generation of redundant reports with the maven-site-plugin the below mentioned plugins are used.

<project>
 <reporting>
 <plugins>
 <plugin>
 <groupId>org.jacoco</groupId>
 <artifactId>jacoco-maven-plugin</artifactId>
 <reportSets>
 <reportSet>
 <reports>
 <! — select non-aggregate reports →
 <report>report</report>
 </reports>
 </reportSet>
 </reportSets>
 </plugin>
 </plugins>
 </reporting>
</project>

Jacoco is a free library for Java for code coverage.
It provides a complete code coverage report in a .html format or which can be stored as an artifact and can be downloaded.

The codeQuality stage:

GitLab provides an in-built template that can simply be used to check the quality of code. To include that template, the following lines are added in the script:

include:
 — template: Code-Quality.gitlab-ci.yml

And thereafter, the following script is added to perform the needful tasks.

code_quality:
 stage: codeQuality
 artifacts:
 reports:
 codequality: gl-code-quality-report.json
 after_script:
 — cat gl-code-quality-report.json

Code Quality makes sure that the project is easy to read, simple and other team members can understand and contribute easily.
GitLab uses free, open-source, code climate engines, which are run in the pipeline.
In pipelines, it runs using a pre-built Docker image built in GitLab’s code quality project.
A template can also be used provided by GitHub, as done in this project.
SonarJava code analyzer can be used to extend climate check, for climate checks in Java Projects.
It is included in the Code quality tab on the pipeline page.

Security checks, the SAST stage:

Security testing is done to spot the security issues, because at the time of building software, less concern is given to the security issues that could be left in the code. This is because of meeting the deadlines faster, focusing on the functionality and working of the code, which could result in unattended security issues in the code.
Similar to the code quality template, GitLab also provides the security template, which needs to be included.

include:
 — template: SAST.gitlab-ci.yml

Since our project is Java based, along with Maven, so the SAST makes use of the SpotBugs along with find-sec-bugs plugin as a scan tool.

spotbugs-sast: 
dependencies: 
— build 
script: 
— /analyzer run -compile=false 
variables: 
MAVEN_REPO_PATH: ./.m2/repository 
artifacts: 
reports: 
sast: gl-sast-report.json

It has dependency on the build stage, as all the artifacts are passed to this stage.
The script - /analyzer run -compile=false is used to verify whether all the artifacts have been included in the working directory.
The path to the maven repository is also provided for the SpotBugs analyzer setting to Maven's local repository.
The reports generated are stored as an artifact which are in the .json format.
The results are viewed in the GitLab pipeline page under the security tab.

The package stage involves the docker-build job.

docker-build: 
stage: package 
script: — docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com 
— docker build -t registry.gitlab.com/ishitasinghal/messageboard . 
— build-arg BUILDKIT_INLINE_CACHE=1 
— docker push registry.gitlab.com/ishitasinghal/messageboard

In this stage, the application is being packaged into a Docker container.
Through the docker commands written in the script, the image are being built, and pushed into the GitLab container registry provided by GitLab.

docker login -u gitlab-ci-token -p $CI_BUILD_TOKENTo login into the GitLab’s container registry, the $CI_BUILD_TOKEN is an enviro nment variable that is generated automatically and enables us to login to the GitLab container registry.
docker build -t registry.gitlab.com/ishitasinghal/messageboard . --build-arg BUILDKIT_INLINE_CACHE=1The above command builds an image. BuildKit is used as a build backend for caching mechanism, which is more faster and reliable.
Finally, the image is pushed into the GitLab container registry. by docker push.

The deploy stage:

This stage consists the Kubernetes deploy job, where the application is deployed to the Google Kubernetes Engine.
For this, we need to create a GCP account, which can be created from here. After going to the GCP dashboard and create a service account, thereafter a cluster under the Kubernetes engine.

And then create a cluster by any name.

After all the prerequisites have been done on GCP, we will write the deployment script in our .gitlab-ci.yml.

k8s-deploy: 
image: google/cloud-sdk:latest 
stage: deploy 
script: — echo “$GOOGLE_KEY” > key.json 
— gcloud auth activate-service-account ishita-singhal@my-project-ishita.iam.gserviceaccount.com — key-file key.json — gcloud container clusters get-credentials cluster-1-ishita — zone us-central1-c — project my-project-ishita — kubectl delete secret $DPRK_SECRET_KEY_NAME 2>/dev/null || echo “secret does not exist” — kubectl create secret docker-registry $DPRK_SECRET_KEY_NAME — docker-username=”$DPRK_DOCKERHUB_INTEGRATION_USERNAME” — docker-password=”$DPRK_DOCKERHUB_INTEGRATION_PASSWORD” — docker-email=”$DPRK_DOCKERHUB_INTEGRATION_EMAIL” — docker-server=”$DPRK_DOCKERHUB_INTEGRATION_URL”/ — kubectl apply -f deployment.yml

google/cloud-sdk:latest is used as it contains all the basic dependencies and components of the google software development kit.
$GOOGLE_KEY is a secret variable along with $DPRK_DOCKERHUB_INTEGRATION_USERNAME, $DPRK_DOCKERHUB_INTEGRATION_PASSWORD, $DPRK_DOCKERHUB_INTEGRATION_EMAIL, $DPRK_DOCKERHUB_INTEGRATION_URL, whose values have been put in GitLab secret variables, as it contains secret details of a user to connect to the services.
For that, Go to project’s Settings>CI/CD, and then expand the Variables tab and add all the secret variables there that includes the user details.

The $GOOGLE_KEY puts the json key value of the service account on GCP.
gcloud auth activate-service-account ishita-singhal@my-project-ishita.iam.gserviceaccount.com --key-file key.json command performs the authentication process of the service account. If the value mismatches, the authentication fails, and it throws an error.
gcloud container clusters get-credentials cluster-1-ishita --zone us-central1-c --project my-project-ishita command we download the kubectl configuration file to run the further script and connects to the cluster made on GCP.
kubectl delete secret $DPRK_SECRET_KEY_NAME 2>/dev/null || echo "secret does not exist" command is required because Kubernetes API doesn't have a replace option for docker-registry, therefore if this statement is not used, it shows an error that a registry already exists.
kubectl create secret docker-registry $DPRK_SECRET_KEY_NAME --docker-username="$DPRK_DOCKERHUB_INTEGRATION_USERNAME" --docker-password="$DPRK_DOCKERHUB_INTEGRATION_PASSWORD" --docker-email="$DPRK_DOCKERHUB_INTEGRATION_EMAIL" --docker-server="$DPRK_DOCKERHUB_INTEGRATION_URL"/
This command authenticates with our private GitLab container registry, and downloads the images pushed in the registry.
kubectl apply -f deployment.yml finally uses the deployment file defined, and deploys the images to the GCP Kubernetes cluster.

The secrets jobs in the pipeline is an analyzer used by the SAST.

This analyzer is for leaked analyzer based on tools like GitLeaks and TruffleHog.
This explores the possible secret details that can be leaked in the source code and the files residing in our repository.
truffleHog is a Python script that finds risks with secrets in Git repository.
GitLeaks is a SAST tool, used to detect hardcoded secrets like tokens and passwords in Git repositories.

GitLab-CI Pipeline

Configuring .gitlab-ci.yml for project

Understanding the configuration file

Links Referred:

Written by Ishita Singhal