GitLab-CI Pipeline (Concepts)

Continuous Integration

Continuous Integration is a practice followed by developers, that involves integrating code changes and commit frequently in a shared repository. Version Control Systems like GitHub, BitBucket can be used for the purpose.

This involves automated builds and tests whenever a code change is committed to the VCS. Earlier developers used to work in isolation, and when it was time to integrate the code, it became very difficult to resolve the bugs and conflicts, which would consequently waste time as well as efforts. Such problems can be avoided by the help of CI. Working on different branches may keep the work isolated for a short span just to make sure no that there is no conflict on merging and the master branch can meet the desired results.

Continuous Integration also involves automated tests on the application so that any bug in the code is caught early in the development process and fixed. These automated tests are run for every build. This doesn’t make the code bug free, but it just takes care that the codes are caught and fixed beforehand, rather than at a later stage when it becomes too complicated to fix them.

Therefore, the practice of CI is being adopted by developers to have faster builds, automating the process of builds and tests to increase efficiency and do it at a faster rate.

Continuous Deployment

Continuous Deployment is a part of the release pipeline, a plan to release software only after it has passed all the tests successfully and is automatically released into the production environment.
It just deploys the feature to the production environment but not running the code until the team decides to release the feature.

Continuous Delivery

Continuous Delivery is similar to continuous deployment, what is different is just the last manual approval of whether or not to release the product into the production. In the delivery stage, developers review the code changes, merge them, and then package them into an artifact. The package waits for approval, in which the package is opened and reviewed again with a system of automated checks. When it passes all the sets of tests, at that time, the code is deployed automatically.

About GitLab

GitLab is an open-source project, having over 2000 contributors, made to allow different teams to work together and make the software development better and faster. It is a single application, or a DevOps platform as a single application, to carry out the complete DevOps life cycle. It makes the lifecycle much shorter. The lifecycle includes general stages such as managing applications, planning, creating, verifying, packaging, configuring, monitoring, and making our application more secure. A brief description about them have been provided below.

Managing the application:

GitLab provides a feature to manage teams which lets them be aware of how the business is being performed. It helps the teams to optimize the software delivery life cycle by making use of metrics.

  • New groups and sub-groups can be made by the owners or the administrators.

Planning:

GitLab provides planning tools to make sure that the teams are working on the right track. It makes use of certain tools to feature planning.

  • Issue tracking- Any issue in the project can be created that would provide a medium to collaborate on a new idea, implementation on new code, tracking status.

There are many more features under planning which can be explored and utilized.

Creating the code:

We can create, edit, view, and manage code, can create branches by GitLab repositories. Teams can follow the workflow without disruption.

  • It provides source code management, enabling collaboration, merging branches, tracking changes, and rolling back in case of failures.

Verifying the code:

GitLab also helps us to verify the code maintaining code quality, and coding standards necessary for production. These include-

  • Continuous Integration to automate the builds, tests, security checks, and verify each commit.

Packaging the code:

GitLab enables to package applications and dependencies together and build artifacts so that they work in the supply chain. The feature provides the following things for package management:

  • Package registry, where the project’s packages and dependencies are stored.

Configuring the application:

Application environments can be configured and managed through GitLab. Secret variables can be used to limit access to only authorized access and processes. Kubernetes integration reduces the effort to define the infrastructure required to run the application.

  • Auto DevOps

Monitoring the application:

GitLab monitors metrics automatically to track how the changes made in the code impacted the production environment. Monitoring provides feedback which can be responded quickly.

  • GitLab displays information for the projects.

Security:

Security issues are left unaddressed at the time of development of any application. GitLab provides built-in templates for security checks.

  • SAST to check the security issues within the system.

Releasing the application:

GitLab automates the release pipeline and the delivery of applications. The applications can be delivered with zero-touch. This makes the process quite short and faster.

  • Deliver changes to production with zero-touch and allows release through the path defined for production.

Defending the application:

GitLab also takes care of the security issues of an application during runtime, such as threat detection, data security, and application infrastructure security.

  • In the Auto DevOps feature provided by GitLab, a Web Application Firewall(WAF) feature is enabled which examines the traffic being sent to the application and detects malicious content before it reaches the application and causes harm.

Why GitLab?

  • GitLab has a shorter life cycle. A shorter feedback life cycle also has some more benefits like:

In a nutshell, GitLab gives the following key advantages:

  • Even minute changes are taken care of, fully tested, secured, and are not ignored, resulting in better code quality.

Also, a very good study of GitLab in comparison with the other tools has been done here.

Understanding GitLab functioning

Continuous integration and continuous deployment service are provided by GitLab. GitLab functions on one configuration file which is the GitLab-ci.yml file. For every commit that is pushed to the GitLab repository, the added GitLab-ci.yml file is triggered which is run by the help of GitLab runners. The YAML file tells the runner what all jobs need to be carried out.

The backbone for any project to involve CI/CD is pipelines.

GitLab CI/CD Pipelines

  • In GitLab CI, pipelines are made up of main stages and jobs.
    * Jobs define the task that has been carried out and
    * the stages define the order in which the jobs are to be executed.

A general pipeline consists of mainly four stages, namely:

  • The build stage mainly carried out the compilation of the source code or the compile job.

Other things about the pipeline, including the types of pipelines, can be referenced from here, as it would deviate the main topic.

This is somewhat how the pipeline we are going to develop in the project looks like.

Pipeline we will be making by the end of this project

Usually, a pipeline is automatically triggered when any changes are made in the files lying in the repository, only if the GitLab’s configuration file, that is, the GitLab-ci.yml is present in the root of the repository.

GitLab keeps the whole metrics of the pipeline, the duration it ran, the number of success, which can be viewed in the Analytics Section on the GitLab project’s page.

To get analytics
Complete pipeline Analytics

This is the pipeline analytics that has been shown below in the picture.

GitLab Runners

The jobs in the pipelines are executed by GitLab runners. GitLab provides some runners by default, which are known as shared runners. Although, we can also configure runners according to our needs, which would eventually result in longer pipeline execution time. Below are mentioned a few features of runners:

  • Installation of runners is quite easy on many platforms like Windows, macOS, Linux.

GitLab configuration file

GitLab uses the GitLab-ci.yml file to configure the things that are required to be done in the project. As soon as a change is committed, this file starts to run the job using runners for that commit.

Creating a GitLab Project

We are going to create a CI/CD pipeline that would build an artifact, perform unit tests, package into war files, perform some more tests, and finally deploys it to the Kubernetes cluster on GCP. We will be learning about every stage in the pipeline. The project is built in Java, using Maven, in which test cases have been written by the JUnit framework. So let’s get started!

Firstly, an account is required on GitLab, which is free to create. It allows GitHub integration as well, so we can log in to GitLab through GitHub also. You can create the account from here.

After creating an account, click on the “New project” button.

Click on the New project button

For this demonstration, we would be cloning a project from GitLab itself. To do that, follow with the steps:

After clicking on the new project option, select the “Import project” tab, and select the “git Repo by URL” option.

After doing this, insert “https://gitlab.com/petermyren/messageboard" in the URL tab.

Insert the provided URL here

And click on the “Create” button. This will clone the repository into your GitLab account.

Click on Create

The project will be cloned in a few tweaks. Everything is set now, and we are ready to set up our CI/CD pipeline.

Creating a sample .gitlab-ci.yml file

Pipelines in GitLab are configured using a YAML file, the “.gitlab-ci.yml” file for each project in the root of the directory. It defines the structure of the pipeline and the order in which the jobs are to be executed. Jobs are executed using GitLab runners.
We can configure our own GitLab runners, although, for this project, we will be using the GitLab shared runners.
A pipeline consists of jobs and stages. Try out first a very basic pipeline, that will give an overview of how things are going to work and how the jobs are actually carried.
Create a new file named “.gitlab-ci.yml”, in the root of the repository and put the following script into it and commit the changes done in the repository.

Create a new file named .gitlab-ci.yml
stages: 
- build
- test
- package
- deploy

build:
stage: build
script:
- echo " build"

test:
stage: test
script: echo " unit tests"

package:
stage: test
script: cat file.py file2.py | gzip > packaged.gz
artifacts:
paths:
- packaged.gz

deploy:
stage: deploy
script: echo "Deployed"
Creating .gitlab-ci.yml
Click on commit

The pipeline is automatically triggered and the GitLab runners start to execute the jobs written in .gitlab-ci.yml file.

Navigate to the CI/CD tab in the sidebar and select pipelines to view the built pipeline.

The sample pipeline

So that’s how you built a sample pipeline. Now let’s proceed to actually configure pipeline for our project.

These were the basic concepts, to get a working pipeline, refer blog.