Introduction

In this post, a DevOps environment is created using Compose. The following picture shows the participating components in this post:

  1. Git Server - At its core, a git server is required for developers to push/merge their code. Then, the git server can call a CI/CD server/agent to execute a pipeline based on the defined event/hook. For git server, in this post GitLab CE is used.

  2. CI/CD Server - Jenkins is one of the most famous CI/CD applications. However, GitLab Runner is more convenient as it is the integrated and default CI/CD tools for GitLab server.

  3. Artifact/Image Repository - After source code build, the output artifact must be shipped to the server. Since Docker is used to run applications on servers, the final package must be deployed as Docker image. So a registry is required in this case to mediate images between servers. Sonatype Nexus 3 is a professional repository management tool, and it supports various development/build tools such as maven, npm, and so on.

  4. Deployment Servers/Cluster - At the end of a CD pipeline, a server or cluster is required for executing the container(s). The CD can be translated into following paradigms:

    • Continuous Delivery - it targets a pre-production environment, called development/test, to evaluate the release by QA users or customers.
    • Continuous Deployment - after QA/customer validation and verification cycles, in another pipeline, the final release can be deployed in production.

DevOps

Installation

The DevOps environment is installed via Docker Compose. So it is the prerequisite, and you can learn about it in Docker to the Point - Part 2. It is also highly recommended to setup all the containers on the Linux box without any graphical desktop and NetworkManager must be disabled. You can use a virtual machine software.

Note

During evaluation of this example, some unexpected network issues are encountered. After some search, due to this [Ref], Docker strongly recommends that NetworkManager must be disabled.

Now, create the directory DevOps containing the following files:

  1. .env - a key-value file for defined variables in other files
  2. init.sh - an initialization script
  3. docker-compose.yml - main Compose file
  4. register-runner.sh - a script to register gitlab-runner container in GitLab server

Then, follow the below steps (commands should be executed in DevOps directory):

  1. chmod +x init.sh && sudo ./init.sh
  2. docker-compose up -d
  3. Config GitLab Server
  4. Register GitLab Runner
  5. Config Nexus

.env

Note: So based on the above .env, for the rest of this document ${GITLAB_DOMAIN} means gitlab.devops.local and ${NEXUS_DOMAIN} means nexus.devops.local.

Note: The above format (double-quoted values) are supported in newer version of docker-compose. For this tutorial, version 1.27.4 is used.

init.sh

docker-compose.yml

register-runner.sh

Applications

Now execute docker-compose ps and you must see all containers state Up (GitLab may take some time to become healthy). The following sections describe each application.

Traefik

The Traefik reverse proxy is setup due to previous article Docker to the Point - Part 2:

  • http://HOST:8080 - Traefik dashboard with basic auth admin:admin

traefik01

traefik02

GitLab

The GitLab server is accessible via http://${GITLAB_DOMAIN}. At first, it asks you the password for user root. Then you can login via user root and the entered password:

  • GitLab references for Docker (GitLab Docker)
  • In docker-compose.yml lines 10-13, a default config is passed to container via GITLAB_OMNIBUS_CONFIG:
    • gitlab_rails['registry_enabled'] = false and registry['enable'] = false - default GitLab registry for Docker is disabled
    • gitlab_rails['backup_keep_time'] = 604800 - for backup, 7-days retention window is set
  • If you want to disable Auto DevOps - Goto Admin Area > Settings > CI/CD
  • Create a group for all of your projects and set CI/CD variables in that group
  • The runner container must be added to GitLab server, and you need the token in registry-runner.sh script.
    • Its token is in Admin Area > Overview > Runners in the following picture:

gitlab-runner-token

GitLab Runner

The GitLab Runner is the agent for CI/CD of GitLab server. You can register multiple runners for a single GitLab server, and in this way the CI/CD processes can be spread over multiple nodes and CI/CD becomes scalable.

  • GitLab Runner reference for registration via Docker (Registering Runners)
  • In the previous image copy the token and execute ./register-runner.sh TOKEN. The gitlab-runner container is registered as a runner as depicted in the following picture:

runner

Sonatype Nexus 3

Sonatype Nexus is one of the greatest open source repository management software. It supports various repository types such as Docker registry, Maven, NPM, and so on. This post only presents its Docker registry feature.

Note: In spite of Docker registry feature in GitLab, Nexus 3 is still the favorite repository management application for supporting other types of repository such as maven, npm, and so on. If you only need a Docker registry, then GitLab may be more convenient.

  • http://NEXUS_DOMAIN - First page of Nexus
  • Login with user admin and the password is in ${VOL_BASE_DIR}/nexus/admin.password temp file.
  • Create a blob store called docker_hub
  • Create a proxy Docker repository for hub.docker.com without setting any HTTP port, and set blob docker_hub
  • Create a blob store called docker_private
  • Create a hosted Docker repository on HTTP port 8083, and set blob docker_private
  • Create a group Docker repository on HTTP port 8082 and add previous ones to this
  • Enable Docker Bearer Token Realm
  • To push an image to the hosted repo
    • docker login -u USER -p PASS ${NEXUS_DOMAIN}:8083
    • docker push ${NEXUS_DOMAIN}:8083/NAME:VER
  • To pull image from the group repo using proxy (original from hub.docker.com)
    • docker pull ${NEXUS_DOMAIN}/IMAGE_ADDRESS_ON_HUB
      • docker pull nginx:1.18 -> docker pull ${NEXUS_DOMAIN}/nginx:1.18
      • docker pull confluentinc/cp-kafka:5.3.1-1 -> docker pull ${NEXUS_DOMAIN}/confluentinc/cp-kafka:5.3.1-1
  • To pull from the group repo
    • docker pull ${NEXUS_DOMAIN}/NAME:VER
    • Default port is enabled in Traefik for Nexus with /v2/ router (docker-compose.yml lines 43-51)

Note: The ${NEXUS_DOMAIN} (for pulling) and ${NEXUS_DOMAIN}:8083(for pushing) must be added as insecure-registries in /etc/docker/daemon.json config file.

Setup Finalization

  • To backup your Gitlab, create a symlink in /etc/cron.daily to the following script:
  • Nexus uses OrientDB for its internal usage and config storage. So it is important to back up its config data. For this matter, it has a built-in task. You must create one. However, this task just creates files without any retention. The following script tries to keep some latest files and remove older ones. So create a symlink in /etc/cron.daily to the following script:

Using various types of repos

In this section, using nexus repos for some platforms and languages are described.

Replace NEXUS_DOMAIN with your configuration in next sections.

Maven

  • In Nexus, by default, a maven-public repo is created as a group repository for Maven.
  • Create/Edit $HOME/.m2/settings.xml based on following XML content:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.1.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd">

  <mirrors>
    <mirror>
      <id>central</id>
      <name>central</name>
      <url>http://NEXUS_DOMAIN/repository/maven-public/</url>
      <mirrorOf>*</mirrorOf>
    </mirror>
  </mirrors>

</settings>

Go

  • Create go-group repo as a group repo for Go
  • Just define the environment variable GOPROXY="http://NEXUS_DOMAIN/repository/go-group"

Docker

  • Edit /etc/docker/daemon.json, and add or update insecure-registries like the following line 
    1
2
3

    {
"insecure-registries" : ["NEXUS_DOMAIN"]
}
  • sudo systemctl restart docker

NPM

  • Create npm-group repo as a group repo for NPM
  • Create or update .npmrc file in $HOME or project root directory with line registry=http://NEXUS_DOMAIN/repository/npm-group/