← Tech

Jenkins

Jenkins

Jenkins-master is running as a Docker container on the tools-centralus-aks kubernetes cluster. The service files are defined here.

Here is the docker file for jenkins-master.

There’s also a script that runs weekly to update the packages on jenkins.

Jenkins-master has a service account for each kubernetes cluster using a websocket connection.

There are two types of Jenkinsfiles: declarative and scripted. We mainly use scripted.

Pipelines

There are two types of pipelines: the app pipeline and configuration pipeline (environments). Jenkins handles both.

Application Pipeline

The Application Pipeline is configured per a Jenkinsfile in the project repository.

Projects have a Jenkinsfile in their Github repo which identifies the pipeline for that particular project. For example the abilities-api project uses the bannoScalaPipeline.

The pipelines available to Jenkins to run are defined in the jenkins-shared-pipelines repo.

ConfigureTools

In the Jenkinsfile there is also a way to reference custom build scripts which can be viewed here

For example, the tool() method is what calls the build script path of the configureTools configuration.

def sbtHome = tool(parameters.sbtLaunchToolName)

Then the reference for that executable is made available in the bannoScalaPipeline.

To reference the configureTools scripts directly, you can use tool(“big”) which corresponds to the custom Tool name

Custom Classes

There are custom Java/groovy classes being imported into the jenkins file for use. For example, the class DeploymentChange is defined here.

Environment Pipeline

The environments repository is the desired state of our infrastructure. Each commit captures the changelog of each version change.

Build Agents

There are three types of build agents: AWS, Azure, and Kubernetes build agents. The K8 agents are generally more lightweight in comparison to the larger AWS/Azure agents. The AWS agents have specific network access that the Azure agents do not, which is why they are still in use.

Aws/Azure Agents

The AWS/Azure agents can be configured on the configureClouds page

The packer template file handles the packages for the AWS, Azure build agents.

Labels are pretty important for the build agents. Make sure agents and builds are set to “only build with label expressesions matching this node” otherwise the agent might be made available in the general agent pool.

Kubernetes Agents

The docker file for kubernetes agents is defined here.

The install script handles package installs for agents including Scala,Java, Nodejs, Npm, Yarn, Golang, Android, anything needed for builds.

Access

Jenkins is public, no vpn required.

There is a Github oauth2 proxy in front of jenkins in addition to jenkins’ native github authentication.

Running Locally

There is an option to test the jenkins cluster/agents locally.

See this video for more info.