Ansible and Jenkins – Configuration and Orchestration

A common approach to create an automated pipeline is to combine 2 platforms to orchestrate and then deploy, in an Agile-continuous manner, applications and infrastructure.  Ansible is used to provision environments, infrastructure configurations and app deployment within a CI/CD process using a Jenkins Pipeline. 

Ansible is a powerful agent-less platform used in IT automation and within the CI/CD process to target an environment onto which the application is deployed.  It avoids shell scripting which are difficult to build and maintain. Jenkins on the other hand, is a well-known tool for implementing CI/CD.  Ansible’s has reusable roles and ‘playbooks’ (tasks, configuration details), automated discovery to provision, create or tear down environments.  Jenkins in this use case, would be a process orchestrator, not a shell script executor. 

Common tools used to build out a CI/CD process pipeline using Ansible and Jenkins could include:

• Jenkins to orchestrate the CI/CD pipeline flow
• Ansible to create all infrastructure for this lab and the to provision the application
• Vagrant to create the infrastructure for this lab
• Maven to manage, set and deploy a Java project
• GIT for source code management and control
• SonarSource to assess code and CI/CD quality
• Nexus or JFrog as the repository for artifact binaries

An infrastructure architecture

ALM (Application Lifecycle Management) scenarios can be emulated within a POC environment to prove the CI/CD concept. 

• SonarSource:  1 vCPU with 2028m RAM
• Nexus: 1 vCPU with 1024m RAM
• Application: 1 vCPU 512m RAM
• Jenkins: 1 vCPU with 1024m RAM

A CI/CD POC with the aforementioned components contains the following architectural elements:

Github -  where the project is hosted and where Jenkins will poll for changes to start the pipeline flow.

SonarSource - source code analysis server. If anything goes wrong during the analysis (e.g. not enough unit tests), the flow is interrupted. This step is important to guarantee the source code quality index.

Nexus or JFrog - is the artifact repository. After a successful compilation, unit tests and quality analyses, the binaries are uploaded into it. Later those binaries will be downloaded by Ansible during the application deployment.

Ansible Playbook - which is a YAML file integrated in the application source code, deploys the Spring Boot App on to a CentOS machine.

Jenkins - is our CI/CD process orchestrator. It is responsible to put all the pieces together, resulting in the application successfully deployed in the target machine.

An Ansible Playbook can be used to provision the infrastructure, using roles from the Ansible Galaxy community. Ansible Galaxy a primary source to learn Ansible.  Often the environment for VMs is managed by Vagrant with libvirt.  Details can be accessed in the project Vagrant ALM at Github.

Example Pipeline Flow

Demo environments should do the following pipeline, though real-world prod systems may have slightly more complicated flows depending on the use case, type of apps, existing legacy systems, etc.

Figure 1- The pipeline flow (source: Redhat)

Pipeline Flow:

1. The process starts by pulling the application source code from Github.
2. Project version is updated according to the build number (e.g. your-new-app-1.0.0-123). Version of this build during deployment is now fingerprinted.  Later, can be used as a metric and process control.
3. After updating the project’s version, Jenkins starts building the source code using Maven.
4. After a successful compilation, the pipeline will perform unit tests. If nothing goes wrong during the unit tests, the pipeline flow initiates the integration tests.
5. The test framework creates all the infrastructure needed for the tests: an in-memory database with test data and a small web server to deploy the application.
6. The integration tests are considered a success when all requests were validated against the application deployed in the test environment.
7. The output from unit and integration tests is a coverage report, which will be one of the artifacts used by Sonar server to generate quality metrics. The other one is the application source code. If the quality gate defined in the Sonar server is not met, the pipeline flow will fail.
8. If everything is okay, the flow stops its execution and waits for approval. The Jenkins server provides an interface for someone with the right permissions to manually promote the build.

 

 

9. After approval, the pipeline executes the flow and uploads the compiled artefact to the Nexus or JFrog repo.  Now we have a new app snapshot which can be deployed. 

Ansible now enters the picture.

10. The pipeline flow sets the required parameters such as the compiled artefact URL and the target host to execute the Ansible Playbook afterward. The Playbook is used to automate all target host configuration. In this many environments we can use this to install Java and to prepare the environment to receive the Spring Boot Application as a service. The Playbook is going to install the application as a service and will poll its HTTP port until it receives a valid response to make sure the deployment was successful.

 

11. After the Ansible Playbook execution, the last step could be to send a notification to every stakeholder regarding the results of the new deployment via email or slack.

Infrastructure provisioning

Ansible roles that could automate the actors in a typical design as given in this example including a Sonar and Jenkins server could lead to the following Playbook:

 

Group roles together to provide a desired infrastructure, example Jenkins’ provision:

 

The above defines a host group where Jenkins is deployed and describes the role used to deploy the Jenkins server.  For instance, we may need the EPEL repository for the CentOS to be enabled, the Jenkins installation itself and GIT, Maven and Ansible which are all required for the pipeline. With barely 11 lines of code, we have a Jenkins server up and running prepared to start our CI/CD process.

To get this infrastructure ready, we took the advantage of the integration between Vagrant and Ansible.  With a simple “vagrant up” on the Vagrant file directory, we have the lab environment ready.

Application deployment

The pipeline has been designed to prepare the application binaries, now called “artifact”, and to upload them in Nexus. The artifact can be reached in Nexus by an URL usually called Artifact URL. Ansible is also part of the pipeline and will receive the Artifact URL as the input for deployment. Thus, Ansible will be responsible not only for the deployment but also for the machine provisioning.

Ansible Playbook deploys our application on the target host:

 

This Playbook will only perform three tasks:

1. Install Java based on a pre-defined role from Ansible Galaxy
2. Download the binaries from the Nexus repository based on the input from Jenkins
3. Set up the application as a Spring boot service again using a role from the community

 

The Playbook is in the application repository, which means that the application knows how to deploy itself. By using the Ansible Jenkins plugin, it is possible to call a Playbook from the pipeline by setting the variables required to execute it.

 

To make this happen, the target host must have the Jenkins user and its keys properly configured. This was done while provisioning the lab.  All target hosts are CentOS machines with the Jenkins user and its key already set up.  There’s a Ansible role to configure the user and its keys. This has been done since it is required in order to have Ansible access the host.

Productivity Enhancers

Ansible with Jenkins:

• Prepare the target host with the Jenkins user and its SSH keys. The target host could be a pod on Red Hat OpenShift, a Virtual Machine, bare metal, etc. Doesn’t matter. Ansible needs a way to connect to the host to perform its magic.
• Set the Jenkins user’s private key on the credentials repository. That way you can easily retrieve it on the pipeline code and send it as a parameter to the Ansible plugin.
• Before running the deploy Playbook, consider installing all required roles on the Jenkins server. This could be done by performing a good old shell script run on the requirements file during the pipeline execution: “sh "ansible-galaxy install -vvv -r provision/requirements.yml -p provision/roles/"”.

 

Sometimes we need to deploy the application only on a specific host (for a blue-green deployment, for example). We can do this by using the “-limit” parameter on the Ansible Playbook execution.

 

==END