Abstract

DevSecOps on AWS can use many patterns, based on key principles.  This flexibility can create confusion with clients.  A common question is something like: ‘what is the best way to enable a CI-CD, DevSecOps pipeline, that we can understand and manage’?  A problem with AWS is its very flexibility leads to complexity, cost issues and governance problems.

This whitepaper outlines the key features and benefits of building a continuous integration, continuous delivery (CI/CD) pipeline as part of the DevSecOps process.  This whitepaper assumes that the firm in question is using Agile-Scrum properly and has already enable its Agile teams with proper engineering processes, tools, standards and most likely, the use of a Centre of Excellence to enforce compliance and monitor metrics.

 

Figure: AWS Pipeline using Kubernetes

Magic DevSecOps and Software Delivery on AWS

DevSecOps is the combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver applications and services at high velocity, securely, minimizing vulnerabilities, and increasing quality.

Using DevSecOps principles, organizations can develop and improve products at a faster pace than organizations that use traditional software development and infrastructure management processes. This speed enables organizations to better serve their customers and compete more effectively in the market. 

CI/CD is platform specific (and tied to the operating system).  It is the key to delivering application and digital software features rapidly and reliably.

AWS now offers a full range of DevOps and CI/CD capabilities as a set of developer services.  The table below is an overview of the many existing CI-CD services and related offerings in AWS.

 

CI-CD related

Overview

AWS CodeCommit

Managed Git-based source code repository and version control repository, for binaries, code, and documents.

AWS CodePipeline

Fully managed service which automates the build, test, and release processes within your code pipeline

AWS CodeBuild

Fully managed Continuous Integration service which automates the compiling, building, testing, and releasing of code into a delivery pipeline

AWS CodeDeploy

Automates code deployments to any instance (EC2 or on-premise), across all environments (dev, test, prod)

AWS Artifact

Fully managed artifact service (binary repository), to store, publish, share software packages and dependencies, can integrate with common package managers such as Maven

AWS CodeStar

Managed service which integrates the Continuous Integration with Continuous Delivery by providing a unified template based pipeline using existing services such as Code Commit, CodeBuild, CodePipeline, Cloud9

AWS Xray

Used mainly with micro-services, deploys a daemon on the application which analyses how the application is used (data, services) and provides insights into improvements

Cloud Formation

Infrastructure as Code (IaC) using JSON or YAML.  Use them.  AWS has many templates on github to help a client get started

AWS Lambda

Managed serverless code deployment, can be used to run functions, in event-driven architectures, deploy applications, alerts and other state-change cycles

AWS System Manager

AWS Linux OS only – this is the major drawback.  Any other Linux OS and you will need to develop a security-patch-update runbook and model (a detailed example can be provided)

AWS Xray

Used mainly with micro-services, deploys a daemon on the application which analyses how the application is used (data, services) and provides insights into improvements

AWS AppConfig

Pay for use service, allows the user to validate an application’s configuration data against a JSON/YAML schema or Lambda function to ensure syntax and semantic correctness (part of System Manager)

AWS Config

Service which continuously monitors and audits the deployed assets against the configuration schema and provides alerts (SNS against state changes) and recommendations

AWS Cloud 9

Browser based IDE to build, run, debug, test code in lieu of using a local IDE client

AWS Amplify

Rapidly deploy React JS or Angular JS code and applications with a backend.  This complete stack significantly decreases deployment complexity, allows for IaC, and automates connecting the front-end of the application (UI-Presentation) with the Backend via CFT (Cloud Formation Templates)

ElasticBeanstalk

CFT based service which provides an end-to-end-pipeline for the deployment of Web Sites and applications built in Java, Ruby, Node.js, Python, PHP, Docker and Go

AWS LightSail

Managed service which allows simpler web, application deployments and provides the underlying infrastructure

Cloud Watch, Cloud Trail, VPC logs

Standard services, can be customized, metrics provided on the application and related infra, along with API, network traffic

 

Magic DevSecOps Caveats and the Real Word

Figure: What is DevSecOps

DevSecOps is tightly integrated with Agile Teams and Engineering processes, and a defined Software Development Life Cycle (SDLC) process. Quite often neither of these concepts are well understood within firms.  Agile-DevSecOps entails cross-functional teams (Dev, Operations, Security, Testing, Business) and drives cultural, organizational, tooling, financial budgeting, and business development changes.

Figure: DevSecOps value stream

A screenshot of a cell phoneDescription automatically generated

(See Docker on AWS: Running Containers in the Cloud)