CI/CD With Github

CI/CD (Continuous Integration and Deployment) is tough to get started on, especially if you're a small team constantly dealing with new projects that require re-establishment of the same CI/CD workflows. The basic premise, for those with no background, is how to continuous integrate code into a codebase from a team of developers, and also have structures in place to deploy that code. It's more complicated than that, but that's a pretty good synopsis.

Though there are many tools (GitLab, looking at you) that are incredibly powerful, Github Actions can also run smaller project CI/CD workflows efficiently and without fuss. We do not believe in overcomplicating things if it's not necessary so we use the simple option when possible (hence our name, Minima)

Below, a way oversimplified example of this - obviously in reality we would have different environments etc, but you get the point.

It should be possible to branch from main, test changes locally in the docker-compose setup, then merge the branch into main.

This is a very simplified example, with no Kubernetes or clusters at all. For these purposes there can be one remote instance, running one Docker compose project.

The workflow should be as follows:

1. A dev branches from main and does work
2. Once ready, that dev can submit a pull request to merge the work branch
3. A code owner can review that pull request and approve
4. Once approved, the pull request will merge which will trigger a github action
5. This Github Action will build and push the new Docker images based on the new code

Github Action trigger

This GitHub action will be in the repository, but we don't want to have copy-paste the same code to every new repository we make. So actually, this Github action in the .github folder, will effectively just call the other Github Action.

name: CI/CD Pipeline Trigger

on:
  pull_request:
    types: [closed]
    branches:
      - main

jobs:
  call-reusable-workflow:
    # if: github.event.pull_request.merged == true
    uses: ${your_organization_name}/shared-github-actions/.github/workflows/ci-template.yml@main
    with:
      ecr-repository-prefix: ${your_ECR_repo}
    secrets: inherit

As you can see this Github Action is pretty simple. On a PR merged from main, it will call a Github Action from the shared-github-actions repository and pass it the ECR prefix. ECR is Amazon's container registry and is how this CI/CD workflow will build containers for the project.

GitHub Action Main

This is the Github Action where the real meat is. This action will effectively get the latest version of the code with the merged branch, build all of the docker containers for each service, and then tag them all with a random hash as well as the "LATEST" tag and push them to the container repository.

name: CI Pipeline

on:
  workflow_call:
    inputs:
      ecr-repository-prefix:
        required: false
        type: string

jobs:
  build-and-push:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3.4.0
  
      # see: https://github.com/aws-actions/amazon-ecr-login
      - name: Log in to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1
        
      - name: Fetch All Branches
        run: git fetch --all

      - name: Build Docker images
        id: build-images
        run: |
          docker compose -p ${{ vars.project_name }} build --no-cache
          services=$(docker compose config --services)
          for service in $services; do
              docker tag ${{ vars.project_name }}-${service} ${service}:latest
            fi
          done

      - name: Tag and Push Docker images to Amazon ECR
        run: |
          export $(cat .env | xargs) # Load environment variables from .env
          IMAGE_TAG=${GITHUB_SHA::7}
          services=$(docker compose config --services)
          for service in $services; do
              REPOSITORY_URI=${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ secrets.AWS_REGION }}.amazonaws.com/${{ vars.project_name }}
              IMAGE_NAME=${REPOSITORY_URI}:production-${service}-$IMAGE_TAG
              LATEST_IMAGE_NAME=$IMAGE_NAME-latest

              # Tag images
              docker tag ${{ vars.project_name }}-${service} ${IMAGE_NAME}
              docker tag ${{ vars.project_name }}-${service} ${LATEST_IMAGE_NAME}

              # Push images using Docker CLI
              docker push ${IMAGE_NAME}
              docker push ${LATEST_IMAGE_NAME}
            fi
          done