This article is about my experience of switching from a self-host Jenkins server to using GitHub Actions for CI of my colony.core repository. The colony.core repo is a C++ class library targeted for embedded development on microcontrollers. With respect to CI – the library’s unit tests are built using 4 different compilers: GCC on Linux, MinGW on Windows, Visual Studio on Windows, and the GCC ARM Cortex R/M cross compiler on Windows.

Spoiler Alert: I have successfully switched over to using Github Actions and have disabled my colony.core projects on my Jenkins server. Here is link to my the workflow file for the colony.core repo.

I chose Jenkins because it was a solution that I had experience with and that I could easily self-host on premises (aka, my home office with one of my many ‘older’ PCs). And because the solution was self-hosted – it was straight forward to install all of the necessary compilers and other build tools. My Jenkins solution wasn’t perfect, but it got the job done.

In my professional day job, my team uses GitHub Actions with self-hosted runners for our CI, but I was not involved in the implementation details. However, the pain points I have with my personal Jenkins implementation – I do not have at my day job. So last week I decided to take a look at GitHub Actions to see how ‘painful’ it would be to set up GitHub’s host runners (i.e. not use self-hosted runners) with all of the compilers and other assorted tools I need to build colony.core. After about 10 minutes of googling – I discovered that yes – all of the ‘build toys’ I needed either had examples or off-the-shelf GitHub Actions that I could use.

Highlights, takeaways, lessons learned, etc.

Don’t re-invent the wheel

When building your workflows and/or setting up your host runner – always check (aka search) if there is GitHub Action available – don’t assume that you have to provide your on solution. That said, I was not able to find an off-the-shelf solution for installing the MinGW Compiler on Windows host that can build 32bit binaries. This is on my TODO list to resolve 😉.

Performance is not free

The GitHub free host runners are slow, but you essentially have an unlimited number available – so for a single workflow you can run multiple jobs in parallel, and you can have multiple PRs building simultaneously.

Newlines

I had issues with newline translation when the workflow checked out the source code from GitHub repository. The solution to this was to put an explicit step in the workflow job to issue git commands to set the newline translation rules.

  - name: Checkout code
      run: |
        git config --global core.autocrlf false
        git config --global core.safecrlf false
    - uses: actions/checkout@v2

Code Coverage

Generating the code coverage report was challenging. With Jenkins, its Cobertura plug-in made creating the code coverage report simple, especially with respect to consolidating all of the individual unit test coverage report files into a single report file. I did not find any direct equivalent to the Jenkin’s Cobertura plugin, – so I ended up using ReportGenerator.

Using ReportGenerator required ‘me’ generate a single consolidated code coverage report. There were several options – but I settled on using gcovr native ability to consolidate JSON reports. This led to updates to the tca.py utility script that repo’s build scripts use when generating code coverage reports.

Branch Rules

I had to setup Branch Rules (Require a pull request before merging, Require approvals, and Require status checks to pass before merging) in order to trigger my workflow for PRs.

I was surprised that I had to enable Require approvals – but it wouldn’t work without it checked. When you are a team of one – this is silly – but GitHub allows you to bypass the rule on the PR page if you are an admin user.

Host Environment Differences

When using the Windows host runner, I had issues with running python scripts.  Ultimately, I had to update my build scripts so that all python scripts are launched using via the python executable, e.g. python mybuildscript.py (instead of  mybuildscript.py)

Failures on the Host Runner

Troubleshooting failures on the free host runners is painful.  I found the following items to help with troubleshooting workflows:

• There is a GitHub Action – tmate – that will pause your workflow and allow you to SSH into the host runner
•  You can turn on additional ‘debug’ information in the workflow logs by setting the `ACTIONS_RUNNER_DEBUG` and `ACTIONS_STEP_DEBUG` variables to true.  These variables are set via your repo’s *Settings->Secrets and Variables* option

GitHub Action Pros

• No “CI tool” setup or maintenances (vs. installing and maintaining a Jenkins instance)
• Relatively low barrier to entry assuming you have some exposure to basic CI concepts
  • Or said another way, you don’t have to be an expert/experienced DevOps engineer to get the basics up and running.
• Workflows use YAML syntax which is easy to learn
• For open source projects, GitHub provides hosted virtual machines to run workflows
  • GitHub Actions also support self-hosted machines as well
• Seamless integration with GitHub (as opposed to using a tool like Jenkins)
• Users can create their own GitHub Actions
  • There is a large community of shared GitHub Actions (e.g. a quick search turned up GitHub Actions to install Doxygen and Graphviz on the host runner)
• Newer technology/paradigm than “older” tools like Jenkins
  • For example, “out-of-the-box” solutions for handling repository permissions, security tokens, etc.

GitHub Action Cons

• The free host runners are slow. The free host virtual machines are about 4-5x slower than my mini-PC (i7-13700H) that hosted my Jenkins server
  • To mitigate the performance different – it was very simple to run each “compiler” on its own runner in parallel – so my overall build times only went from ~30min to ~40min.
    • Note: I had setup up my Jenkins jobs to run sequentially (because it was easy) and I didn’t have multiple host instances available.
• No direct access to the GitHub host runners. This makes it difficult to troubleshoot failures (e.g. operating system environment differences) when creating workflows.

Summary

Updating the build scripts and getting the workflow file right took some time – but the majority of that time was busy-waiting on the actual build times (did I mention that the free host runners are slow 😉). All things consider the transition to use GitHub Actions was straight forward and I probably should have made the switch sooner.