Written By Dhruv
Jain, student at Maharaja Agrasen Institute of Technology, Delhi
Modern software development is incomplete without efficient Continuous Integration/Continuous Delivery and Continuous Deployment practices. CI/CD systems not only ensure an error free, reliable, rapid delivery of code, but also helps in maintaining code quality and enables collaboration in open source communities.
As a part of LFX Mentorship and Open Mainframe Project, I got a chance to work on the CI/CD build pipelines for the Galasa Project. I have written in a bit more detail about the Galasa project in my midterm blog here.
In this article, we’ll dive deeper into the process of migrating Galasa’s CI/CD pipelines from Tekton to Github Actions, the motivations behind it and the challenges faced during the implementation.
Why are we doing this ?!
Let’s get to the crux of the problem, historically Galasa’s CI/CD pipelines are hosted using Tekton which is a powerful CI/CD tool native to kubernetes and a preferred choice in various cloud native applications. These build pipelines are hosted in an internal IBM cluster, which gives rise to a major problem.
Galasa being an open source project is witness to a number of contributions from the open source community members, most of these developers do not have access to the Tekton build logs within the internal cluster generated whenever they raise a PR or make any code changes. Github Actions not only resolves the aforementioned problem but acts as a one stop solution for a number of issues,
such as:-
- For Tekton builds it takes a 2 minute delay to start once a PR/push is merged/commited, unlike ‘github actions’ which starts instantly.
- If we move to github actions we’ll be able to use GitHub’s built in Maven registry and Github Docker registry, helping us get rid of the extra infrastructure we need to maintain currently to deploy custom Maven registries and docker images.
- Not only are the build logs publicly accessible but we also realise that github actions are much faster in comparison to Tekton, when used to its full potential.
The workflow Design.
Now comes my favorite part,`The workflow design`.This is quite unique to Galasa, hard to find something like this at least in Open Source projects, the Galasa project is distributed in more than a dozen repositories written in java, Go, shell and built using maven/gradle, docker etc.
Each of these repositories have unique workflows to itself, these workflows build specific components for that particular repository and uploads these as artefacts in the respective maven registry and/or in the docker registry, the fascinating part is that these components are built on top of each other i.e. artefacts from one repository are required for building the components for the next repository and so on.
Below is the flowchart I have taken from the Galasa documentation, describing the flow of workflow build components.
The diagram above shows the links between the core component builds for Galasa, starting from wrapping repository builds. These pipelines are either triggered by the previous build pipelines or they may not be connected to the chain and draw their resources from other builds when re-built. Certain pipelines may be triggered only in case of a specific event such as a ‘Release’ etc.
The Implementation
Each of these workflows would be in a set of two, one is invoked when a PR is raised in a repository and one when the PR is merged. Normally a build workflow would begin by checking out the Repository, setting up the required build process (depending upon the project), which can range from Golang, to Maven or Gradle, and often both Maven and Gradle, this is followed by building up either the whole project or parts of the project using the specified pom/ gradle.build files, signing them using the GPG keys, finally uploading the generated artefacts to the maven repository, additionally the generated artefacts may be used in building the dockerfiles, which is eventually uploaded to the docker registry (the artefacts are signed and uploaded only in case of a main build and not the pr builds).
Each of these repositories have a nuance to its workflows, a few workflows may import code from multiple other repositories to build its executables, some of them may have nested build files to generate there components, a few may make use of programs like kubectl and swagger etc. As a final task we added workflow triggers at the end of each workflow depending upon the event which triggered that particular workflow.
Devil is in the Details!
The process was not as smooth as may it seem, we often ran into roadblocks, when we started with the build process for Golang executables, we realized that github doesn’t provide separate infra to host go executables, we first decided to use maven registry for the same but then had to settle with the generic upload artefact action provided by github actions. 
A major problem occurred when we weren’t able to import our GPG secrets into the workflows for signing maven artefacts, after a lot a contemplation we wrote custom linux commands to achieve the same, later this led to the creation of a custom github action (which is being tested while I write this).
Navigating through github permissions was also a task to reckon with, it was often a challenge to retrieve artefacts within a workflow from another repository, or trigger the Tekton ecosystem from within a github action workflow etc.
What Did We Achieve
After 3 months of hard work and continuous effort of my mentors Jade Carino and Mike Cobett, we were able to create the complete build process for Galasa using Github Actions. I was able to create/contribute to about 28+ workflows along with a local custom Github Action. Our primary goal of easing open source development was thereby achieved.
Apart from this, to our surprise Github Actions workflows were astronomically faster than Tekton builds. A few of these comparisons are mentioned below :-
| Workflows | Tekton Github Actions |
| Wrapping | 3 min 48 sec 1 min 43 sec |
| Gradle | 3 min 23 sec 1 min 23 sec |
| Maven | 2 min 57 sec 1 min 5 sec |
| Framework | 10 min 27 sec 3 min 38 sec |
| Extensions | 7 min 26 sec 2 min 38 sec |
| Managers | 14 min 49 sec 6 min 42 sec |
| OBR | 14 min 35 sec 6 min 37 sec |
| Total | 57 min 25 sec 23 min 45 sec |
According to the above table the workflows are at least 2.4 times faster after shifting to Github Actions.
Not only this, in the future Galasa is set to benefit from the suite of features provided by Github in the form of Github maven registry and Github docker registry, allowing us to do away with extra infrastructure we needed to maintain.
Lastly I would like to express my gratitude to my mentors Jade Carino, Software Engineer, and Mike Cobbett, Senior Software Engineer, at IBM for their hard work. Their detailed instructions, valuable feedback and weekly check-ins made my job so much easier, also thanks to the Open Mainframe and LFX team, especially Tom Slanda, for answering my infinite questions and tolerating my delays (including this one).
Stay tuned here and the Open Mainframe Project social channels for more mentee blogs and videos.
- Follow us on X at @Openmfproject
- Connect with our LinkedIn page
- Subscribe to our Youtube Channel
- Sign up to get our newsletter
