Written by Dhruv Jain, Open Mainframe Project 2024 Summer Mentee
Like most of the people who heard the term open source for the first time, I was intrigued and confused at the same time. Those mammoth codebases took me to task. Fast forward to the present, I was entrusted to work with the OpenMainframe Project as a part of my LFX Mentorship, a premiere open source accelerator program.
What Is Galasa?
Galasa is part of the Open Mainframe Project, an initiative managed by the Linux Foundation, that focuses on developing open source tools for the mainframe community. Galasa is a testing tool for hybrid cloud applications, it is way different from other testing tools, it enables applications to conduct deep integration tests, can integrate itself with the devops pipelines using API endpoints, enable end to end tests without mocking or stubbing, and the biggest benefit it’s completely open sourced and can be integrated with third party applications.
A few Technicalities
Galasa used to be an internal IBM tool by some other name, until the organization realized that it could be useful to its clients as well. It was then developed into an open source tool with some changes. Galasa-dev organization has about a dozen repositories written in Golang, java, shell built using docker, maven, gradle etc.
When I was first introduced to Galasa, I was told that Galasa components are built on top of each other, this is something that makes it different from various other open source projects. It basically means that components built by one repository are required to build the components of the following repository and so on.
Problem Statement
The problem statement I am trying to solve: Galasa has all its CI/CD pipelines built using Tekton, which is hosted inside IBM’s internal servers, and the built artifacts were hosted/ distributed using an apache server. My job was to move the CI/CD build pipelines to Github actions for the following reasons:
- Since the project is now open sourced not every contributor can see the build logs which are present inside the internal server, with github actions this would be rectified.
- For Tekton builds it took a 2 minute delay to start once a PR/push is merged/commited, unlike ‘github actions’ which starts instantly.
- If we move to github actions we’ll be able to use GitHub’s built in Maven registry and get rid of the extra infrastructure we need to maintain currently to deploy custom Maven registries
The Implementation
In the first week, I was introduced to my mentors in the Galasa team – IBM’s Jade Carino and Mike Cobbett. We first worked on getting the basics right – improvements were made in the project proposal I submitted during the time of application, we let go of a few things and gave priority to others keeping time constraints in mind. It was my first time looking at Tekton builds which were basically yaml files. I also got myself accustomed to ‘gradle’ and ‘maven’ build processes since they were used extensively throughout the project.
We began work on the first workflow which was for a repository written in golang, the only challenge here was that github packages doesn’t provide separate infrastructure to store and host golang executables, our first guess was to wrap these executables into a jar format and then store it in github maven package registry. Soon we realised that it was not a practical decision, we then had to rely on the upload artifacts action which github provides, which stores our built artefacts below the workflow log, a bit improper but worked for us.
All workflows although unique in themselves have some similarities, which are basically building the code, uploading the artefacts, successfully building and uploading a dockerfile.
Next we took up a maven build workflow, for context we build two workflows for each repository one is a PR build which executes once a PR is raised, and a Main build workflow which executes once a PR is merged.
The difference being the artefacts aren’t signed and uploaded in case of a PR build, whereas in the case of main build pipelines the artefacts are duly signed using GPG keys and then uploaded to the github maven registry along with the dockerfile(s).
Everything looked normal until we realized that we weren’t able to sign these artefacts from within the workflows for some reason, it looked like the workflow environment couldn’t import our GPG private keys from github secrets, we changed the way our keys were configured probably a gazillion times, configuring and importing it manually didn’t help either, but keeping the time constraint in mind we moved forward with the gradle build process, while a few other team members were roped in for this issue.
Following the suit for the gradle build process we started with setting up gradle in the workflow, setting up required environment variables, making a few configuration changes in the build.gradle and finally building, signing and uploading the artefacts into the github package maven registry.
The signing error in the maven build workflows was eventually resolved when we manually imported a custom settings.xml file stored as a secret, as well as our GPG keys into the workflow, the components produced by the workflows up until now are to be used for building the following repository workflows in some way or another.
Although there was a lot to learn and more to come from the remaining journey, the key takeaways from my experience:
- Scheduling of tasks and keeping time constraints in mind can get the work done more efficiently.
- How teamwork can help us navigate through various challenges which otherwise may seem insurmountable.
Stay tuned
Mentees will be blogging about their experiences. Stay tuned here and the Open Mainframe Project social channels for updates.
- Follow us on X at @Openmfproject
- Connect with our LinkedIn page
- Subscribe to our Youtube Channel
- Sign up to get our newsletter
- Learn more about the Open Mainframe Project Mentorship Program
