Gitlab CI Automation: Nice Things to Know Before Starting

Ready to build better conversations?

Simple to set up. Easy to use. Powerful integrations.

Get free access

At Aircall, we recently migrated from Bitbucket to Gitlab. At the same time, we also decided to get away from Circle CI to run self-hosted Gitlab CI runners managed by our SRE team. This move resulted in the migration of 6 pipelines and 19 jobs for Gitlab CI.

In this post, I will share some advice that I think is nice to have in mind when automating your CI with Gitlab CI. I will not focus on how to get your Gitlab CI running because there is already a lot of nice articles or tutorials showing how to do it. Let’s get started!

? Plan what the CI will do

Sometimes the most overlooked step during the setup of a CI, but for me the most important one. A well-planned setup will save you time and frustration. Also, keep in mind that you don’t need to always stick to your plan, changes will happen when getting used to the CI and discovering functionalities.

The first thing when doing the planning is to try to lay out on a piece of paper the pipelines and jobs that you will implement in your CI and how they will be triggered. It will give you a clear overview of what needs to be done so you don’t go in one direction and then back off in the middle of it because you forgot to think about an important thing. Also, don’t skip this if you migrate from an existing CI so you can improve your pipelines!

After doing this first draft see if you can regroup some of the jobs together. In my opinion, it’s better to have one job with a variable parameter versus 3 jobs that have this variation hardcoded.

After all of this is done, you should have a clear overview of what needs to be implemented!

? Template your rules

One of the things that started to add a lot of noise during my migration was the list of rules for my jobs. Since a lot of our jobs are reused for different pipelines, rules start to add up and pollute our files. For example, we have a job that runs sonarqube and is used during 5 pipelines.

Now imagine all of your jobs in all your yaml files looking like above. There must be a way to make it look better?

Thanks to the !reference tag we can select keyword configuration from other jobs to add them to our job. What we did was create templates that define the rules and then thanks to !reference we used those templates in our jobs. The previous job now looks like below.

If we check what develop-schedule looks like inside it is as simple as below.

With this method, the boilerplate introduced by the rules is reduced and you can see more easily all the different triggers for a pipeline if you correctly name your template.

? Don’t leak your sensitive data

This is a big one. In every decision you make when working on your CI always ask yourself whether you are leaking any sensitive data, file, or token.

• Variables/Secrets: Try to put them as masked so they are not displayed when printed in CI logs and put them as protected if you only use them in protected branches or tags. Also, avoid hardcoding them in your .yml.

• Files: The best way would be to store them on a protected remote service, such as a S3, behind credentials.

• Artifacts: Your job can output artifacts that are then available for download in the Gitlab UI or with the API. Now let’s say in a job you are retrieving a sensitive file from your S3 and then you make it available for the next job with artifacts. This will make your sensitive file available for download. So always think carefully before passing artifacts.

• Secret rotation: Think about changing your secrets on a regular basis or when someone leaves the company.

• CI files: You should not be able to modify them without approval by your team. Always carefully review changes made on them. If you have any doubt ask other people or even better the security team of your company (if you have one).

? Tokens and Gitlab account

When creating jobs and schedules you’ll be tempted to use your personal account. Instead, we recommend creating a shared account (added in a password manager for example) to do those tasks.

For schedules, it will allow everyone in your team to edit their settings because only the account that created a schedule can change it.
You can see below a snippet of a Fastlane that we run every day to create a Merge Request (MR) when new translations are available (More info about it here).

At first, you’ll be tempted to create this job with your own account and your own Gitlab token like we did. Since on our project we don’t allow MR author to approve their own MR, he was not able to validate these automated jobs. Also, this job would not work until the token was replaced by a new one if the author had left the company.

Creating the schedule and the Gitlab token with a shared account finally allowed us to approve this MR and in the future, this job will not be impacted by employee turnover. Finally, with a shared account you can with just one look differentiate MR created by automation and MR created by your colleagues.

? Push and MR events

When we started working on the CI, we had the habit of triggering jobs with push events. It resulted in jobs being run every time a new commit was pushed on a branch.

The main problem was that before opening an MR, we didn’t want the CI to run with our changes. The previous code was transformed quite easily into the following one to allow the CI to run on a new commit only when an MR is opened.

Be careful to use the correct variable when adding a rule for a push event or an MR event. Running against MR also solved the issue we had where Sonarqube quality comment was not appearing on our MRs. At Aircall we have our sanity pipelines that run on push events and our working/shipping pipelines that run on MR events.

Quick tips: An easy way to see if a pipeline is running against an MR event is to check on the Gitlab UI if it is labeled as detached.

a detached pipeline

Thanks for reading my first Article! I hope that everything explained in it will help you with your migration to Gitlab CI. A special thank you to Olivier Buiron for his assistance during the work on the CI. A big shoutout to Julien Salvi and Jortscaroff for reviewing the article!

Aircall is looking for new talents, if you are interested you can check our list of current open positions.

Published on March 23, 2022.