Migrate from CircleCI to Github Actions

[spirosoik]

Summary

We gradually moving away from CirlceCI to Github Actions as we are consolidating the CIs we use. In parallel we are introducing trivy in the PR level and we block the PR about the number of critical vulnerabilities.

Update: 15/02

• We also bumped operator version from 0.4.0 to 0.6.2
• We bumped MySQL default provisioning from 5.7 to 8.0
• Bumped kind version to 0.17.0

⚠️ Breaking changes

The MySQL version upgrade will affect users of the operator that are using this under their production workloads. They need to specify the MySQL version on the Mattermost spec to 5.7 manually before the upgrade, so when the operator starts reconciling after the update it'll avoid creating a new database or potentially breaking the current one.

Ticket Link

Ticket: https://mattermost.atlassian.net/browse/CLD-4711

Release Note

NONE

[fmartingr]

After a lot of work @mirshahriar and I managed to get the github action working again for the manifest generation. Now the e2e tests are failing because we do not have docker-in-docker in the action. I have seen some actions around that provide kind working in actions, but I'm unsure how strict are we in using external actions in our own workflows and if those need to be checked up by security. @gabrieljackson any input on this?

[fmartingr]

After a lot of work @mirshahriar and I managed to get the github action working again for the manifest generation. Now the e2e tests are failing because we do not have docker-in-docker in the action. I have seen some actions around that provide kind working in actions, but I'm unsure how strict are we in using external actions in our own workflows and if those need to be checked up by security. @gabrieljackson any input on this?

I have created a tentative upgrade under #329

e2e works (kind cluster is created and tests run) but is failing for mysql instances... which is the same that happens to me locally (both on mac and linux) because errors on the init container of the mysql operator.

[gabrieljackson]

Nice work getting this further along. I am glad that my discussion with Mir was indeed on the right track. As for the use of external actions, I am not sure if we are okay with that or not. The move it GitHub actions is being spearheaded by @spirosoik so let's see what he has to say. If external actions are not allowed here then we can just copy what we need to manage it ourselves.

[spirosoik]

@gabrieljackson @mirshahriar which action you want to use? Overall we want the external actions we are going to use to be officially verified by github.

[fmartingr]

@gabrieljackson @mirshahriar which action you want to use? Overall we want the external actions we are going to use to be officially verified by github.

An action to automatically install kind: #329

[spirosoik]

/update-branch

[spirosoik]

/update-branch

[spirosoik]

/update-branch

[toninis]

/update-branch

[fmartingr]

@mirshahriar Can you take a look at b8c1879 ?

[fmartingr]

So I have made some tests and it seems that the mysql-operator doesn't upgrade versions if you already have one cluster set up, so for the users that have a dev environment using the mysql-operator this should be a no-op, cluster will remain in 5.7 (even if the resource is updated with the 8.0 version).

A notice should be added in the release notes either way, just in case.

[spirosoik]

@fmartingr do you need any help here?

[fmartingr]

@fmartingr do you need any help here?

Mysql-operator is giving problems again, and something have changed with github actions logs processing because it reaches a point where the disk is full (form our waiting for deployment to be log lines), which causes the workflow to fail (because the mysql-operator does not launch correctly), but you can only see that if you have the tab open while the workflow is running, if you refresh the e2e step doesn't show any data, nor in the archive download :/

[spirosoik]

What if you enable debug logs?

[spirosoik]

We need to push this a bit as we need to finalise the CirleCI effort. Let me know how we can help unblock this.

[fmartingr]

We need to push this a bit as we need to finalise the CirleCI effort. Let me know how we can help unblock this.

Hey, I'm currently with other e2e issue on another project, so I couldn't look this up. Basically now we have logs but mysql pods doesn't come up, but I didn't add the ssh step to check what's going on on the cluster in github infra. Running E2E test locally works just fine (on the mac).

[fmartingr]

Everything is working except that the pipeline is failing due to the trivy scan result. I'm not sure how to make that step optional since results are being "pushed" to Github. @toninis @spirosoik any ideas here? We could also leave it as it is (so we don't forget to re-enable it) until we fix our dependencies in the operator. What do you think?

[toninis]

Everything is working except that the pipeline is failing due to the trivy scan result. I'm not sure how to make that step optional since results are being "pushed" to Github. @toninis @spirosoik any ideas here? We could also leave it as it is (so we don't forget to re-enable it) until we fix our dependencies in the operator. What do you think?

@fmartingr This is a Code Scanning result . There is an option on the repo to make this not to fail and just be informative .

It should be ok now . @gabrieljackson is pending for review .
You can re-enable this from the settings page once we fix the vulnerabilities

[fmartingr]

Everything is working except that the pipeline is failing due to the trivy scan result. I'm not sure how to make that step optional since results are being "pushed" to Github. @toninis @spirosoik any ideas here? We could also leave it as it is (so we don't forget to re-enable it) until we fix our dependencies in the operator. What do you think?

@fmartingr This is a Code Scanning result . There is an option on the repo to make this not to fail and just be informative .

It should be ok now . @gabrieljackson is pending for review . You can re-enable this from the settings page once we fix the vulnerabilities

Thanks!

[gabrieljackson]

LGTM. We have a plan to make improvements to this along with preparing the operator 2.0 release.