Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix failing non-cloud functional tests by using built-in caching on the setup-go action #8276

Conversation

brooke-hamilton
Copy link
Member

Description

This PR is to remediate frequent failures of the scheduled functional test (noncloud) workflow. We are having issues with the caching logic. A test failure example is here.

This PR updates the workflow to remove the caching steps from the workflow and replace it with a new feature of the actions/setup-go@v5 action, in which caching of Go dependencies is enabled by setting the cache parameter to true. See the docs for details.

I also tested using the ubuntu-latest test runner, under the assumption that the non-cloud tests will have fewer artifact outputs and not need the larger disk on ubuntu-latest-m.

Changes:

  • .github/workflows/functional-test-noncloud.yaml: Enabled caching for Go dependencies by setting the cache parameter to true in the setup-go action, and removed the manual cache path setup and cleanup steps to streamline the workflow.
  • .github/workflows/functional-test-noncloud.yaml: Updated the operating system from ubuntu-latest-m to ubuntu-latest. Testing for this workflow in a fork did not encounter disk space limits we have seen on other workflows where we tested ubuntu-latest.
     

Type of change

  • This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).

Fixes: #8274

Contributor checklist

Please verify that the PR meets the following requirements, where applicable:

  • An overview of proposed schema changes is included in a linked GitHub issue.
  • A design document PR is created in the design-notes repository, if new APIs are being introduced.
  • If applicable, design document has been reviewed and approved by Radius maintainers/approvers.
  • A PR for the samples repository is created, if existing samples are affected by the changes in this PR.
  • A PR for the documentation repository is created, if the changes in this PR affect the documentation or any user facing updates are made.
  • A PR for the recipes repository is created, if existing recipes are affected by the changes in this PR.

Change to ubuntu-latest

Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@radius-functional-tests
Copy link

radius-functional-tests bot commented Jan 22, 2025

Radius functional test overview

🔍 Go to test action run

Name Value
Repository brooke-hamilton/radius
Commit ref c06141e
Unique ID funcfdd9c2caa2
Image tag pr-funcfdd9c2caa2
Click here to see the list of tools in the current test run
  • gotestsum 1.12.0
  • KinD: v0.20.0
  • Dapr:
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.3.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcfdd9c2caa2
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcfdd9c2caa2
  • dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-funcfdd9c2caa2
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-funcfdd9c2caa2
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcfdd9c2caa2
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

Copy link

codecov bot commented Jan 22, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 59.82%. Comparing base (f81b546) to head (c06141e).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #8276   +/-   ##
=======================================
  Coverage   59.82%   59.82%           
=======================================
  Files         590      590           
  Lines       39520    39520           
=======================================
  Hits        23643    23643           
  Misses      14121    14121           
  Partials     1756     1756           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@lakshmimsft
Copy link
Contributor

there was a recent PR to remediate the cache log issues which updated multiple files and set cache=false. /~https://github.com/radius-project/radius/pull/8147/files.
Do we want to monitor changes to this workflow and if it runs without issues for multiple runs, we update the other workflows as well? thoughts?

@brooke-hamilton
Copy link
Member Author

there was a recent PR to remediate the cache log issues which updated multiple files and set cache=false. /~https://github.com/radius-project/radius/pull/8147/files. Do we want to monitor changes to this workflow and if it runs without issues for multiple runs, we update the other workflows as well? thoughts?

@lakshmimsft Yes I agree: let's test a change on this workflow and then apply it to others if successful. We had five failures of this workflow since yesterday, related to part of the logic that does caching. The new caching logic that is now built into the setup-go action is intended to simplify and remove the need for custom caching setup like we currently have, and this PR is testing the ability to use that logic. I was able to test this workflow successfully on my fork, so I'm hopeful that it will resolve the occasional failures we have been seeing.

@lakshmimsft
Copy link
Contributor

checked logs and reran tests once. we're not seeing the "..Cannot open: File exists" logs. 👍

@brooke-hamilton brooke-hamilton merged commit 16660a5 into radius-project:main Jan 22, 2025
29 checks passed
brooke-hamilton added a commit that referenced this pull request Feb 25, 2025
# Description

This PR sets the `cache: true` option on the `setup-go` actions in the
workflows. Previously we used a common caching technique to cache the go
install and modules folders, but this feature is now built into the
[`setup-go`
action](/~https://github.com/actions/setup-go/blob/main/docs/adrs/0000-caching-dependencies.md).
This change was tested with PR #8276. Following that, this PR updates
the remaining workflows to use the new caching feature.

The default value for the `cache` option is `true` on the `setup-go@v5`
action. This PR explicitly sets the value to `true` on the action where
it is not specified so that the option will be obvious, even though it
will not change the behavior of the action for cases when the option was
not set.

- `build.yaml`: Explicitly set cache option to true, which is the same
as the default value.
- `functional-test-cloud.yaml`: Changed the cache option from false to
true and removed the original caching actions.
- `lint.yaml`: Explicitly set cache option to true, which is the same as
the default value.
- `long-running-azure.yaml`: Explicitly set cache option to true, which
is the same as the default value.
- `publish-docs.yaml`: Explicitly set cache option to true, which is the
same as the default value.

> NOTE: This PR is created from a branch in this repo instead of a fork
from an external repo to allow us to run the cloud test workflows.

## Type of change

- This pull request adds or changes features of Radius and has an
approved issue (issue link required).

Related to issue: #8274
Related to PR #8276

## Contributor checklist
Please verify that the PR meets the following requirements, where
applicable:

- [ ] An overview of proposed schema changes is included in a linked
GitHub issue.
- [ ] A design document PR is created in the [design-notes
repository](/~https://github.com/radius-project/design-notes/), if new
APIs are being introduced.
- [ ] If applicable, design document has been reviewed and approved by
Radius maintainers/approvers.
- [ ] A PR for the [samples
repository](/~https://github.com/radius-project/samples) is created, if
existing samples are affected by the changes in this PR.
- [ ] A PR for the [documentation
repository](/~https://github.com/radius-project/docs) is created, if the
changes in this PR affect the documentation or any user facing updates
are made.
- [ ] A PR for the [recipes
repository](/~https://github.com/radius-project/recipes) is created, if
existing recipes are affected by the changes in this PR.

---------

Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Scheduled functional test (noncloud) failed - Run ID: 12912790930
2 participants