smoke tests - automatically test against stable build #127799
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR attempts to include the smoke tests that are usually only running when you provide
--stable-build
in our CI to fully automate smoke tests and not having to run them in endgame.By using our existing
vscode-test
module it is relatively easy to download latest stable bits.However, when I run the following command for the first time in a fresh checkout:
I am greeted with this error:
When I run again, it works fine.
vscode-test
is storing the downloaded bits into a.vscode-test
folder intest/smoke
and will not download it again if it is found there.I tried all things like adding a timeout to ensure the build is there but couldn't figure it out. Need your help @joaomoreno if you maybe got further ideas what this could be.