Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore/SK-1468 | Add new CI tests with integration with studio #833

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

Wrede
Copy link
Member

@Wrede Wrede commented Mar 3, 2025

Introduces new CI tests with integration with Studio using the new features in fedn cli.

Obs that the test scripts can be used by anyone with an studio account. Good for quick testing of examples. However, the user can NOT have any other projects in studio when running this test. A fix will hopefully come soon.

Obs that all clients will be killed after any EXIT. The clients are running as background processes with id $PID, client outputs are logged into a file test_.log in FEDN_EXAMPLE folder. Suffix is a number [0,FEDN_NR_CLIENTS -1]

Configurations for the test is taken from .ci/tests/studio/.env and can be changed:

"FEDN_NR_ROUNDS": 5
"FEDN_ROUND_TIMEOUT": 180
"FEDN_BUFFER_SIZE": -1
"FEDN_NR_CLIENTS": 2 # this can be scaled but will run as subprocesses from test client (your machine if running there)
"FEDN_CLIENT_TIMEOUT": 60 # Max time to wait for clients "online" status before giving up
"FEDN_FL_ALG": "fedavg" # agg settings will hopefully come in the future
"FEDN_NR_EXPECTED_AGG": 2 # Number of expected aggregated models per combiner, you might for example expect nr_agg < nr_clients if you want to simulate stragglers
"FEDN_SESSION_TIMEOUT":300 # Session timeout in seconds, all rounds must be finished within this time
"FEDN_SESSION_NAME": test
"FEDN_EXAMPLE": mnist-pytorch # the test will do a pushd/popd into /examples/$FEDN_EXAMPLE

"STUDIO_HOST": api.studio.scaleoutsystems.com # host adress to API
"STUDIO_USER": github@scaleoutsystems.com

The only ENV which is not in the .env is "STUDIO_PASSWORD" and needs to be exported by user/workflow:

export STUDIO_PASSWORD=

To run the test it's recommended that there are "precreated" projects before the test. From the root folder of fedn source:
.ci/tests/studio/studio.sh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant