Run Python OP tests in a single Python process to improve test time. #8286
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
helinwang
force-pushed
the
stash
branch
2 times, most recently
from
991fa02 to
53c1376
Compare
helinwang
changed the title
helinwang
force-pushed
the
stash
branch
2 times, most recently
from
f3b1e59 to
065c9ee
Compare
|
Still testing if this solution works, would love to have some initial feedback: @typhoonzero @reyoung @qingqing01 @dzhwinter @luotao1 |
Currently, our tests run with 2 GPUs, the init time is absurdly long: about 4s for each process. Currently, we run each OP test on different processes. This PR: 1. create cmake function py_test_modules which will generate the Makefile that runs a list of Python unittest module in a single Python process. 1. move all "python unittest compatible" (e.g., used the unittest package, not just a regular python file). from fluid/tests to fluid/tests/unittests. 1. cmake now will run all OP tests in fluid/tests/unittests in a single process, except the time-consuming tests, they are separated into different processes to utilize parallelism. Please make sure to use the unittest package if you put the python test file in fluid/tests/unittests 1. remove all exit(0) from fluid/tests/unittests/*.py, exit(0) is used to disable unittest, we can not do it when running all tests in a single process since it will terminate the process without running the other tests. Instead, the test is disabled in fluid/tests/unittests/CMakeLists.txt. FIXME is added for each disabled item. Please disable the unittest from fluid/tests/unittests/CMakeLists.txt, instead of adding exit(0) to the Python file, for all Python file in fluid/tests/unittests/.
|
|
|
Done by #8362 , some of the commits in this PR is transferred to that PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, our tests run with 2 GPUs, the init time is absurdly long: about 4s for each process.
Currently, we run each OP test on different processes. This PR:
py_test_moduleswhich will generate the Makefile that runs a list of Python unittest module in a single Python process.unittestpackage, not just a regular python file). fromfluid/teststofluid/tests/unittests.fluid/tests/unittestsin a single process, except the time-consuming tests, they are separated into different processes to utilize parallelism. Please make sure to use theunittestpackage if you put the python test file influid/tests/unittestsexit(0)fromfluid/tests/unittests/*.py,exit(0)is used to disable unittest, we can not do it when running all tests in a single process since it will terminate the process without running the other tests. Instead, the test is disabled influid/tests/unittests/CMakeLists.txt. FIXME is added for each disabled item. Please disable the unittest fromfluid/tests/unittests/CMakeLists.txt, instead of addingexit(0)to the Python file, for all Python file influid/tests/unittests/.