-
Notifications
You must be signed in to change notification settings - Fork 534
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TCK: Receptacle#expectError timeout approach #452
Conversation
Motivation: `defaultTimeoutMillis` is typically used as a "max time to wait until an expected event occurs". For example this value is used as the argument to a `ArrayBlockingQueue#poll(timeout)` [1][2][3] method call and therefore when things work as expected you rarely wait this full duration. However [Receptacle#expectError](/~https://github.com/reactive-streams/reactive-streams-jvm/blob/master/tck/src/main/java/org/reactivestreams/tck/TestEnvironment.java#L1030) does a `Thread.sleep(..)` and then checks if there is something present in the error list (which is a [CopyOnWriteArrayList](/~https://github.com/reactive-streams/reactive-streams-jvm/blob/master/tck/src/main/java/org/reactivestreams/tck/TestEnvironment.java#L45)). The value for the sleep is typically derived from [defaultTimeoutMillis](/~https://github.com/reactive-streams/reactive-streams-jvm/blob/master/tck/src/main/java/org/reactivestreams/tck/TestEnvironment.java#L521-L526) which leads to unconditional delays. This makes it challenging to use `defaultTimeoutMillis` in tests to specify "maximum amount of time to wait until some event occurs". The impacts are if `defaultTimeoutMillis` is set very small you are more likely to see timeouts (especially on over utilized CI servers), but if `defaultTimeoutMillis` the tests take a long time to complete. [1] /~https://github.com/reactive-streams/reactive-streams-jvm/blob/master/tck/src/main/java/org/reactivestreams/tck/TestEnvironment.java#L1019 [2] /~https://github.com/reactive-streams/reactive-streams-jvm/blob/master/tck/src/main/java/org/reactivestreams/tck/TestEnvironment.java#L978 [3] /~https://github.com/reactive-streams/reactive-streams-jvm/blob/master/tck/src/main/java/org/reactivestreams/tck/TestEnvironment.java#L990 Modifications: - TestEnvironment now supports an additional poll timeout parameter to be used when there is no explicit async event to trigger off of. Result: TestEnvironment's defaultTimeoutMillis can be used as "max time until some event occurs" where defaultPollTimeoutMillis provides the polling interval to check for an event if there is no explicit async event to trigger off of. Fixes reactive-streams#451
I have updated the CopyrightWaiver.txt as part of #450 |
Thanks Scott! Seems I'll be able to review this, yay! I'll give it a look now 👍 |
Thanks for opening this PR @Scottmitch! @reactive-streams/contributors Please chime in! |
The new config has to be able to be configured from the env variables, as some people rely on those -- commented in line how to do it. And the param has to be documented as well in the tck/README.md
Along the way there's some typos there we could fix (the above text should be fine to copy verbatim if you like it @Scottmitch :) Sadly we have the readme in two places, since JDK and RS versions, so this would also have to be done in: /~https://github.com/reactive-streams/reactive-streams-jvm/tree/master/tck-flow/README.md |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this looks promising, needs some final polish though to "fit in" into the existing infra 👍
Thanks a lot @Scottmitch!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good now, thanks for the explanations and adding the ENV support @Scottmitch!
LGTM!
thanks for review @ktoso and @viktorklang! Any other comments, or can this be merged? |
Looks good as I said above here. Ping @viktorklang. Sadly I never got merge powers here... ¯_(ツ)_/¯ |
@Scottmitch Thanks for the great contribution! @ktoso Thanks for helping out reviewing, Konrad! ^^ |
Motivation:
defaultTimeoutMillis
is typically used as a "max time to wait until an expected event occurs". For example this value is used as the argument to aArrayBlockingQueue#poll(timeout)
[1][2][3] method call and therefore when things work as expected you rarely wait this full duration. However Receptacle#expectError does aThread.sleep(..)
and then checks if there is something present in the error list (which is a CopyOnWriteArrayList). The value for the sleep is typically derived from defaultTimeoutMillis which leads to unconditional delays. This makes it challenging to usedefaultTimeoutMillis
in tests to specify "maximum amount of time to wait until some event occurs". The impacts are ifdefaultTimeoutMillis
is set very small you are more likely to see timeouts (especially on over utilized CI servers), but ifdefaultTimeoutMillis
the tests take a long time to complete.[1] /~https://github.com/reactive-streams/reactive-streams-jvm/blob/master/tck/src/main/java/org/reactivestreams/tck/TestEnvironment.java#L1019
[2] /~https://github.com/reactive-streams/reactive-streams-jvm/blob/master/tck/src/main/java/org/reactivestreams/tck/TestEnvironment.java#L978
[3] /~https://github.com/reactive-streams/reactive-streams-jvm/blob/master/tck/src/main/java/org/reactivestreams/tck/TestEnvironment.java#L990
Modifications:
when there is no explicit async event to trigger off of.
Result:
TestEnvironment's defaultTimeoutMillis can be used as "max time until some event
occurs" where defaultPollTimeoutMillis provides the polling interval to check
for an event if there is no explicit async event to trigger off of.
Fixes #451