Feature/filter duplicate in evc #630
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
During execution of the experiment the producer verifies that suggested trials do not already exist in parent or children, but race conditions can lead to duplicates. Also, in attempt to solve Epistimio#576, we will need to duplicate trials that are not completed in parents into executed experiments to allow reserving and executing the trials. This will lead to more potential duplicated trials and raise the important of handling duplicate properly. When fetching from the EVC, we should ignore duplicates from parent or children if the trials are available in current experiment. This will recursively solve the issue during recursive fetch from EVC. This will also simplify the handling of potential duplicates during {naive-}algorithm updates, as there will simply be no duplicates. How: During the call to adaptors, a set of hash is generated from trials of current nodes based on hyperparameter values (ignores experiment id). Any trials from the parents or child that has a hash found in this set of hash will be filtered out. When there is a duplicate, only the trial of the current node is kept. This also applies recursively to call from children experiments to grand-children.
Why: When a dimension is deleted or added, the adaptor should not transform them with a default value of None if there was no default values. This would lead to invalid trials if None is an invalid value of this dimension. How: If the default value is the unique NO_DEFAULT_VALUE object, then the trials should be filtered out.
bouthilx
added
the
enhancement
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[Fixes #628]
Why:
During execution of the experiment the producer verifies that suggested
trials do not already exist in parent or children, but race conditions
can lead to duplicates. Also, in attempt to solve #576, we will need to
duplicate trials that are not completed in parents into executed
experiments to allow reserving and executing the trials. This will lead
to more potential duplicated trials and raise the important of handling
duplicate properly.
When fetching from the EVC, we should ignore duplicates from parent or
children if the trials are available in current experiment. This will
recursively solve the issue during recursive fetch from EVC. This will
also simplify the handling of potential duplicates during
{naive-}algorithm updates, as there will simply be no duplicates.
How:
During the call to adaptors, a set of hash is generated from trials of
current nodes based on hyperparameter values (ignores experiment id).
Any trials from the parents or child that has a hash found in this set
of hash will be filtered out. When there is a duplicate, only the trial
of the current node is kept. This also applies recursively to call from
children experiments to grand-children.