Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compute cardinality for loguniform with precision #635

Merged
merged 4 commits into from
Aug 23, 2021

Conversation

bouthilx
Copy link
Member

[Fixes #633]

Why:

With loguniform the number of possible values is limited if precision is
used. Cardinality computation should account for this otherwise
algorithms may get stuck in suggest(). It happened to a user with a
prior loguniform(1e-4, 1e-2, precision=2). This gives only 181 possible
values.

How:

If real dimension has precision and prior loguniform, then compute
cardinality.

There is a problem with transformed space however. A linearized
dimension for instance would attempt to compute the cardinality with the
linearized bounds. What matters is the smallest cardinality between the
transformed space and the original space. The only case where
cardinality is smaller in transformed space is when real values are
discretized. Therefore, we only compute cardinality of transformed
dimensions if transformation lead to integer, otherwise we use the
cardinality of the original dimension.

Also: Add single executor for debugging

We cannot use python debugger (or pytest.set_trace()) during the
execution of the workers with joblib backend. We should have a simple
executor backend that is not using multithreading or multi-processing to
enable simple debugging. Also, since client's workon() helper function
does not support parallelism, it should use this simple executor.

How:

Use functools.partial to wrap submitted functions for future execution.

Why:

We cannot use python debugger (or pytest.set_trace()) during the
execution of the workers with joblib backend. We should have a simple
executor backend that is not using multithreading or multi-processing to
enable simple debugging. Also, since client's `workon()` helper function
does not support parallelism, it should use this simple executor.

How:

Use functools.partial to wrap submitted functions for future execution.
Why:

With loguniform the number of possible values is limited if precision is
used. Cardinality computation should account for this otherwise
algorithms may get stuck in suggest(). It happened to a user with a
prior loguniform(1e-4, 1e-2, precision=2). This gives only 181 possible
values.

How:

If real dimension has precision and prior loguniform, then compute
cardinality.

There is a problem with transformed space however. A linearized
dimension for instance would attempt to compute the cardinality with the
linearized bounds. What matters is the smallest cardinality between the
transformed space and the original space. The only case where
cardinality is smaller in transformed space is when real values are
discretized. Therefore, we only compute cardinality of transformed
dimensions if transformation lead to integer, otherwise we use the
cardinality of the original dimension.
@bouthilx bouthilx added bug Indicates an unexpected problem or unintended behavior medium The bug breaks a feature but it can still be used or causes a confusing user experience labels Aug 20, 2021
@bouthilx bouthilx added this to the v0.1.16 milestone Aug 20, 2021
@bouthilx bouthilx merged commit ce644dc into Epistimio:develop Aug 23, 2021
@bouthilx bouthilx deleted the hotfix/precision_cardinality branch August 23, 2021 16:03
@bouthilx bouthilx mentioned this pull request Aug 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Indicates an unexpected problem or unintended behavior medium The bug breaks a feature but it can still be used or causes a confusing user experience
Projects
None yet
Development

Successfully merging this pull request may close these issues.

orion hunt not starting a new experiment without changing random seed in orion config
1 participant