Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: bv_decide inequality regression tests #5714

Merged
merged 2 commits into from
Oct 15, 2024

Conversation

alexkeizer
Copy link
Contributor

This takes a few standalone bitvector problems, about inequalties, from LNSym, and adds them as a benchmark to prevent further regressions with bv_decide.

These problems are particularly interesting, because they've previously had a bad interaction with bv_decides normalization pass, see #5664.

@hargoniX
Copy link
Contributor

!bench to get a baseline with ac_nf enabled

@github-actions github-actions bot added the toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN label Oct 14, 2024
@leanprover-community-bot
Copy link
Collaborator

leanprover-community-bot commented Oct 14, 2024

Mathlib CI status (docs):

  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase 3d175ab25fb0f548be3b5d29f376ed8e32f10137 --onto 225e08965d644715e8961cd205ffedf1cf7d24c2. (2024-10-14 21:42:44)
  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase adfbc56f9100257da410e9c36f6f90ca203b6eeb --onto 225e08965d644715e8961cd205ffedf1cf7d24c2. (2024-10-14 22:12:04)
  • ❗ Batteries CI can not be attempted yet, as the nightly-testing-2024-10-15 tag does not exist there yet. We will retry when you push more commits. If you rebase your branch onto nightly-with-mathlib, Batteries CI should run now. (2024-10-15 08:54:25)

@leanprover-bot
Copy link
Collaborator

Here are the benchmark results for commit 59413c1.
There were significant changes against commit 3d175ab:

  Benchmark            Metric       Change
  ==================================================
- import Lean          task-clock     8.3%  (60.2 σ)
- import Lean          wall-clock     8.3%  (71.8 σ)
+ lake build clean     wall-clock    -1.4% (-11.0 σ)
- lake config import   task-clock     8.2%  (11.7 σ)
- lake config import   wall-clock     8.2%  (10.9 σ)
- unionfind            task-clock     6.6%  (16.3 σ)
- unionfind            wall-clock     6.6%  (16.1 σ)

@alexkeizer alexkeizer force-pushed the bvdecide-regressions branch from 59413c1 to 16acc72 Compare October 14, 2024 21:53
@hargoniX
Copy link
Contributor

!bench and now without ac_nf

@alexkeizer
Copy link
Contributor Author

Note that the worst offender has the normalization pass explicitly disabled (because it times out otherwise), so the baseline will be somewhat skewed

@leanprover-bot
Copy link
Collaborator

Here are the benchmark results for commit 16acc72.Found no runs to compare against.

@hargoniX hargoniX force-pushed the bvdecide-regressions branch from 16acc72 to 432d6f8 Compare October 15, 2024 08:34
@hargoniX hargoniX marked this pull request as ready for review October 15, 2024 08:34
@hargoniX hargoniX enabled auto-merge October 15, 2024 08:34
@hargoniX hargoniX added this pull request to the merge queue Oct 15, 2024
Merged via the queue into leanprover:master with commit 94dd1d6 Oct 15, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants