Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

assess scan execution could be parallelized #2294

Open
adpaco-aws opened this issue Mar 10, 2023 · 1 comment
Open

assess scan execution could be parallelized #2294

adpaco-aws opened this issue Mar 10, 2023 · 1 comment
Labels
[C] Feature / Enhancement A new feature request or enhancement to an existing feature.

Comments

@adpaco-aws
Copy link
Contributor

adpaco-aws commented Mar 10, 2023

Goto binary serialization (introduced in #2205) showed significant time/memory improvements. It's not unreasonable to think that these improvements could allow us to parallelize the assess scan feature so its execution on a set of workspaces can be done in parallel.

With this change, we would get better runtimes for assess scan (currently takes >1hr? to run the top-100 crates experiments).

@adpaco-aws adpaco-aws added the [C] Feature / Enhancement A new feature request or enhancement to an existing feature. label Mar 10, 2023
@adpaco-aws adpaco-aws changed the title assess scan execution could be parallelized using --write-goto-binary assess scan execution could be parallelized Mar 10, 2023
@celinval
Copy link
Contributor

You might want to be careful when we are executing the entire assess flow (i.e.: without --only-codegen). I believe assess already tries to run the test harnesses in parallel. So if scan also invokes assess using threads, we should factor in how we compute the number of threads used to run the test harnesses (or maybe disable it).

In the long run, we could leverage tokio or some thread pool to do this work and not worry about the load.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[C] Feature / Enhancement A new feature request or enhancement to an existing feature.
Projects
None yet
Development

No branches or pull requests

2 participants