Implemented a priority executor for bevy_task

[IGreyGooI]

A Priority Executor is implemented on top of the async_executor::Executor, bevy_task is modified to use PriorityExecutor instead. None of TaskPool API is modified(but should expect an API change). IoTaskPool and AsyncComputeTaskPool across all crates are replaced by ComputeTaskPool for testing purpose.

Summery of behavior of PriorityExecutor:

1. PriorityExecutor.run() has two states:
   a. Running tasks with Priority::FinishWithinFrame task
   b. Running tasks with Priority::AcrossFrame and Priority::IO tasks
2. State a will transition to b automatically after all dispatched Priority::FinishWithinFrame tasks are finished.
3. State b will transition to a after a event is received and the worker thread is not occupied by task.

This is meant to be a solution of #1907, However, more tests are needed to ensure:

1. Priority Executor execute all of tasks with the highest priority first before starting any lower priority task.
2. Priority Executor exits runtime properly.

Passed:

1. cargo run -p ci
2. cargo test --all-targets --workspace

[NathanSWard]

It would be nice to see benchmarks on the previous implementation vs this new PriorityExecutor.

[NathanSWard]

Also would love to see some PriorityExecutor specific tests in a #[cfg(test)] module :)

[IGreyGooI]

@NathanSWard Regarding the benchmark
I did some benchmark test. By benching taskpool.scope(). it appears the performance with fewer workers(4, 8workers) has regressed by 5% - 20%, and with larger worker pool(16, 32 workers) the performance is indifferent. I suspect the main difference is that async_executor::Executor::run() uses a Runner with a local cache as well as a variety of methods for finding work, but in PriorityExecutor::run(), it only uses async_executor::Executor.try_tick(), which find a task by popping the global queue.
I will try to replace async_executor::Executor.try_tick() with a Runner to see if the situation is better with fewer worker threads.

the benchmark is uploaded at /~https://github.com/IGreyGooI/bevy_task_pool_branchmark

[IGreyGooI]

@NathanSWard Regarding the benchmark
I did some benchmark test. By benching taskpool.scope(). it appears the performance with fewer workers(4, 8workers) has regressed by 5% - 20%, and with larger worker pool(16, 32 workers) the performance is indifferent. I suspect the main difference is that async_executor::Executor::run() uses a Runner with a local cache as well as a variety of methods for finding work, but in PriorityExecutor::run(), it only uses async_executor::Executor.try_tick(), which find a task by popping the global queue.
I will try to replace async_executor::Executor.try_tick() with a Runner to see if the situation is better with fewer worker threads.

the benchmark is uploaded at /~https://github.com/IGreyGooI/bevy_task_pool_branchmark

Runner is not exposed async_executor, and no other cheap workaround could be found.
Until we have our own executor, this could still be an improvement regarding performance on the assumption, which is the more threads that could be being utilized by solving #1907 bring more performance than the loss of using async_executor::Executor.try_tick() instead of async_executor::Executor::run().

Further improvement could be done by implementing a modified version of async_executor, or other potential workaround.

[mtsr]

You could test the effect by using a patched async_executor with Runner exposed. Just to see if it's worth bringing it up with the author.

[NathanSWard]

Until we have our own executor, this could still be an improvement regarding performance on the assumption, which is the more threads that could be being utilized by solving #1907 bring more performance than the loss of using async_executor::Executor.try_tick() instead of async_executor::Executor::run().

yep that does make sense, I'd would be nice to see some diagnostics that all available cores are being used based on the config (and that it is actually a performance improvement).
However, I don't know if we should just bring in a change that fixes some of the cases but regresses performance in others.
It may be worth creating a separate issue about bevy having it's own executor that directly supports priority scheduling and then pursuing that path instead.
However, I'm also open to other solutions 😄

[IGreyGooI]

I refer to the benchmark under bevy/benches/bevy_tasks. For benchmarks under overhead_par_iter/threads, which evaluate taskpool.scope() as well, I check my commit against current main, and no significant improvement or regress is noticable(the difference is close to the noise of running same benchmark twice).

Could someone else run cargo bench iter inside bevy/benches with two different commit to confirm it?

[IGreyGooI]

Here is the benchmark result from running cd ./benches & cargo bench iter on 73f4a9d
benchmark_bevy_main_73f4a9d.log

[IGreyGooI]

Here is the benchmark result from running cd ./benches & cargo bench iter on 01b46f3
benchmark_bevy_priority_executor_01b46f3.log, which is compared to benchmark_bevy_main_73f4a9d.log

[alice-i-cecile]

@IGreyGooI, can you comment in #2373? We've since relicensed to MIT + Apache, and I'd like to either ensure that this can be picked up or close it out.

[hymm]

this is potentially superceded by #4740 too.