Skip to content

Commit

Permalink
Windows - Only set thread affinity on Server with auto affinity (#19318)
Browse files Browse the repository at this point in the history
### Description
Only set thread affinity on Server with auto affinity. Auto affinity =
when API user does specify thread settings or affinity themselves.

### Motivation and Context
On client best to let OS scheduler handle. On big (P-Core) / little
(E-Core) CPU designs affinity overrides win32 Quality of Service (QoS)
and has high power usage. Specifically on background workloads whose
process is tagged QoS Utility (Background), this affinity setting
overrides the OS scheduler that only wants to schedule on the E-Cores.
Thus P-Cores waking up uses more energy than intended on client and
users gets less battery life.

Foreground AI workloads would be tagged QoS High and would run the ORT
threads on all cores.
  • Loading branch information
ivberg authored and rachguo committed Feb 8, 2024
1 parent 5269e93 commit c1ce74d
Showing 1 changed file with 10 additions and 0 deletions.
10 changes: 10 additions & 0 deletions onnxruntime/core/util/thread_utils.cc
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

#ifdef _WIN32
#include <Windows.h>
#include <versionhelpers.h>
#endif
#include <thread>
#include "core/session/ort_apis.h"
Expand Down Expand Up @@ -98,7 +99,16 @@ CreateThreadPoolHelper(Env* env, OrtThreadPoolParams options) {
}
options.thread_pool_size = static_cast<int>(default_affinities.size());
if (options.auto_set_affinity) {
#ifdef _WIN32
// Only set thread affinity on Server with auto affinity.
// On client best to let OS scheduler handle.
// On big (P-Core) / little (E-Core) CPU designs affinity overrides QoS and has high power usage
if (IsWindowsServer()) {
to.affinities = std::move(default_affinities);
}
#else
to.affinities = std::move(default_affinities);
#endif
}
}
if (options.thread_pool_size <= 1) {
Expand Down

0 comments on commit c1ce74d

Please sign in to comment.