compute_ctl: Perform more startup actions in parallel #11008
+114
−62
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hlinnaka
requested review from
ololobus,
MMeent and
tristan957
and removed request for
a team
|
Note: This is part of a 3-part series of PRs: |
7744 tests run: 7364 passed, 0 failed, 380 skipped (full report)Flaky tests (2)Postgres 17
Code coverage* (full report)
* collected from Rust tests only The comment gets automatically updated with the latest test results
e54eda2 at 2025-02-28T13:57:09.036Z :recycle: |
This was referenced Feb 26, 2025
hlinnaka
force-pushed
the
heikki/refactor-compute_ctl-2
branch
from
2f8b394 to
dae4c2a
Compare
hlinnaka
force-pushed
the
heikki/parallelize-start_compute
branch
from
ea66b21 to
abd86d1
Compare
tristan957
approved these changes
Feb 26, 2025
hlinnaka
force-pushed
the
heikki/refactor-compute_ctl-2
branch
from
dae4c2a to
7bc981e
Compare
hlinnaka
force-pushed
the
heikki/parallelize-start_compute
branch
from
3a6fefc to
6244f72
Compare
tristan957
reviewed
Feb 26, 2025
hlinnaka
force-pushed
the
heikki/parallelize-start_compute
branch
from
6244f72 to
2a0918e
Compare
MMeent
reviewed
Feb 27, 2025
compute_tools/src/compute.rs
Outdated
| // If there are any remote extensions in shared_preload_libraries, start downloading them | ||
| if pspec.spec.remote_extensions.is_some() { | ||
| let (this, spec) = (self.clone(), pspec.spec.clone()); | ||
| pre_tasks.spawn_blocking_child(move || this.download_preload_extensions(&spec)); |
There was a problem hiding this comment.
With async closures (async || {}) I think this should not need the clone() operations.
There was a problem hiding this comment.
I couldn't get that to work
Comment on lines
+613
to
+588
| let (this, cs) = (self.clone(), compute_state.clone()); | ||
| pre_tasks.spawn_blocking_child(move || this.prepare_pgdata(&cs)); |
hlinnaka
force-pushed
the
heikki/parallelize-start_compute
branch
from
2a0918e to
32f2bc7
Compare
hlinnaka
force-pushed
the
heikki/refactor-compute_ctl-2
branch
from
14de70f to
33a2459
Compare
To speed up compute startup. Resizing swap in particular takes about 100 ms on my laptop. By performing it in parallel with downloading the basebackup, that latency is effectively hidden. I would imagine that downloading remote extensions can also take a non-trivial amount of time, although I didn't try to measure that. In any case that's now also performed in parallel with downloading the basebackup.
hlinnaka
force-pushed
the
heikki/parallelize-start_compute
branch
from
22d1fb2 to
dd4c55d
Compare
ololobus
reviewed
Feb 28, 2025
| @@ -669,20 +679,6 @@ impl ComputeNode { | |||
| let config_time = Utc::now(); | |||
| if pspec.spec.mode == ComputeMode::Primary { | |||
| self.configure_as_primary(&compute_state)?; | |||
| let conf = self.get_conn_conf(None); | |||
| tokio::task::spawn_blocking(|| { | |||
| let res = get_installed_extensions(conf); | |||
There was a problem hiding this comment.
Hm, looks like this part got removed completely, but we need this for the extensions' metrics
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To speed up compute startup. Resizing swap in particular takes about 100 ms on my laptop. By performing it in parallel with downloading the basebackup, that latency is effectively hidden. I would imagine that downloading remote extensions can also take a non-trivial amount of time, although I didn't try to measure that. In any case that's now also performed in parallel with downloading the basebackup.