Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: adapter-node shutdown event #12153

Merged
merged 14 commits into from
Jun 13, 2024

Conversation

karimfromjordan
Copy link
Contributor

@karimfromjordan karimfromjordan commented Apr 22, 2024

This is a small improvement and the missing piece to the recent graceful shutdown feature.
After the server has closed gracefully Node will emit the exit event. As others have pointed out however if you have something running in the background the app will continue to run and exit will never be emitted. The exit event isn't ideal anyway because it will close your app as soon as possible so we can't do any custom async cleanup work in a process.on('exit', () => { ... }) handler. That means calling process.exit() to manually emit the exit event isn't an option. It defeats the point of a graceful shutdown. This PR emits a custom event instead that users of adapter-node can listen to and use to perform any kind of cleanup work right after the HTTP server has closed.

process.on('sveltekit:shutdown', async (reason) => {
  await jobs.stop();
  await db.close();
});

We briefly discussed the option of exporting a function to register a hook instead but with the way adapters work this doesn't seem to be possible today.


Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: /~https://github.com/sveltejs/rfcs
  • This message body should clearly illustrate what problems it solves.
  • Ideally, include a test that fails without this PR but passes with it.

Tests

  • Run the tests with pnpm test and lint the project with pnpm lint and pnpm check

Changesets

  • If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running pnpm changeset and following the prompts. Changesets that add features should be minor and those that fix bugs should be patch. Please prefix changeset messages with feat:, fix:, or chore:.

Edits

  • Please ensure that 'Allow edits from maintainers' is checked. PRs without this option may be closed.

Copy link

changeset-bot bot commented Apr 22, 2024

🦋 Changeset detected

Latest commit: bf75322

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@sveltejs/adapter-node Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@@ -186,6 +186,22 @@ By default `adapter-node` gracefully shuts down the HTTP server when a `SIGTERM`

> If you want to customize this behaviour you can use a [custom server](#custom-server).

You can listen to the `sveltekit:shutdown` event which is emitted after the HTTP server has finished shutting down. Unlike Node's `exit` event, the `sveltekit:shutdown` event is always emitted when a shutdown is triggered and also supports handling asynchronous code.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unlike Node's exit event, the sveltekit:shutdown event is always emitted when a shutdown is triggered

What do you mean by this? When isn't exit emitted? I don't see anything about it only happening sometimes in the Node docs

and also supports handling asynchronous code.

maybe we could say "asynchronous operations" instead of "asynchronous code" to match the wording of the Node docs

Copy link
Contributor Author

@karimfromjordan karimfromjordan Jun 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exit is emitted automatically when Node has no work left to do. But there might be the scenario where the SvelteKit server shuts down because we listen for a SIGINT but the Node app still doesn't close because the user has a Mongo DB connection in the background that prevents it. That's what's happening in the comment I linked to at the beginning. In that case, even though the SvelteKit server closed, an exit event is not emitted. And if we manually emit the exit event after the server has closed using process.exit() the user can't run any async cleanup operations because process.exit() forcefully closes the app as quickly as possible.

if (shutdown_timeout_id) {
shutdown_timeout_id = clearTimeout(shutdown_timeout_id);
}
if (idle_timeout_id) {
idle_timeout_id = clearTimeout(idle_timeout_id);
}

// @ts-expect-error custom events cannot be typed
process.emit('sveltekit:shutdown', reason);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit torn as to whether this is the right API. It is more convenient to work with but kind of duplicating/aggregating existing events. The alternative would be to have the user do this:

process.on('SIGINT', shutdownHandler)
process.on('SIGTERM', shutdownHandler)
process.on('sveltekit:idle', shutdownHandler)

I think what you're doing is probably nicer, but I thought I'd note it for posterity or in case other reviewers have a different opinion

Copy link
Contributor Author

@karimfromjordan karimfromjordan Jun 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that this doesn't work with the graceful shutdown. If the user listens for the SIGINTand SIGTERM events to run some cleanup code then that cleanup code will run at the same time as the server is shutting down and when some requests might not have finished yet. But ideally you want to close the db connection etc. after the server has closed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, got it. I was a bit thrown by the process.on('SIGTERM', shutdown); line. Do you think we could rename shutdown to graceful_shutdown to make it a bit clearer in the code that it's not shutting down immediately?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah makes sense. I just renamed it.

@@ -83,7 +90,7 @@ server.server.on(
server.server.closeIdleConnections();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm. I wonder if users would want an event to know that all requests have finished processing. if you issue a sveltekit:shutdown when receiving SIGINT and shutdown your database connections then maybe your requests can't actually finish and fail because the database connection is gone, which would kind of defeat the purpose of having a shutdown timeout.

maybe it'd be better to issue an event here rather than in shutdown. if users want to do something earlier they can still listen for SIGTERM or SIGINT individually without us firing an event that simply echoes them (/~https://github.com/sveltejs/kit/pull/12153/files#r1637507349)

Copy link
Contributor Author

@karimfromjordan karimfromjordan Jun 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was the idea. The sveltekit:shutdown event is emitted inside the server.close((error) => { ... }) callback. And that callback runs after all requests/connections have finished and the server closed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I didn't realize that server.close() was graceful. I added suggestion for a comment here: /~https://github.com/sveltejs/kit/pull/12153/files#r1638706765

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>
Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>
if (requests === 0 && shutdown_timeout_id) {
// when all requests are done, close the connections, so the app shuts down without delay
if (shutdown_timeout_id) {
// close connections as soon as they become idle, so the app shuts down without delay
Copy link
Member

@benmccann benmccann Jun 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above basically... I don't understand exactly what this is doing. If we call .close() isn't that closing idle connections already? Why handle it here as well?

Copy link
Contributor Author

@karimfromjordan karimfromjordan Jun 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a connection was opened with a keep-alive header close() will wait for the connection to time out rather than close it early even if the connection is not handling any requests at the moment. That's one of the reasons why closeIdleConnections() was added to Node.js which also closes idle keep-alive connections. So to make sure that all idle connections are closed as soon as possible we should call both methods, see also nodejs/node#41578

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw. I added that commit today because it occurred to me that if we don't close the connections asap it could theoretically happen that a connection stays open, continues to receive requests and therefor later has to be forcefully closed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome. great job covering the bases on this! that'd be so easy to miss. it's really great you figured out that needed to be done

Copy link
Member

@benmccann benmccann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

really great job on this!

thank you so much for seeing this through and being patient with me trying to handle our long queue of PRs to review. I also appreciate the updates to help with readability and comments since we don't have tests for this package yet! one of many things I should probably get to one day...

@benmccann benmccann merged commit 1aa1f32 into sveltejs:main Jun 13, 2024
8 of 12 checks passed
@github-actions github-actions bot mentioned this pull request Jun 13, 2024
@karimfromjordan karimfromjordan deleted the adapter-node/shutdown-event branch June 14, 2024 05:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants