-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rust: add short style guide #4399
Merged
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a little lukewarm about this rule, in particular when it comes to protobuf definitions that we clearly control ourselves (like the ones in tensorboard.data; and possibly in the future if we were to explicitly fork the event / summary protos and stop mirroring TF). Had responded on the thread, ICYMI: #4385 (comment)
I realize that it's a little hassle to have to go and update the code everywhere if we decide to change the proto, but that work isn't any different from if we'd made the proto definition a Rust struct natively and had decided to add a field, and yet we don't write all our struct literals with a trailing
..Default::default()
in case they're expanded in the future. I think this is more or less a similar argument as described in https://abseil.io/tips/147 about exhaustive switch cases in C++; it seems okay to me to say that we will sign up for either 1 or 2 (3 wouldn't apply unless we were to include the mirrored TF protobufs in this as well).I'd be more sympathetic here if we could at least make this a compile-time check or a linter check (or if Prost generated a proto-creation API that was future-proof, like a
withFoo()
builder pattern, that we could use instead), but if we can't, it seems like there's a decent chance eventually at least one case will slip through, and since we can't prevent that possibility we'll want to be aware of the possibility that a protobuf schema addition will break the Rust build even if we try to adhere to this rule.Would be curious to hear thoughts from others and whether you think there's a particularly ironclad reason we should strive to provide protobuf changes with this kind of guarantee. (FWIW, the only backwards-compatibility guarantee I'm aware of with protobuf is at the wire level; I am not aware of any general guarantees about backwards-compatibility for generated code when the definition changes, though I'll concede that in other generated code I've worked with, at least adding a field has always been backwards-compatible.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I had indeed missed that; thanks for surfacing it!
It’s fine with me to waive this suggestion for protos that we clearly
control ourselves: i.e., those whose source of truth is in this repo.
My main goal with this suggestion was that running
update.sh
to pullin new TensorFlow protos should be a safe operation, because (a) it may
be run by people who aren’t familiar with this code, and (b) it tends to
happen around release time, so we would prefer not to rush any changes.
That’s why I see it differently than changing a normal Rust struct: in
that case, you’re deliberately making a change to the Rust code rather
than just making what should be an automatable change.
Agreed, hence my tips for handling such a situation in this document.
I don’t mind the mutation, but the problem that you run into here is the
loss of type information:
…which gets worse with more structure (deeper nesting, repeated fields).
I guess you can kind of roll your own nested builders thanks to Rust’s
expression blocks:
…and while this does literally about the
..Default::default()
spreads,it doesn’t really seem to satisfy the spirit of your objection. That is:
it’s still a bit noisy.
$0.02: I personally don’t find the
..Default::default()
spam thatbad, but I accept that reasonable people may, and if there were shorter
sugar then I would happily use it.
We could macro up some builders:
…and while I won’t run out and do this unilaterally, I also wouldn’t mind it
if it would make you happier and if it doesn’t break type-directed
autocompletion (which I think I can wrangle).
You kind of implicitly have this on any
proto_library
with publicvisibility in Google’s monorepo, though, right? I agree that they don’t
apply to all wire-compatible changes (renames, most obviously), but my
impression has been that people feel free to add fields.
Also, https://google.aip.dev/180#adding-components says that adding
“components” (incl. fields) is permitted within the same major version,
which implies that they are source-compatible, since “[code] written
against a previous version must compile against a newer version”.
What do you think about keeping the recommendation just for protos whose
source of truth is not in this repo? i.e., those not affected by the
update.sh
script?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried the macro thing for 15 minutes. A bit harder than I thought. Not
pursuing further at this time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's fine by me as you have it, and thanks for the pointer to https://google.aip.dev/180#adding-components - that does indeed formalize this rule. That being said, I think our problem is a bit more fundamental than one that could be patched up by an optional convention (whether macro-assisted or not): Prost generated code in combination with Rust struct literals having to be exhaustive just fundamentally isn't compatible with the API, since it's trivially possible to write code against the generated API that breaks when you add a field.
I think we would need rust-lang/rfcs#1806 (see also related rust-lang/rfcs#1458) to fix this, but it's been postponed :( Either that or some way that Prost structs could disallow use struct literal initialization entirely in favor of forcing you through some sort of builder API, but I'm guessing that eliminates a lot of the "idiomatic Rust flavor" that Prost was trying to offer in the first place.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay; thanks! Agreed, I think; builders are fairly common in Rust
(stdlib examples:
Command
,thread::Builder
,DirBuilder
,probably more), though certainly struct initialization is simpler.
I’ve opened a Prost feature request for builders. If maintainers
seem open to it, maybe I’ll take a stab at implementation.