This repository has been archived by the owner on Nov 1, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Harden and simplify yaml stream decoding/encoding #1931
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2opremio
changed the title
Support contigous yaml EOD markers
Support contigous yaml SOD markers
Apr 12, 2019
2opremio
force-pushed
the
1925-contigous-yaml-EOD
branch
from
April 12, 2019 19:10
7e6125b
to
395cf48
Compare
hiddeco
reviewed
Apr 12, 2019
Uhm, I think I got something wrong. The test seems to pass without the code changes ... |
2opremio
changed the title
Support contigous yaml SOD markers
[WIP] Support contigous yaml SOD markers
Apr 12, 2019
2opremio
force-pushed
the
1925-contigous-yaml-EOD
branch
from
April 13, 2019 00:47
395cf48
to
d051c92
Compare
2opremio
changed the title
[WIP] Support contigous yaml SOD markers
[WIP] Harden and simplify yaml stream parsing
Apr 13, 2019
2opremio
force-pushed
the
1925-contigous-yaml-EOD
branch
2 times, most recently
from
April 13, 2019 01:47
9e0ed2c
to
cf189e2
Compare
2opremio
changed the title
[WIP] Harden and simplify yaml stream parsing
Harden and simplify yaml stream parsing
Apr 13, 2019
2opremio
pushed a commit
to 2opremio/kubeyaml
that referenced
this pull request
Apr 13, 2019
Technically it's correct to add an end-of-document mark to the output. However, `kubectl` and `Flux` (before fluxcd/flux#1931 ) choke on it.
2opremio
pushed a commit
to 2opremio/kubeyaml
that referenced
this pull request
Apr 13, 2019
Technically it's correct to add an end-of-document mark to the output. However, `kubectl` and `Flux` (before fluxcd/flux#1931 ) choke on it. This change makes sure we always use `---` markers to separate documents and that we never add `...`.
2opremio
pushed a commit
to 2opremio/kubeyaml
that referenced
this pull request
Apr 13, 2019
Technically it's correct to add an end-of-document mark to the output. However, `kubectl` and `Flux` (before fluxcd/flux#1931 ) choke on it. This change makes sure we always use `---` markers to separate documents and that we never add `...`.
2opremio
pushed a commit
to 2opremio/kubeyaml
that referenced
this pull request
Apr 13, 2019
Technically it's correct to add an end-of-document mark to the output. However, `kubectl` and `Flux` (before fluxcd/flux#1931 ) choke on it. This change makes sure we always use `---` markers to separate documents and that we never add `...`.
Flux choked on end-of-document markers (`...`). To avoid complicating the existing multidoc parser (stolen from kubernetes) I abused go-yaml's Decoder to obtain the raw documents from the stream by unmarshalling to an interface{} and marshalling again.
Use go-yaml's Decoder instead of our own
2opremio
force-pushed
the
1925-contigous-yaml-EOD
branch
from
April 13, 2019 03:53
cf189e2
to
8edd13f
Compare
2opremio
changed the title
Harden and simplify yaml stream parsing
Harden and simplify yaml stream decoding/encoding
Apr 14, 2019
2opremio
force-pushed
the
1925-contigous-yaml-EOD
branch
from
April 14, 2019 18:01
2a47817
to
abe9db8
Compare
hiddeco
approved these changes
Apr 15, 2019
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Fons 🍪
Q: Does this mean comments are no longer preserved when files are updated by flux? |
A: no, because the kubeyaml-using code still just reads a whole file in, and writes a whole file out. |
Yep, exactly, sorry I didn't get to answer the question in time. |
squaremo
pushed a commit
to squaremo/kubeyaml
that referenced
this pull request
Apr 24, 2019
Technically it's correct to add an end-of-document mark to the output. However, `kubectl` and `Flux` (before fluxcd/flux#1931 ) choke on it. This change makes sure we always use `---` markers to separate documents and that we never add `...`.
This was referenced Apr 29, 2019
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Flux choked on end-of-document markers (
...
).To avoid complicating the existing multidoc parser (stolen from kubernetes) I abused
go-yaml
'sDecoder
to obtain the raw documents from the stream by unmarshalling to aninterface{}
and marshalling again.Also, I replaced our ad-hoc stream parsing and generation by a
go-yaml
Decoder
andEncoder
.I, however, left the encoding part of
fluxctl save
untouched sinceyaml.Encoder
generates different output (no start-of-document prefix) and users may be relying on it.During testing I made sure that changes I made in
fluxd
'sExport()
are compatible withfluxctl save
prior to this change.Fixes #1925