Support decoding Decimal types from numeric (int/float) values
#463
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If a field is typed as decimal, JSON numbers will be coerced to a `decimal.Decimal` value. This is done by parsing the raw JSON bytes, and won't result in a loss of precision by going through a `double` first.
This finishes up support for converting int/float inputs to
`decimal.Decimal` types in both `msgspec.convert` and
`msgspec.msgpack.decode`.
Note that there may be precision loss when converting from float to
decimal, value dependent. We don't pass the value directly to
`decimal.Decimal`, but instead convert to a str expressing the shortest
roundtrippable value for the IEEE754 decimal. This is effectively
equivalent to
```
if isinstance(input, float):
return decimal.Decimal(str(input))
```
For YAML/TOML inputs we _could_ ensure zero precision loss if needed
with some extra work, since floats for these protocols are also
serialized as str values. I'm punting on this for now until someone asks
about it.
For msgpack there's no way around this - float values are serialized as
their binary inputs.
In all cases you really should be serializing `decimal.Decimal` values
as strings, since this is the format that is easiest to avoid precision
loss, and is widely and uniformly supported across tools.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds support for converting int/float inputs to
decimal.Decimaltypes for all protocols'decodemethods, as wellmsgspec.convert.The intent of this is to provide more flexible input support for
decimal.Decimaltypes, since not everyone wants to encode them as strings.For JSON fields there is no loss of precision, the numeric representation is parsed directly into the Decimal object, same as if it was a string.
For all other protocols, there is a possibility of precision loss, when converting from float -> decimal. We don't pass the value directly to
decimal.Decimal, but instead convert to a str expressing the shortest roundtrippable value for the IEEE754 decimal. This is effectively equivalent toIn common simple cases this likely does what the user wants. Still, there's a chance of precision loss, see the added documentation for examples of cases where this matters.
For YAML/TOML inputs we could ensure zero precision loss if needed with some extra work, since floats for these protocols are also serialized as str values (same as JSON). I'm punting on this for now until someone asks about it.
For msgpack there's no way around this; since it's a binary protocol, float values are serialized as their binary inputs.
In all cases you really should be serializing
decimal.Decimalvalues as strings, since this is the format that is easiest to avoid precision loss, and is widely and uniformly supported across tools.