What are the core "data" types?

[cwebber]

We should list them out.

@erights already wrote up about the scalars in first CapTP implementation in Agoric's code.

Goblins' CapTP has:

• Booleans
• Floats (yes, this needs to be broken down since that's a big ol' rathole)
• Signed integers
• Bytestrings (really, arbitrary binary data)
• Strings
• Symbols (think lisp or ruby symbols)
• Dictionaries/maps
• Lists/arrays
• Records (yes, arbitrary user "records" in preserves/syrup... this is the "extension-type type"... these are encoded as <'user-record record-tag [record-arg1, record-arg2, ...]> on the wire)
• Unordered sets
• Strings (unicode)
• Refs, both "local" and "remote", with each distinctly typed for "promise" vs "object" (so, really four kinds of refs, but really on the wire this is about incoming vs outgoing) (btw "local" and "remote" are akin to near and far generally, but in Goblins near and far refer to "in the same vat", and multiple vats may be exposed by the same captp... I know this is up for discussion because of the possibility of vat migration)
• Errors (right now, only the "mystery" error, in the future, more types)
• Void (equivalent to "undefined")

Sorta on the wire there's some other things too, like the signed-handoff-give certificates, but that's not something that ever reaches code, so I consider that a different thing... more of an AST discussion.

[erights]

See /~https://github.com/Agoric/agoric-sdk/pull/2909/files for a draft more detailed semantics of the Agoric CapTP's Passable. The parts above the divider (currently the entire PR) are the abstract syntax and semantics. Only the part below the divider are about our concrete encoding into JSON. Switching to a different concrete seriailzation (e.g., syrup) should not affect the part above the divider.

Although somewhat biased towards JS, the abstract semantics above the divider is intended to be language independent enough to serve as a basis of inter-language interoperability, e.g., with Goblins and perhaps with Cap'n Proto. We'll see.

[kriskowal]

I propose the following straw-man mapping to develop upon:

JavaScript	Racket	Serialization
boolean	boolean
bigint	signed integers	byte length prefixed decimal, zig-zag varint, or varint with sign in type
number	flonum	e.g., 64 bit big endian IEEE floating point number
Uint8Array	bytestring	byte length prefixed bytes
string UTF-16	string	byte length prefixed UTF-8
Array	list	zero or more heterogenous values
record Objects	?	correspond to structs, alternating string or symbol keys and values, or similar
Error	error	error tag with a message and a stochastic trace identifier a la jaeger/opentelemetry
null	void

symbols I propose that symbols are debatable. It might be that we allow
symbols in limited places like method invocation and record keys, which would
allow them to appear as idiomatic strings in JavaScript and symbols proper in
Racket. A wire representation for symbols that normalizes language case
convention would be possible too. As Kenton mentioned, case normalization is a
solved problem in IDL systems.

records JavaScript APIs will require records with keys.
The keys might either be strings or symbols.
With an IDL, JavaScript records can be modeled as structs with positional or
numerically tagged fields like Protobuf.
Without an IDL, the field names must appear on the wire.

integers Does Racket impose an upper bound on the precision? JavaScript
bigint does not. If one language or the other has an upper bound, we’ll need
to make that 100% clear and establish an understanding that these will get
truncated or errored if too big to deserialize on the other side.

null and void I propose that JavaScript null needs to be explicit on the
wire, and perhaps undefined only occurs to note a missing record field.
I believe @erights’s position is that we need both null and undefined.

float JavaScript would have to promote all float32 to float64. I propose that
we only need float64 and that if brevity on the wire is a concern, the mantissa
is likely to have a lot of trailing (in big endian) zeroes that are either easy
to compress or truncate based on the length implied by a type tag.

enums Without an IDL, enums must be either symbols or strings. With an IDL,
they must be numbers. In both cases, the necessity comes from schema migration
and proxies. In the face of schema migration, a proxy might not know all the
values of an enum that pass through. If there's no IDL, an unknown string can
be passed blindly. If there is an IDL, an unknown number can be passed blindly.
With a hybrid of strings and numbers, the proxy would have to model the enum as
both a string and a number to transit any unrecognized value.

Sets and maps are too hard and I would like to not talk about them yet.

[erights]

Let's compare to /~https://github.com/Agoric/agoric-sdk/blob/captp-semantics/packages/marshal/src/types.js which is from the fork corresponding to the PR Agoric/agoric-sdk#2909 . I think these are in close alignment. Comparing them will make the differences clear. I'm going to focus only on the abstract syntax and semantics. I certainly agree we need to actually agree on concrete syntax to interoperate, but I don't much care about that, as long as it is full fidelity to the abstract syntax and semantics. On that page, the abstract issues are above the divider. Below the divider is our concrete encoding into JSON which I think we're all eager to kill.

The key thing I tried to write down in that proposal is to account for

• embedded leafcaps, which Agoric's current serialization puts into a separate slot array, but serializes a reference to the slot,
• equality semantics semantics, which is required of anything that can be used in a key in a map or set
  • remotables have an unforgeable testable identity (that must support grant matching)
  • promises, which cannot be compared for equality
• canonicalization, such that two things that can be used as keys in maps or sets are equal iff they have the same serialization
  • this requires that set-like things that can be used as keys have sortable parts
  • remotables cannot be canonically ordered, and so such sortable parts can't depend on remotable order.

The rows of Kris' table

• boolean -- no problem.
• bigint -- no problem
• number -- IEEE 64 bit floating point with one canonical NaN and without a negative zero
• Uint8Array -- a TODO on my page, but I don't expect a protocol problem. (Separately, JS will have a representation problem until we can freeze Uint8Array instances.)
• string -- I think so. I agree it should encode using the UTF-16 encoding. But it should be able to encode any string representable in JSON, which therefore must cover such horrors as unpaired surrogates.
• Array -- here is where the parameterization starts to get interesting. The arrays must be able to include remotables and promises. But, for example, arrays containing promises cannot be compared for equality, and so cannot serve as keys in maps or sets. Also for example, two singleton sets each containing a different remotable cannot be ordered. And thus a map containing each as a key cannot be serialized in a canonical order. Arrays themselves of course have a canonical internal order.
• Records aka structs -- Their property names can only be strings. However, it is conceivable that the scheme mapping will turn property names into scheme (records? structs? dictionaries?) with corresponding interned symbols as keys. This should not affect the protocol or its abstract meaning, where these are strings. Because these strings can be sorted, records have the same conditional equality and ordering properties as arrays.
• Error -- yeah, probably what Kris said. That trace identifier has complex implications on equality and canonicalization, but the need seems overwhelming. I'm not at all happy with what I wrote down on that page.
• null -- as Kris warned everyone about, I'm going to insist on two bottom values, which we can call null and undefined. The problem is that Agoric's code, and most sensible JS code, almost always uses only undefined. But JSON can only do null and not undefined. Thus, JS code that gets data from JSON will still be handling nulls. I just don't think there's anything else we can do without creating a bug farm for JS programmers. I'm really sorry about this one.

What else is mentioned on that page but absent from Kris' table?

• Convenient names for non-leaf groups, such as LeafData for all the scalars. This only becomes interesting when these labels represent semantics, like PassableKey which can include, for example, arrays of numbers but not promises.
• symbol -- I am open both to doing this or to not doing this, depending what the tradeoffs look like as we talk this out. What we're all agreed on is that if symbols are admitted as data distinct from strings, then each symbol is uniquely associated with a string. Two symbols are equal iff their associated strings are equal. This decision is independent of whether
  • records property name strings correspond, for a given language (scheme) to symbol-named properties, for the symbols named by those strings.
  • whether method name strings (which are outside our table for now anyway) are similarly taken by a given language to name symbols rather than strings
• LeafCaps, i.e., remotables and promises.
• Sets and Maps, which Agoric does not implement yet. So their inclusion is more speculative on my part. But they're important to examine because the equality of their keys is what makes all this a non-trivial exercise.

[erights]

For a binary representation of floating point, there remains the question of which NaN representation is the canonical one. We can follow ewasm on this one. wasm leaves NaN encodings up to the implementation, as does JS. But ewasm, as well as any other use of wasm on the blockchain, must ensure that all validators stay in agreement. All the blockchain wasms canonicalize all NaNs to some one NaN. We should use the same one they did.

[zenhack]

integers Does Racket impose an upper bound on the precision?

No, racket integers are proper bigints too.

---

To stay sane and avoid ambiguity, I'm going to suggest standardizing on some terminology:

• "records" denote a product type with a fixed set of labeled fields, consistent with how I believe @kriskowal is using the term. This is also consistent with the way the term is used in most FP languages that have them. Note that this is entirely unrelated to what preserves, and by extension syrup, call a "record."
• I would like to propose that what preserves/syrup calls records we instead call "tags," taking a cue from CBOR, which has a similar construct. So, a tag refers to the application of a label to some other value(s). This is a natural way to represent sum types for languages which have them, but is also useful for "extending" the set of available data types. CBOR's version of this expects numeric labels and exactly one argument, while preserves/syrup is more flexible, allowing any value as a label and zero or more arguments.

---

Some other thoughts:

• I am very antagonistic to baking in both undefined and null as a primitives, and I think it would be better to encode this as a tag. I think being able to describe undefined without inventing a one-off type is also a good test case for whether the data model is likely to be flexible enough to handle the curveballs in the types provided by $LANGUAGE as well.
• The nature of errors/exceptions is probably a topic that deserves it's own issue. This is something where Cap'n Proto has some requirements. I will open an issue for this.

[erights]

First, I like your terminology. That is also what I mean by a record.

I don't know what it would mean to standardize while leaving the set of tags open. What namespace defines the meaning of these names, if the interoperable standard does not.

I am completely open to using tags that we define to represent abstract syntax that we need. In particular, it is fine with me to use a tag for undefined. We will also need to fit remotables and promises into tags, probably with the same kind of slot encoding that Agoric is currently using.

I look forward to hearing about Cap'n Proto errors/exceptions. I have not been able to get to clarity on this starting only with our requirements. I am unsatisfied with our current errors.

[erights]

To avoid painting myself into a corner, let me double check: If undefined is encoded as a standardized tag vs "as a primitive" should have no effect on its abstract syntax or semantics, right? Otherwise I'm missing something crucial about tags.

[kriskowal]

My understanding is that whether undefined or null are serialized as primitives or tags has no effect on the abstract syntax or semantics of the CapTP. I believe there’s an implicit sub-conversation about where the codec ends and the CapTP begins. My understanding is that with Syrup we would use tags as an extension point to fill in CapTP features that are not met by the underlying serialization. If we take the Cap’n Proto meta-IDL approach, there will be a similar line between what’s expressible just using Cap’n Proto’s data model and what of our CapTP’s concerns will need to be filled out by the meta-IDL. I do not think that question is important to answer at this time. That being said, if we use Syrup-as-defined-today as a foundation for our CapTP, tags (née syrup records) would have to be the way we would express the four kinds of references. My intuition is that, with Syrup at least, we won’t be able to preserve a separation of these two layers in the CapTP implementation. For example, the Syrup I wrote yesterday is able to enforce the canonical representation of data on the wire: the decoder compares adjacent keys pairwise, and the encoder sorts keys in composing “syrup dictionaries / JavaScript struct-like Object records”. When we add maps to CapTP, the serializer will need to know not just the bytes of keys, but whether those bytes contain any non-sortable references. That suggests that whether something is a primitive or a tag in the CapTP that emerges from the serialization and some well-defined tags is really not germane.

…

On Fri, Apr 23, 2021 at 9:25 PM Mark S. Miller ***@***.***> wrote: To avoid painting myself into a corner, let me double check: If undefined is encoded as a standardized tag vs "as a primitive" should have no effect on its abstract syntax or semantics, right? Otherwise I'm missing something crucial about tags. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5 (comment)>, or unsubscribe </~https://github.com/notifications/unsubscribe-auth/AAAOXBR6OAV4KTTVULLEF6TTKJB2BANCNFSM43BQCS4Q> .

[zenhack]

I guess my goal here is that an implementer who doesn't care about javascript specifically should not have to worry about and/or specifically implement support for oddball concepts that don't exist anywhere else (such as undefined) -- the core data types should be things which are fairly universal, and then we add tags on top of that, which implementations can expose in some generic way (In javascript, you might make them objects with a special Tag prototype that can be used to identify them). So, the javascript implementation can know about the encoding of undefined as a tag and decode it specially as javascript undefined, but other languages will just see it as a generic tag, and if the users of that language need to talk to a JS vat that uses undefined in its API, they can construct that by doing Tag("undefined") or the like. In js-to-js communication you would likely not use the explicit tag prototype much if at all, but it would be there to talk about types in other environments that don't have obvious analogs without having to come up with an ad-hoc mapping in each case.

We could still put common tags in the a somewhere, without requiring each implementation to handle each of them specially. CBOR actually has an IANA registry where numeric labels are registered on a first-come-first-serve basis.

Re: remotables/promises as tags, goblins does this, but note that it is able to avoid namespace collisions with application-defined tags by "escaping" them, so e.g. <foo 32> would be re-encoded as <user-record foo 32>. (though we might want rename it to user-tag).

[zenhack]

(Note that this principle could also cut the other way -- rather than fuss about how to encode scheme symbols in js, we could just drop them from the core data types and encode them as tags, with a spec-dictated label and a string as an argument).

[zarutian]

@erights wrote:

The rows of Kris' table

• boolean -- no problem.
• bigint -- no problem
• number -- IEEE 64 bit floating point with one canonical NaN and without a negative zero

That would be IEEE 754 double precision in big endian (network) order?

• Uint8Array -- a TODO on my page, but I don't expect a protocol problem. (Separately, JS will have a representation problem until we can freeze Uint8Array instances.)

I thought those were freezable? Today I learned.

• string -- I think so. I agree it should encode using the UTF-16 encoding. But it should be able to encode any string representable in JSON, which therefore must cover such horrors as unpaired surrogates.

Surely you meant that strings should be utf-8 encoded on the wire?

• Array -- here is where the parameterization starts to get interesting. The arrays must be able to include remotables and promises. But, for example, arrays containing promises cannot be compared for equality, and so cannot serve as keys in maps or sets. Also for example, two singleton sets each containing a different remotable cannot be ordered. And thus a map containing each as a key cannot be serialized in a canonical order. Arrays themselves of course have a canonical internal order.

Hmm, this only applies what in E were called ConstLists, no? (Shallow immutable selfless)

• Records aka structs -- Their property names can only be strings. However, it is conceivable that the scheme mapping will turn property names into scheme (records? structs? dictionaries?) with corresponding interned symbols as keys. This should not affect the protocol or its abstract meaning, where these are strings. Because these strings can be sorted, records have the same conditional equality and ordering properties as arrays.

Gets me into the mind of tagged tuples with named positions in them.

• Error -- yeah, probably what Kris said. That trace identifier has complex implications on equality and canonicalization, but the need seems overwhelming. I'm not at all happy with what I wrote down on that page.

What about Errors that contain sealed and possibly access-level-tag redacted callstack trace?

• null -- as Kris warned everyone about, I'm going to insist on two bottom values, which we can call null and undefined. The problem is that Agoric's code, and most sensible JS code, almost always uses only undefined. But JSON can only do null and not undefined. Thus, JS code that gets data from JSON will still be handling nulls. I just don't think there's anything else we can do without creating a bug farm for JS programmers. I'm really sorry about this one.

One explanation on the difference of null and undefined is that the former is an marker for empty while the latter is a marker for missing altogether.

What else is mentioned on that page but absent from Kris' table?

• Convenient names for non-leaf groups, such as LeafData for all the scalars. This only becomes interesting when these labels represent semantics, like PassableKey which can include, for example, arrays of numbers but not promises.

So, basically compound pure data datums that could have been shunted out to named-by-content-hash distributed storage.

• symbol -- I am open both to doing this or to not doing this, depending what the tradeoffs look like as we talk this out. What we're all agreed on is that if symbols are admitted as data distinct from strings, then each symbol is uniquely associated with a string. Two symbols are equal iff their associated strings are equal. This decision is independent of whether

  • records property name strings correspond, for a given language (scheme) to symbol-named properties, for the symbols named by those strings.
  • whether method name strings (which are outside our table for now anyway) are similarly taken by a given language to name symbols rather than strings

• LeafCaps, i.e., remotables and promises.

• Sets and Maps, which Agoric does not implement yet. So their inclusion is more speculative on my part. But they're important to examine because the equality of their keys is what makes all this a non-trivial exercise.

Oi! You started this by pining for Symbol.asyncIterator for notifier/subscription purposes! But I do not mind that symbols are supported though.

[erights]

@zarutian wrote

@erights wrote:

The rows of Kris' table

• boolean -- no problem.
• bigint -- no problem
• number -- IEEE 64 bit floating point with one canonical NaN and without a negative zero

That would be IEEE 754 double precision in big endian (network) order?

I don't recall anyone mentioning the endianness, but yet, that's what I would expect. Funny how big endian came to dominate networks and little endian came to dominate processors, forcing the maximal amount of conversion on everyone. At least we no longer need to worry about the DEC VAX mixed endianness.

• Uint8Array -- a TODO on my page, but I don't expect a protocol problem. (Separately, JS will have a representation problem until we can freeze Uint8Array instances.)

I thought those were freezable? Today I learned.

There is now a tc39 proposal by @Jack-Works that would let us freeze Uint8Arrays, as well as other good effects.

• string -- I think so. I agree it should encode using the UTF-16 encoding. But it should be able to encode any string representable in JSON, which therefore must cover such horrors as unpaired surrogates.

Surely you meant that strings should be utf-8 encoded on the wire?

Practically, yes. Technically, IIUC, no. IIUC, utf-8 is specified such that it encodes unicode characters. Lone UTF-16 surrogates are not unicode characters, and so cannot technically be encoded in utf-8. But actual utf-8 encoders and decoders, I would guess, ignore this prohibition and do what we need here.

• Array -- here is where the parameterization starts to get interesting. The arrays must be able to include remotables and promises. But, for example, arrays containing promises cannot be compared for equality, and so cannot serve as keys in maps or sets. Also for example, two singleton sets each containing a different remotable cannot be ordered. And thus a map containing each as a key cannot be serialized in a canonical order. Arrays themselves of course have a canonical internal order.

Hmm, this only applies what in E were called ConstLists, no? (Shallow immutable selfless)

Yes, exactly. A lot of the reasoning behind the semantics of Passables comes from the E experience.

• Records aka structs -- Their property names can only be strings. However, it is conceivable that the scheme mapping will turn property names into scheme (records? structs? dictionaries?) with corresponding interned symbols as keys. This should not affect the protocol or its abstract meaning, where these are strings. Because these strings can be sorted, records have the same conditional equality and ordering properties as arrays.

Gets me into the mind of tagged tuples with named positions in them.

I don't know what you're referring to. Do you have a particular programming language in miind?

• Error -- yeah, probably what Kris said. That trace identifier has complex implications on equality and canonicalization, but the need seems overwhelming. I'm not at all happy with what I wrote down on that page.

What about Errors that contain sealed and possibly access-level-tag redacted callstack trace?

We've been going in a Causeway-like direction, where each site logs the stack traces of sent errors locally without sending the stack traces. The trace identifier is logged on the sending side associated with that stack trace, and is also sent to the receiving side. The receiving side annotates the unserialized error with that trace identifier to communicate it to the local logging system there without making it available from the error object. Then, to trace back across sites each involved site proprietor has to decide what locally logged error stacks to reveal, so these can be stitched together.

Our current state has taken some steps towards this, but is currently a mess (my fault).

• null -- as Kris warned everyone about, I'm going to insist on two bottom values, which we can call null and undefined. The problem is that Agoric's code, and most sensible JS code, almost always uses only undefined. But JSON can only do null and not undefined. Thus, JS code that gets data from JSON will still be handling nulls. I just don't think there's anything else we can do without creating a bug farm for JS programmers. I'm really sorry about this one.

One explanation on the difference of null and undefined is that the former is an marker for empty while the latter is a marker for missing altogether.

I don't know what this would mean in practice. Concretely, where would I use null and where would I use undefined. Frankly, were it not for JSON I would be tempted to drop null altogether. But we're not in that world, so we gotta have both.

What else is mentioned on that page but absent from Kris' table?

• Convenient names for non-leaf groups, such as LeafData for all the scalars. This only becomes interesting when these labels represent semantics, like PassableKey which can include, for example, arrays of numbers but not promises.

So, basically compound pure data datums that could have been shunted out to named-by-content-hash distributed storage.

Which Passable type are you referring to?

• symbol -- I am open both to doing this or to not doing this, depending what the tradeoffs look like as we talk this out. What we're all agreed on is that if symbols are admitted as data distinct from strings, then each symbol is uniquely associated with a string. Two symbols are equal iff their associated strings are equal. This decision is independent of whether

  • records property name strings correspond, for a given language (scheme) to symbol-named properties, for the symbols named by those strings.
  • whether method name strings (which are outside our table for now anyway) are similarly taken by a given language to name symbols rather than strings

• LeafCaps, i.e., remotables and promises.

• Sets and Maps, which Agoric does not implement yet. So their inclusion is more speculative on my part. But they're important to examine because the equality of their keys is what makes all this a non-trivial exercise.

Oi! You started this by pining for Symbol.asyncIterator for notifier/subscription purposes! But I do not mind that symbols are supported though.

Turns out, we are currently still undecided on symbols.

[zarutian]

@erights wrote:

@zarutian wrote

@erights wrote:

The rows of Kris' table

• boolean -- no problem.
• bigint -- no problem
• number -- IEEE 64 bit floating point with one canonical NaN and without a negative zero

That would be IEEE 754 double precision in big endian (network) order?

I don't recall anyone mentioning the endianness, but yet, that's what I would expect. Funny how big endian came to dominate networks and little endian came to dominate processors, forcing the maximal amount of conversion on everyone. At least we no longer need to worry about the DEC VAX mixed endianness.

Just wanted to be sure.

• Uint8Array -- a TODO on my page, but I don't expect a protocol problem. (Separately, JS will have a representation problem until we can freeze Uint8Array instances.)

I thought those were freezable? Today I learned.

There is now a tc39 proposal by @Jack-Works that would let us freeze Uint8Arrays, as well as other good effects.

• string -- I think so. I agree it should encode using the UTF-16 encoding. But it should be able to encode any string representable in JSON, which therefore must cover such horrors as unpaired surrogates.

Surely you meant that strings should be utf-8 encoded on the wire?

Practically, yes. Technically, IIUC, no. IIUC, utf-8 is specified such that it encodes unicode characters. Lone UTF-16 surrogates are not unicode characters, and so cannot technically be encoded in utf-8. But actual utf-8 encoders and decoders, I would guess, ignore this prohibition and do what we need here.

Surrogates in utf-16 came to be due to the Unicode consortium realized thatthey would have many more codepoints than fit in 16 bits. At least that is how the story goes.

• Array -- here is where the parameterization starts to get interesting. The arrays must be able to include remotables and promises. But, for example, arrays containing promises cannot be compared for equality, and so cannot serve as keys in maps or sets. Also for example, two singleton sets each containing a different remotable cannot be ordered. And thus a map containing each as a key cannot be serialized in a canonical order. Arrays themselves of course have a canonical internal order.

Hmm, this only applies what in E were called ConstLists, no? (Shallow immutable selfless)

Yes, exactly. A lot of the reasoning behind the semantics of Passables comes from the E experience.

• Records aka structs -- Their property names can only be strings. However, it is conceivable that the scheme mapping will turn property names into scheme (records? structs? dictionaries?) with corresponding interned symbols as keys. This should not affect the protocol or its abstract meaning, where these are strings. Because these strings can be sorted, records have the same conditional equality and ordering properties as arrays.

Gets me into the mind of tagged tuples with named positions in them.

I don't know what you're referring to. Do you have a particular programming language in miind?

Thought I saw this feature in a scanned in manual for Oberon.
Basically a syntactic sugar for mapping names, chosen at datatype declaration, to positions in a tagged tuple (the tag was implict part of each such tuple and could be checked at runtime)

• Error -- yeah, probably what Kris said. That trace identifier has complex implications on equality and canonicalization, but the need seems overwhelming. I'm not at all happy with what I wrote down on that page.

What about Errors that contain sealed and possibly access-level-tag redacted callstack trace?

We've been going in a Causeway-like direction, where each site logs the stack traces of sent errors locally without sending the stack traces. The trace identifier is logged on the sending side associated with that stack trace, and is also sent to the receiving side. The receiving side annotates the unserialized error with that trace identifier to communicate it to the local logging system there without making it available from the error object. Then, to trace back across sites each involved site proprietor has to decide what locally logged error stacks to reveal, so these can be stitched together.

Our current state has taken some steps towards this, but is currently a mess (my fault).

Neat to here the spirit of Causeway lives on (and may perhaps be resurected at later date)

• null -- as Kris warned everyone about, I'm going to insist on two bottom values, which we can call null and undefined. The problem is that Agoric's code, and most sensible JS code, almost always uses only undefined. But JSON can only do null and not undefined. Thus, JS code that gets data from JSON will still be handling nulls. I just don't think there's anything else we can do without creating a bug farm for JS programmers. I'm really sorry about this one.

One explanation on the difference of null and undefined is that the former is an marker for empty while the latter is a marker for missing altogether.

I don't know what this would mean in practice. Concretely, where would I use null and where would I use undefined. Frankly, were it not for JSON I would be tempted to drop null altogether. But we're not in that world, so we gotta have both.

What else is mentioned on that page but absent from Kris' table?

• Convenient names for non-leaf groups, such as LeafData for all the scalars. This only becomes interesting when these labels represent semantics, like PassableKey which can include, for example, arrays of numbers but not promises.

So, basically compound pure data datums that could have been shunted out to named-by-content-hash distributed storage.

Which Passable type are you referring to?

Pretty much anything that passes unscathed through the E guards, :Data, :Transparent, and :Selfless and could be stuffed into a json blob.

• symbol -- I am open both to doing this or to not doing this, depending what the tradeoffs look like as we talk this out. What we're all agreed on is that if symbols are admitted as data distinct from strings, then each symbol is uniquely associated with a string. Two symbols are equal iff their associated strings are equal. This decision is independent of whether

  • records property name strings correspond, for a given language (scheme) to symbol-named properties, for the symbols named by those strings.
  • whether method name strings (which are outside our table for now anyway) are similarly taken by a given language to name symbols rather than strings

• LeafCaps, i.e., remotables and promises.

• Sets and Maps, which Agoric does not implement yet. So their inclusion is more speculative on my part. But they're important to examine because the equality of their keys is what makes all this a non-trivial exercise.

Oi! You started this by pining for Symbol.asyncIterator for notifier/subscription purposes! But I do not mind that symbols are supported though.

Turns out, we are currently still undecided on symbols.

We sould decide otherwise they would get symbolic of other yet to be found issues :3