Epic: Evolution of Zoe Installations

[kriskowal]

This issue tracks the user-facing changes expected in the sequence of planned changes to the Zoe contract installation API.

Status

ZCF install currently supports three bundle formats and the default format is nestedEvaluate.

getExports A single script format suitable for bootstrapping off of a hand-crafted script or a bundle. bundleSource uses Rollup.
nestedEvaluate A script with modules embedded as strings and the code needed to initialize and link those modules.
endoZipBase64 A Base-64 encoded Zip file containing UTF-8 encoded modules and a “Comaprtment Map”. The runtime for this creates a Compartment per package. #1983

Switch default bundle format to endoZipBase64

This change will break any program that presumed that a bundle was in nestedEvaluate or getExports format and manipulated the bundle’s source property. This pattern existed in Zoe tests as a stopgap for verifying the integrity of a bundle using source spot-checks.

• Use Endo archives between bundleSource and importBundle #2684
• refactor: Switch default archive format to endoZipBase64 #3273

Bundle integrity consistent-hash

In a future change, Zoe’s Installation type acquires a new method named getHash or getSha512. This allows the holder to check the integrity of an installation by comparing the hash to the hash of a known artifact.

• Bundle hashing #3859

Add new opaque-hash bundle format

This change would add a new bundle format that resembles {moduleFormat: 'SHA-512', sha512: 'xxxxx'}. The deployment mechanism for a contract would create a different kind of bundle, submit the bundle to the chain out-of-band (without enqueuing a message), and Zoe would be obliged to retrieve the bundle from this other channel. The ZCF install method would accept this format.

• controller.installBundle() #3269

This bundle format would allow for a sequence of optimizations between deploy and install that might be invisible to end-users. The initial version would just send the bundle as a single blob. Subsequent versions would be able to decompose contracts into individual files, create a bill of materials, submit the bill to the chain, receive a list of materials absent on the chain, and then only send the missing bits. A subsequent optimization might be able to pre-compile XSnap modules and allow for code sharing between XSnap instances with a memmap per Static Module Record.

Zoe consolidates bundle formats

This would be deprecation and subsequent breaking-changes for Zoe:

• ZCF install would withdraw support for all bundle formats except the opaque-hash. This would eliminate the possibility of sending large payloads for contract installation over the message list.
• ZCF Installation could withdraw support for getBundle() entirely since it would become redundant with getHash().

[dckc]

I see discussion of...

jq -r .endoZipBase64 bundle.json | base64 -d > bundle.zip
unzip bundle.zip

It would be nice to have a JS API for that. But I guess I can't think of a requirement.

[kriskowal]

Between obtaining the bundle and writing the files to disk, the API is like:

import { ZipReader } from '@endo/zip';
import { decodeBase64 } from '@endo/base64';
const data = decodeBase64(bundle.endoZipBase64);
const zip = new ZipReader(data);
const { files } = zip; // files is a Map of name to Uint8Array

I think this is less useful than the shell analog, but less useful than a new agoric-cli subcommand that gets the bundle and extracts it locally to a specified directory. @katelynsills / @michaelfig What would the command need to take as arguments to do that today? I think in a later phase, it would just take a bundle hash and reach out to the solo or the chain for the corresponding data, possibly forgoing the intermediate zip file entirely someday.

[dckc]

Does this get us to sharing Zoe/ERTP libraries as in #2391 ? Is that issue still relevant?

How does the story above relate to the zygote stuff? #2268

[dckc]

When doing forensics / debugging on slog data, with nestedEvaluate, we had this handy kludge for getting a name from a bundle:

EPRE = re.compile(r'const entrypoint = "([^"]+)"')

-- /~https://github.com/Agoric/agoric-sdk/blob/master/packages/SwingSet/misc-tools/vat-map.py#L26

I'm struggling to figure out how to do likewise with endo zip archives. Meanwhile...

Does it make sense for Zoe installations to have a name or description? Or is that too much of a phishing risk?

[kriskowal]

You can get the name from:

• decode base 64
• decode zip
• extract compartment-map.json
• parse the JSON
• read the name property

Getting the hash is a similar process. We can build a utility for the same.

[katelynsills]

Does it make sense for Zoe installations to have a name or description? Or is that too much of a phishing risk?

I think you're right to identify phishing as a problem. We definitely don't want names or descriptions. For instance, for ERTP, we have an "allegedName" that is only used in debugging and never used anywhere else, including UI. The petnames of users should cover the name.

[dckc]

One of our fans noticed installation.bundle.source in E(zoe).getInstallation(invitation).

Is that documentation now out of date?

Do we plan to restore the ability to get source code?

you can get the original source code of any contract as a built-in feature of the SDK. In Ethereum someone tells you a hardcoded address, then you ask an explorer for the source, AFAIK. In Agoric, you can access it in the base API.
-- https://twitter.com/least_nathan/status/1451273772577685504

I suppose that means you can even do automated source code analysis within contracts?

Scifi: Imagine something like "my contract will only send withdrawals to destinations with code that type checks and meets these other static analysis guarantees".
-- https://twitter.com/least_nathan/status/1451273774418980866

[kriskowal]

This is indeed stale documentation. There is currently a way to get at source. Eventually, Zoe will only have direct access to the hash.

…

On Mon, Oct 25, 2021 at 11:40 AM Dan Connolly ***@***.***> wrote: One of our fans noticed installation.bundle.source in E(zoe).getInstallation(invitation) <https://agoric.com/documentation/zoe/api/zoe.html#e-zoe-getinstallation-invitation> . Is that documentation now out of date? Do we plan to restore the ability to get source code? you can get the original source code of any contract as a built-in feature of the SDK. In Ethereum someone tells you a hardcoded address, then you ask an explorer for the source, AFAIK. In Agoric, you can access it in the base API. -- https://twitter.com/least_nathan/status/1451273772577685504 I suppose that means you can even do automated source code analysis within contracts? Scifi: Imagine something like "my contract will only send withdrawals to destinations with code that type checks and meets these other static analysis guarantees". -- https://twitter.com/least_nathan/status/1451273774418980866 — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#3871 (comment)>, or unsubscribe </~https://github.com/notifications/unsubscribe-auth/AAAOXBQ24AXCONACSPYQNMDUIWQANANCNFSM5EPJII6Q> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[katelynsills]

Turned stale documentation into an issue: Agoric/documentation#567

[Tartuffo]

Main body of work for this, modulo consolidation, is done. @kriskowal will create an issue for the work that remains for the MN-2 / 3 timeframe.