Fix: Create a static copy of signatures as part of verification. #2287
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🐛 As part of verifying signatures and annotations create copies of what we are downloading to avoid later calls to `Payload()` redownloading the signature blob. Examining trace data for policy-controller downstream, I observed that the same signature blob data was being fetched three times as part of verification. I believe for attestations this may be even worse because we have an additional call to `Payload()` where we parse out the contents of the attestation's predicate for policy evaluation. This change eagerly fetches the signature metadata and stores it in a clone via `static.Copy`. This clone is what we verify, and if it fails verification is it immediately discarded. However, if it passes verification this clone is returned as one of the "verified signatures" that the downstream logic can now access without refetching data from the registry. /kind bug Signed-off-by: Matt Moore <mattmoor@chainguard.dev>
Codecov Report
@@ Coverage Diff @@
## main #2287 +/- ##
==========================================
- Coverage 29.02% 28.82% -0.21%
==========================================
Files 131 131
Lines 7872 7928 +56
==========================================
Hits 2285 2285
- Misses 5275 5331 +56
Partials 312 312
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
mattmoor
changed the title
dlorenc
approved these changes
Sep 27, 2022
jeff-mccoy
referenced
this pull request
in zarf-dev/zarf
Oct 7, 2022
[](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/sigstore/cosign](https://togithub.com/sigstore/cosign) | require | minor | `v1.12.1` -> `v1.13.0` | --- ### Release Notes <details> <summary>sigstore/cosign</summary> ### [`v1.13.0`](https://togithub.com/sigstore/cosign/blob/HEAD/CHANGELOG.md#v1130) [Compare Source](https://togithub.com/sigstore/cosign/compare/v1.12.1...v1.13.0) > # Highlights > > - For users who have deployed a private instance of Fulcio release v0.6.x and issue certificates with the Username identity, you will need to upgrade to use this version." #### Enhancements - Add support for Fulcio username identity in SAN ([/~https://github.com/sigstore/cosign/pull/2291](https://togithub.com/sigstore/cosign/pull/2291)) - Data race in FetchSignaturesForReference ([/~https://github.com/sigstore/cosign/pull/2283](https://togithub.com/sigstore/cosign/pull/2283)) - Check error on chain verification failure ([/~https://github.com/sigstore/cosign/pull/2284](https://togithub.com/sigstore/cosign/pull/2284)) - feat: improve the verification message ([/~https://github.com/sigstore/cosign/pull/2268](https://togithub.com/sigstore/cosign/pull/2268)) - feat: use stdin as an input for predicate ([/~https://github.com/sigstore/cosign/pull/2269](https://togithub.com/sigstore/cosign/pull/2269)) #### Bug Fixes - fix: make tlog entry lookups for online verification shard-aware ([/~https://github.com/sigstore/cosign/pull/2297](https://togithub.com/sigstore/cosign/pull/2297)) - Fix: Create a static copy of signatures as part of verification. ([/~https://github.com/sigstore/cosign/pull/2287](https://togithub.com/sigstore/cosign/pull/2287)) - Fix: Remove an extra registry request from verification path. ([/~https://github.com/sigstore/cosign/pull/2285](https://togithub.com/sigstore/cosign/pull/2285)) - fix pivtool generate key touch policy ([/~https://github.com/sigstore/cosign/pull/2282](https://togithub.com/sigstore/cosign/pull/2282)) #### Others - use scaffolding 0.4.8 for tests. ([/~https://github.com/sigstore/cosign/pull/2280](https://togithub.com/sigstore/cosign/pull/2280)) #### Contributors - Asra Ali ([@​asraa](https://togithub.com/asraa)) - Batuhan Apaydın ([@​developer-guy](https://togithub.com/developer-guy)) - Carlos Tadeu Panato Junior ([@​cpanato](https://togithub.com/cpanato)) - Hayden Blauzvern ([@​haydentherapper](https://togithub.com/haydentherapper)) - Matt Moore ([@​mattmoor](https://togithub.com/mattmoor)) - Ross Tannenbaum ([@​RTann](https://togithub.com/RTann)) - Ville Aikas ([@​vaikas](https://togithub.com/vaikas)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, click this checkbox. --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/defenseunicorns/zarf). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzMi4yMjIuMyIsInVwZGF0ZWRJblZlciI6IjMyLjIyMi4zIn0=--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
](https://renovatebot.com){kind=link}
Noxsios
referenced
this pull request
in zarf-dev/zarf
Mar 8, 2023
[](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/sigstore/cosign](https://togithub.com/sigstore/cosign) | require | minor | `v1.12.1` -> `v1.13.0` | --- ### Release Notes <details> <summary>sigstore/cosign</summary> ### [`v1.13.0`](https://togithub.com/sigstore/cosign/blob/HEAD/CHANGELOG.md#v1130) [Compare Source](https://togithub.com/sigstore/cosign/compare/v1.12.1...v1.13.0) > # Highlights > > - For users who have deployed a private instance of Fulcio release v0.6.x and issue certificates with the Username identity, you will need to upgrade to use this version." #### Enhancements - Add support for Fulcio username identity in SAN ([/~https://github.com/sigstore/cosign/pull/2291](https://togithub.com/sigstore/cosign/pull/2291)) - Data race in FetchSignaturesForReference ([/~https://github.com/sigstore/cosign/pull/2283](https://togithub.com/sigstore/cosign/pull/2283)) - Check error on chain verification failure ([/~https://github.com/sigstore/cosign/pull/2284](https://togithub.com/sigstore/cosign/pull/2284)) - feat: improve the verification message ([/~https://github.com/sigstore/cosign/pull/2268](https://togithub.com/sigstore/cosign/pull/2268)) - feat: use stdin as an input for predicate ([/~https://github.com/sigstore/cosign/pull/2269](https://togithub.com/sigstore/cosign/pull/2269)) #### Bug Fixes - fix: make tlog entry lookups for online verification shard-aware ([/~https://github.com/sigstore/cosign/pull/2297](https://togithub.com/sigstore/cosign/pull/2297)) - Fix: Create a static copy of signatures as part of verification. ([/~https://github.com/sigstore/cosign/pull/2287](https://togithub.com/sigstore/cosign/pull/2287)) - Fix: Remove an extra registry request from verification path. ([/~https://github.com/sigstore/cosign/pull/2285](https://togithub.com/sigstore/cosign/pull/2285)) - fix pivtool generate key touch policy ([/~https://github.com/sigstore/cosign/pull/2282](https://togithub.com/sigstore/cosign/pull/2282)) #### Others - use scaffolding 0.4.8 for tests. ([/~https://github.com/sigstore/cosign/pull/2280](https://togithub.com/sigstore/cosign/pull/2280)) #### Contributors - Asra Ali ([@​asraa](https://togithub.com/asraa)) - Batuhan Apaydın ([@​developer-guy](https://togithub.com/developer-guy)) - Carlos Tadeu Panato Junior ([@​cpanato](https://togithub.com/cpanato)) - Hayden Blauzvern ([@​haydentherapper](https://togithub.com/haydentherapper)) - Matt Moore ([@​mattmoor](https://togithub.com/mattmoor)) - Ross Tannenbaum ([@​RTann](https://togithub.com/RTann)) - Ville Aikas ([@​vaikas](https://togithub.com/vaikas)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, click this checkbox. --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/defenseunicorns/zarf). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzMi4yMjIuMyIsInVwZGF0ZWRJblZlciI6IjMyLjIyMi4zIn0=--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🐛 As part of verifying signatures and annotations create copies of what we are downloading to avoid later calls to
Payload()redownloading the signature blob.Examining trace data for policy-controller downstream, I observed that the same signature blob data was being fetched three times as part of verification. I believe for attestations this may be even worse because we have an additional call to
Payload()where we parse out the contents of the attestation's predicate for policy evaluation.This change eagerly fetches the signature metadata and stores it in a clone via
static.Copy. This clone is what we verify, and if it fails verification is it immediately discarded. However, if it passes verification this clone is returned as one of the "verified signatures" that the downstream logic can now access without refetching data from the registry.UPDATE: I have confirmed downstream by manually vendoring this in that the redundant blob fetches go away and the admission webhook requests are noticeably faster. It isn't quite 3x because of other overheads, but it is pretty considerable.
/kind bug
Signed-off-by: Matt Moore mattmoor@chainguard.dev
Release Note
Speed up some of the verification methods by eliminating redundant fetches of the signature blob data from the registry using a temporary in-memory copy.
Documentation