Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cirrus: Automate releasing of tested binaries #3106

Merged
merged 1 commit into from
Jul 10, 2019

Conversation

cevich
Copy link
Member

@cevich cevich commented May 10, 2019

It's desirable to make archives available of builds containing actual
tested content. While not official distro-releases, these will enable
third-party testing, experimentation, and development for both branches
(e.g. "master") and pull requests (e.g. "pr3106").

  • Add a Makefile targets for archiving both regular podman binaries
    and the remote-client. Encode release metadata within these
    archives so that their exact source can be identified.

  • Fix bug with cross-compiling remote clients for the Windows and Darwin
    platforms.

  • Add unit-testing of cross-compiles for Windows and Darwin platforms.

  • A few small CI-script typo-fixes

  • Add a script which operates in two modes:

    1. Call Makefile targets which produce release archives.
      Upload the archive to Cirrus-CI's built-in caching system
      using reproducible cache keys.

    2. Utilize reproduced cache keys to attempt download of cache
      from each tasks. When successful, parse the file's
      release metadata, using it to name the archive file. Upload
      all recovered archives to a publicly accessible storage bucket
      for future reference.

  • Update the main testing task to call the script in mode Initial checkin #1 for
    all primary platforms.

  • Add a new $SPECIALMODE task to call the script in mode Initial checkin #1 for
    Windows and Darwin targets.

  • Add a new 'release' task to the CI system, dependent upon all other
    tasks. This new tasks executes the script in mode libpod create and run #2.

  • Update CI docuemntation

@cevich
Copy link
Member Author

cevich commented May 10, 2019

Note: I have no idea if this will actually work, but I need a PR to see if it does. This is that.

@cevich cevich force-pushed the cirrus_release branch 2 times, most recently from b0a8796 to 81cb012 Compare May 10, 2019 23:42
@cevich
Copy link
Member Author

cevich commented May 10, 2019

design note: I'm generating the canonical name in the Makefile to enable developers to retain local tarball and not have them clash with each-other as content changes. However, that requires renaming of the tarball when running under CI, and this may be an unnecessary complication. Unf. there are some env. vars. only available under CI, and I'm a bit loathe for the Makefile to depend upon them.

Copy link
Member

@edsantiago edsantiago left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can't possibly work; I'll take a fresh look once these issues are taken care of

contrib/cirrus/release_cache.sh Outdated Show resolved Hide resolved
contrib/cirrus/cache_release.sh Outdated Show resolved Hide resolved
contrib/cirrus/cache_release.sh Outdated Show resolved Hide resolved
contrib/cirrus/cache_release.sh Outdated Show resolved Hide resolved
contrib/cirrus/cache_release.sh Outdated Show resolved Hide resolved
contrib/cirrus/cache_release.sh Outdated Show resolved Hide resolved
@cevich cevich force-pushed the cirrus_release branch 2 times, most recently from 427bdeb to 1e617e0 Compare May 13, 2019 19:17
@rh-atomic-bot
Copy link
Collaborator

☔ The latest upstream changes (presumably #3127) made this pull request unmergeable. Please resolve the merge conflicts.

@openshift-ci-robot openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 16, 2019
@openshift-ci-robot openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 20, 2019
@cevich cevich changed the title Cirrus: Automate releasing of tested tarballs WIP: Cirrus: Automate releasing of tested tarballs May 20, 2019
@openshift-ci-robot openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 20, 2019
@cevich cevich force-pushed the cirrus_release branch 8 times, most recently from 3d0533b to 96787f9 Compare May 21, 2019 18:44
@cevich cevich force-pushed the cirrus_release branch 2 times, most recently from 38e2352 to 51c24ae Compare June 13, 2019 19:39
@cevich cevich changed the title Cirrus: Automate releasing of tested tarballs Cirrus: Automate releasing of tested binaries Jun 13, 2019
@cevich
Copy link
Member Author

cevich commented Jun 14, 2019

@edsantiago mind taking a peek at this now that it's working? Also @baude or @jwhonce PTAL since y'all are most heavily involved in the remote-client efforts.

@rh-atomic-bot
Copy link
Collaborator

☔ The latest upstream changes (presumably #3308) made this pull request unmergeable. Please resolve the merge conflicts.

@openshift-ci-robot openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 17, 2019
@openshift-ci-robot openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 25, 2019
@cevich cevich force-pushed the cirrus_release branch 2 times, most recently from a8e0c42 to 28fc023 Compare June 25, 2019 17:57
@cevich cevich requested a review from edsantiago June 27, 2019 15:49
Copy link
Member

@edsantiago edsantiago left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried very hard to understand all this; I can't. I have, as best I can, confirmed that you've addressed my earlier review comments. I've skimmed the rest of the code, mostly feeling lost. My two comments below may therefore be nonsensical. I'll just approve and hope for the best.

contrib/cirrus/cache_release_archive.sh Show resolved Hide resolved
contrib/cirrus/cache_release_archive.sh Outdated Show resolved Hide resolved
It's desirable to make archives available of builds containing actual
tested content.  While not official distro-releases, these will enable
third-party testing, experimentation, and development for both branches
(e.g. "master") and pull requests (e.g. "pr3106").

* Add a Makefile targets for archiving both regular podman binaries
  and the remote-client.  Encode release metadata within these
  archives so that their exact source can be identified.

* Fix bug with cross-compiling remote clients for the Windows and Darwin
  platforms.

* Add unit-testing of cross-compiles for Windows and Darwin platforms.

* A few small CI-script typo-fixes

* Add a script which operates in two modes:

  1. Call Makefile targets which produce release archives.
     Upload the archive to Cirrus-CI's built-in caching system
     using reproducible cache keys.

  2. Utilize reproduced cache keys to attempt download of cache
     from each tasks.  When successful, parse the file's
     release metadata, using it to name the archive file.  Upload
     all recovered archives to a publicly accessible storage bucket
     for future reference.

* Update the main testing task to call the script in mode #1 for
  all primary platforms.

* Add a new `$SPECIALMODE` task to call the script in mode #1 for
  Windows and Darwin targets.

* Add a new 'release' task to the CI system, dependent upon all other
  tasks.  This new tasks executes the script in mode #2.

* Update CI documentation

Signed-off-by: Chris Evich <cevich@redhat.com>
@cevich
Copy link
Member Author

cevich commented Jul 3, 2019

mostly feeling lost.

aww, sorry 😢 Probably a sign that better (any) documentation is needed. Hmmm, or maybe a blog post (advertising availability of automatic binary-builds for testing). I've added both to my TODO list. Thanks for reviewing what you were able, it's still helpful 👍

@cevich
Copy link
Member Author

cevich commented Jul 3, 2019

/test images

2 similar comments
@edsantiago
Copy link
Member

/test images

@cevich
Copy link
Member Author

cevich commented Jul 5, 2019

/test images

@cevich
Copy link
Member Author

cevich commented Jul 8, 2019

@baude this is good-to-go, PTAL (you originally requested it)

@rhatdan
Copy link
Member

rhatdan commented Jul 8, 2019

/approve

@openshift-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cevich, edsantiago, rhatdan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 8, 2019
@baude
Copy link
Member

baude commented Jul 10, 2019

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 10, 2019
@openshift-merge-robot openshift-merge-robot merged commit 81e722d into containers:master Jul 10, 2019
@cevich cevich deleted the cirrus_release branch June 30, 2021 18:11
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 22, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants