Automate build and release to PyPI

[hugovk]

Currently the release process has many steps, including preparing things in this repo, and then starting a build in /~https://github.com/python-pillow/pillow-wheels, before downloading the sdist + wheel artifacts to disk, and manually uploading them to PyPI using twine.

I'd like us to move towards more automated releases, including using the new Trusted Publishers to upload wheels directly from the CI to PyPI, which is more secure:

• https://blog.pypi.org/posts/2023-04-20-introducing-trusted-publishers/

We can do this in a phased manner, not all of these things would need to be done at once, to ensure we can still make releases with just some of them completed.

We could potentially deal with the Travis CI wheels later, and tackle GitHub Actions first as they're a bigger win and we don't know when Travis CI will go away.

Big picture:

• Prepare release
• Push a tag
• The CI builds release artifacts
• The CI uploads artifacts to PyPI

I'd suggest something along these lines:

1. Move Linux and macOS /~https://github.com/python-pillow/pillow-wheels to this repo /~https://github.com/python-pillow/Pillow. We already have Windows wheels build here.

   • We'll need to add paths: / paths-ignore to workflows so we only trigger the main test workflows for "normal" code changes, and only trigger the wheel workflow for things like release and modifying its config
   • Upload wheels as GitHub artifacts in this repo instead

2. Add automation to download artifacts and upload them to PyPI via Trusted Publishing

   • This may need refactoring to move all the wheel building to the same workflow (but separate jobs), and then a final job that runs when the wheels have all been built, that builds sdist, checks and uploads them all.
   • See /~https://github.com/ultrajson/ultrajson/blob/main/.github/workflows/deploy.yml and /~https://github.com/ultrajson/ultrajson/actions/workflows/deploy.yml for examples.
   • We also have https://test.pypi.org/project/Pillow/ which we can use to verify the upload machinery is working smoothly before release. For example, we can upload for non-tagged wheel builds.

Thoughts?

[wiredfool]

This sounds like a great step.

[aclark4life]

Love it!!

[radarhere]

I've created #7418 for 1.

[hugovk]

I've opened #7544 to create the sdist during the release build.

---

Next, I'd like us to move the Windows wheel building from test-windows.yml into the same wheels.yml workflow.

So we get something like this:

And the Windows wheels should be part of the same dist.zip artifact containing the Linux and macOS wheels, and the sdist.

I think we also probably want to use a reusable workflow, similar to wheels-linux.yml and wheels-macos.yml.

We don't need to build the wheels anymore in test-windows.yml. If there's a lot of common steps needed for testing (in test-windows.yml) and building wheels, a reusable workflow could be useful here.

This doesn't all need to be done in one go, incremental steps are fine, as long as they're atomic changes and we're still able to make releases during any intermediate step.

Any thoughts on how to approach this? cc @nulano

---

After all this is done, we have all* the release files created in one place on the CI, and do the upload to PyPI! But one step at a time :)

(* All the GHA files, not Travis CI, which will still need manual upload, but they should be ready before all the automation.)

[nulano]

I've managed to use cibuildwheel to build wheels locally, I should be able to make a PR on top of #7552 for Windows wheels once it is merged.
Edit: I've got a WIP branch ready, including ARM64 wheels (although I have no way to test them!).
Edit2: I've created PR #7580.