Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GHA workflow to validate pants BUILD files #5732

Merged
merged 3 commits into from
Sep 20, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions .github/workflows/pants.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
name: Validate Pants Metadata

on:
# temporarily only allow manual runs until we have BUILD files and lockfiles
workflow_dispatch:
#push:
# branches:
# # only on merges to master branch
# - master
# # and version branches, which only include minor versions (eg: v3.4)
# - v[0-9]+.[0-9]+
# tags:
# # also version tags, which include bugfix releases (eg: v3.4.0)
# - v[0-9]+.[0-9]+.[0-9]+
#pull_request:
# type: [opened, reopened, edited]
# branches:
# # Only for PRs targeting those branches
# - master
# - v[0-9]+.[0-9]+
Comment on lines +5 to +21
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will uncomment this and remove workflow_dispatch when we add the BUILD files.


jobs:
pants-tailor:
name: Make sure pants BUILD files are up-to-date
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2
with:
# a test uses a submodule, and pants needs access to it to calculate deps.
submodules: 'true'

- name: Initialize Pants and its GHA caches
uses: pantsbuild/actions/init-pants@c0ce05ee4ba288bb2a729a2b77294e9cb6ab66f7
# This action adds an env var to make pants use both pants.ci.toml & pants.toml.
# This action also creates 3 GHA caches (1 is optional).
# - `pants-setup` has the bootsrapped pants install
# - `pants-named-caches` has pip/wheel and PEX caches
# - `pants-lmdb-store` has the fine-grained process cache.
Comment on lines +34 to +40
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/~https://github.com/pantsbuild/actions/tree/main/init-pants

The action handles installing pants, adding the PANTS_CONFIG_FILES env var (to use our new pants.ci.toml file), and it manages the GHA caches for us. Using this action means that we don't have to repeat those steps in every workflow that uses pants.

One of these GHA cahces will include the pip cache, so we will also be able to simplify our cache usage in other workflows once we are using pants everywhere.

# If we ever use a remote cache, then we can drop this.
# Otherwise, we may need an additional workflow or job to delete old caches
# if they are not expiring fast enough, and we hit the GHA 10GB per repo max.
with:
# To ignore a bad cache, bump the cache* integer.
gha-cache-key: cache0-BUILD
# This hash should include all of our lockfiles so that the pip/pex caches
# get invalidated on any transitive dependency update.
named-caches-hash: ${{ hashFiles('requirements.txt' }}
# enable the optional lmdb_store cache since we're not using remote caching.
cache-lmdb-store: 'true'
Comment on lines +50 to +51
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we experience any issues with the 10GB GHA cache limit (a per-repo limit), we may need to add a workflow that periodically clears out older unused caches - Github expires unused caches after 7 days, but we might want a shorter timespan, as this cache is a per-commit cache. We can also bump the gha-cache-key input var if the cache gets too big to ignore it and start fresh.


- name: Check BUILD files
run: |
./pants tailor --check update-build-files --check ::

- name: Upload pants log
uses: actions/upload-artifact@v2
with:
name: pants-log-py${{ matrix.python-version }}
path: .pants.d/pants.log
if: always() # We want the log even on failures.
Comment on lines +57 to +62
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my experience, this log is often nearly empty, but it can be very useful when debugging CI issues. Adding this step is recommended here: https://www.pantsbuild.org/docs/using-pants-in-ci#tip-store-pants-logs-as-artifacts

We will want to add this step in each workflow that runs pants.

2 changes: 1 addition & 1 deletion CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ Added

* Begin introducing `pants <https://www.pantsbuild.org/docs>`_ to improve DX (Developer Experience)
working on StackStorm, improve our security posture, and improve CI reliability thanks in part
to pants' use of PEX lockfiles. This is not a user-facing addition. #5713 #5724 #5726 #5725
to pants' use of PEX lockfiles. This is not a user-facing addition. #5713 #5724 #5726 #5725 #5732
Contributed by @cognifloyd

Changed
Expand Down
13 changes: 13 additions & 0 deletions pants.ci.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# This config is for CI. It extends the config in pants.toml.
# See https://www.pantsbuild.org/docs/using-pants-in-ci

[GLOBAL]
# Colors often work in CI, but the shell is usually not a TTY so Pants
# doesn't attempt to use them by default.
colors = true
Comment on lines +4 to +7
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


[stats]
# "print metrics of your cache's performance at the end of the run,
# including the number of cache hits and the total time saved thanks
# to caching"
log = true
Comment on lines +9 to +13
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommended under "Tip: check cache performance with [stats].log" at the end of the Directories to Cache section of the docs.