Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pants: Add pack_metadata targets to BUILD files #5871

Merged
merged 14 commits into from
Jan 27, 2023
Merged

Pants: Add pack_metadata targets to BUILD files #5871

Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
e5d4f18
pants: add pack_metadata targets
cognifloyd May 31, 2021
f18e1a0
add packs_metadata and related targets
cognifloyd May 31, 2021
8aebc18
Add pack_metadata dependencies
cognifloyd May 31, 2021
d995003
Add pack_metadata for more fixtures
cognifloyd Jun 1, 2021
e1614fc
Allow pants to detect packs fixture usage
cognifloyd Jun 1, 2021
e2bef63
Cleanup fixtures BUILD files
cognifloyd Jun 2, 2021
0f2c7cc
add BUILD metadata for st2tests/testpacks
cognifloyd Jun 1, 2021
adf97fa
depend on pack_metadata in st2tests.fixtures.packs.core.fixture
cognifloyd Jun 4, 2021
521397b
Add pack_metadata targets in st2reactor pack fixtures
cognifloyd Jun 8, 2021
d864817
Enable pants-plugins/pack_metadata
cognifloyd Jan 23, 2023
26132d1
update changelog entry
cognifloyd Jan 23, 2023
f4062f7
disable spurious pack_metadata target where there is no metadata
cognifloyd Jan 23, 2023
e18e3ae
add pack_metadata to contrib/debug with `./pants tailor ::`
cognifloyd Jan 23, 2023
54012ec
oops. executable bit should not be set on BUILD files
cognifloyd Jan 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Added
working on StackStorm, improve our security posture, and improve CI reliability thanks in part
to pants' use of PEX lockfiles. This is not a user-facing addition.
#5778 #5789 #5817 #5795 #5830 #5833 #5834 #5841 #5840 #5838 #5842 #5837 #5849 #5850
#5846 #5853 #5848 #5847 #5858 #5857 #5860 #5868
#5846 #5853 #5848 #5847 #5858 #5857 #5860 #5868 #5871
Contributed by @cognifloyd

* Added a joint index to solve the problem of slow mongo queries for scheduled executions. #5805
Expand Down
3 changes: 3 additions & 0 deletions contrib/chatops/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
pack_metadata(
name="metadata",
)
8 changes: 7 additions & 1 deletion contrib/core/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
python_sources()
pack_metadata(
name="metadata",
)

python_requirements(
name="reqs",
Expand All @@ -7,3 +9,7 @@ python_requirements(
# /~https://github.com/pantsbuild/pants/pull/17390
module_mapping={"mail-parser": ["mailparser"]},
)

python_sources(
dependencies=[":metadata"],
)
3 changes: 3 additions & 0 deletions contrib/debug/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
pack_metadata(
name="metadata",
)
3 changes: 3 additions & 0 deletions contrib/default/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
pack_metadata(
name="metadata",
)
4 changes: 4 additions & 0 deletions contrib/examples/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
pack_metadata(
name="metadata",
)

# Using `python_requirements()` here results in:
# ">1 target exports this module, so it is ambiguous which to use"
# This error refers to the "requests" module.
Expand Down
3 changes: 3 additions & 0 deletions contrib/hello_st2/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
pack_metadata(
name="metadata",
)
4 changes: 4 additions & 0 deletions contrib/linux/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
pack_metadata(
name="metadata",
)

python_requirements(
name="reqs",
)
3 changes: 3 additions & 0 deletions contrib/packs/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
pack_metadata(
name="metadata",
)
9 changes: 8 additions & 1 deletion contrib/packs/tests/fixtures/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,8 @@
python_sources()
python_sources(
dependencies=[
"./stackstorm-test:metadata",
"./stackstorm-test2:metadata",
"./stackstorm-test3:metadata",
"./stackstorm-test4:metadata",
],
)
3 changes: 3 additions & 0 deletions contrib/packs/tests/fixtures/stackstorm-test/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
pack_metadata(
name="metadata",
)
3 changes: 3 additions & 0 deletions contrib/packs/tests/fixtures/stackstorm-test2/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
pack_metadata(
name="metadata",
)
3 changes: 3 additions & 0 deletions contrib/packs/tests/fixtures/stackstorm-test3/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
pack_metadata(
name="metadata",
)
3 changes: 3 additions & 0 deletions contrib/packs/tests/fixtures/stackstorm-test4/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
pack_metadata(
name="metadata",
)
2 changes: 1 addition & 1 deletion pants.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ backend_packages = [
# internal plugins in pants-plugins/
"pants.backend.plugin_development",
"api_spec",
#"pack_metadata",
"pack_metadata",
"sample_conf",
"schemas",
]
Expand Down
8 changes: 7 additions & 1 deletion st2reactor/tests/fixtures/packs/pack_with_rules/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2reactor/tests/fixtures/packs/pack_with_sensor/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/aliases/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Slightly confused. In the dummy_pack_23 we don't include the pack_metadata as there is no pack.yaml in that pack.

For things like this BUILD file and others at this level in the hierarchy, why do they have the pack_metatdata?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pack_metadata includes more than just pack.yaml.

st2/pants-plugins/pack_metadata/target_types.py

Lines 27 to 40 in a73225c

class PackMetadataSourcesField(ResourcesGeneratingSourcesField):
required = False
default = (
# metadata does not include any python, shell, or other sources.
"pack.yaml",
"config.schema.yaml",
"*.yaml.example",
"**/*.yaml",
"**/*.yml",
"icon.png", # used in st2web ui
# "requirements*.txt", # including this causes target conflicts
# "README.md",
# "HISTORY.md",
)

It also includes metadata yaml files for resources that go in packs: actions, aliases, sensors, etc.

Several of the directories at this level do not have pack.yaml, but they do have the other metadata yaml files. Examples:

Then the other fixtures do similar things. They all seem to have mongo docs in the form of yaml files (after-all the db effectively just holds all of the metadata files in json format, so going in the reverse makes sense).

We need resources for all of these yaml files. PackMetadata subclasses Resources, so everything in the pack_metadata target also counts as a resource. So, we either need to:

  • add resources(sources=...) targets that encompass all of the yaml files, and make the python fixture.py depend on them
  • add pack_metadata and make the python fixture.py depend on it

It was quick and easy to just treat these as packs using pack_metadata so that's what I did. Would it make more sense to you if we used resources() instead for these?

Copy link
Member Author

@cognifloyd cognifloyd Jan 26, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh. And one more clarification, dummy_pack_23 does not have ANY yaml files, not just pack.yaml. So, including the pack_metadata target there results in warnings because none of the sources globs match anything. All of these other fixtures do have yaml files, so the **/*.yaml glob matches.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fine - thanks for clarification.
My point for not reading the resource correctly / I did go and look and must have not scrolled past the pack example yaml line.

All good!

)
Comment on lines +1 to +3
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be functionally equivalent without using the pack_metadata target. Would you prefer this for these fixtures?

Suggested change
pack_metadata(
name="metadata",
)
resources(
name="metadata",
sources=["**/*.yaml"],
)


python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/backstop/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/descendants/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/generic/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/localrunner_pack/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
3 changes: 1 addition & 2 deletions st2tests/st2tests/fixtures/packs/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@
# instructions on how to checkout the submodules if they are not checked out.
# The test_content_version* targets are dependencies of ./test_content_version_fixture

# pack_metadata_in_git_submodule(
resources(
pack_metadata_in_git_submodule(
name="test_content_version_metadata",
sources=[
"test_content_version/pack.yaml",
Expand Down
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/action_chain_tests/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
5 changes: 5 additions & 0 deletions st2tests/st2tests/fixtures/packs/dummy_pack_1/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[
":metadata",
"st2tests/st2tests/fixtures/packs/configs/dummy_pack_1.yaml",
],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_10/BUILD
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
5 changes: 5 additions & 0 deletions st2tests/st2tests/fixtures/packs/dummy_pack_11/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[
":metadata",
"st2tests/st2tests/fixtures/packs/configs/dummy_pack_11.yaml",
],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_12/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_13/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_14/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_15/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_16/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_17/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_18/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
5 changes: 5 additions & 0 deletions st2tests/st2tests/fixtures/packs/dummy_pack_19/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[
":metadata",
"st2tests/st2tests/fixtures/packs/configs/dummy_pack_19.yaml",
],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_2/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
pack_metadata(
name="metadata",
)

resource(
name="pack_requirements",
source="requirements.txt",
)

python_sources()
python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_20/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_21/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
5 changes: 5 additions & 0 deletions st2tests/st2tests/fixtures/packs/dummy_pack_22/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[
":metadata",
"st2tests/st2tests/fixtures/packs/configs/dummy_pack_22.yaml",
],
)
10 changes: 9 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_23/BUILD
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,9 @@
python_sources()
# There are no metadata files in dummy_pack_23 yet.
# It is used in the pack copy tests.
# pack_metadata(
# name="metadata",
# )

python_sources(
# dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_3/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_4/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
5 changes: 5 additions & 0 deletions st2tests/st2tests/fixtures/packs/dummy_pack_5/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[
":metadata",
"st2tests/st2tests/fixtures/packs/configs/dummy_pack_5.yaml",
],
)
5 changes: 5 additions & 0 deletions st2tests/st2tests/fixtures/packs/dummy_pack_6/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[
":metadata",
"st2tests/st2tests/fixtures/packs/configs/dummy_pack_6.yaml",
],
)
5 changes: 5 additions & 0 deletions st2tests/st2tests/fixtures/packs/dummy_pack_7/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[
":metadata",
"st2tests/st2tests/fixtures/packs/configs/dummy_pack_7.yaml",
],
)
8 changes: 7 additions & 1 deletion st2tests/st2tests/fixtures/packs/dummy_pack_8/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
python_sources()
pack_metadata(
name="metadata",
)

python_sources(
dependencies=[":metadata"],
)
Loading