feat(opa): allow importing custom OPA modules in OPA policies

[nacx]

Description

This change allows importing existing OPA packages to be used in the main OPA policies.

In scenarios where several controls are implemented, it is desirable to have a way to define common functions that can be reused across them, to keep the main policies understandable and maintainable. OPA already provides a nice way of packaging modules, and this PR adds a modules configuration knob to the OPA provider to allow importing existing rego modules.

Related Issue

N/A

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Other (security config, docs update, etc)

Checklist before merging

• Test, docs, are added or updated as needed
• Schema Updates applied
• Contributor Guide Steps followed

[brandtkeller]

very minor schema adjustment (I'll file an issue to integrate schema validation more natively) otherwise this looks good.

[brandtkeller]

Looks great. Appreciate the minor adjustments. Will get an additional set of eyes on this so that we can merge.

[meganwolf0]

Love this! Shout out to the first PR I put up for Lula that was attempting to do this (your implementation is much nicer!)

Pulled this down, tested, all lgtm, the only thing I might call out in the docs (or could probably be added in the code) is that validate.rego should be a protected key, I think it will override the policy if someone happens to specify a module key with the same text.

[nacx]

Thanks!
You're right. I can change the code to add the user modules first, then the validate.rego to make sure it is not messed up. WDYT?
I'm out until 11th, though, I'll do the changes once I'm back.

[meganwolf0]

You're right. I can change the code to add the user modules first, then the validate.rego to make sure it is not messed up.

Sounds great, plus the doc point that zirain pointed out should cover that edge case. Thank you!

Had another thought, and this certainly shouldn't block the current PR but more of a future point of expansion perhaps - if you are executing many validations that are all using the same common module it'll be repeatedly downloaded; might be nice to look at caching that data across validations somehow. Might add as a follow-on issue once this is merged 😄

[brandtkeller]

You're right. I can change the code to add the user modules first, then the validate.rego to make sure it is not messed up.

Sounds great, plus the doc point that zirain pointed out should cover that edge case. Thank you!

Had another thought, and this certainly shouldn't block the current PR but more of a future point of expansion perhaps - if you are executing many validations that are all using the same common module it'll be repeatedly downloaded; might be nice to look at caching that data across validations somehow. Might add as a follow-on issue once this is merged 😄

I'd agree - I think we can look at accomplishing this from multiple levels. One is general file caching within Lula - or more appropriately how do we "package" everything for portability. The other is the module references which are not retained across validations. Feels like a few issues worth scoping separately.

Impending changes required

[nacx]

Hi! I'm back and have addressed the review comments :)
I've opted to return a clear error if the validate.rego is used as a custom module name, to make sure validations don't run with an unexpected configuration leading to inaccurate results.

I agree caching would be great, but I'd like to keep that out of the scope of this PR.

[brandtkeller]

Great catch on the reserved module name @meganwolf0