Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(api domain): add support for Post #801

Merged
merged 8 commits into from
Nov 22, 2024
Merged

Conversation

mildwonkey
Copy link
Contributor

@mildwonkey mildwonkey commented Nov 14, 2024

Description

This PR adds support for Post http calls in the API Domain. A new field, body, is available for a string representation of the body you'd like to send to the request (this is TODO #1: get actual examples of putting json strings in yaml, I don't know that I'm doing it well/correctly and it's definitely not a pleasant UX so far 😂 ). A request can be marked Executable in the spec, at which point Lula will treat the entire domain as executable (ie, requiring user verification before running). I wasn't sure the best way to test this, but I had to add validation.WithAllowExecution to get the e2e test working when the domain was executable, which seemed pretty solid.

UPDATE
I added two fields to the API Domain's standard DomainResources response, so the output now looks like this:

{ 
  "name":  {
    "status": 200,
    "response": < unmarshalled json here>,
    "raw": <json.RawMessage copy of the original API response>
  }
}

The user-facing implication of this is that policies need to reference input.<name>.response instead of just input.<name>. This was the simplest implementation that allowed me to accept any shape API response while still returning a map[string]interface with "status" as one of the return values. It is absolutely possible for this to work without that extra "response" field, but once I added status (and decided I wanted to add the "raw") this made the most sense to me. I didn't update the documentation just yet, waiting to see what y'all think about the new shape first.


Note for reviewers:
I had difficulty writing a test to validate the Executable behavior, because it turns out we swallow errors returned by Validate in the validation-store:
/~https://github.com/defenseunicorns/lula/blob/main/src/pkg/common/validation-store/validation-store.go#L120-L129

To test this theory out, I refactored things so that we actually returned that error and was able to write the test I expected to. Unfortunately, a couple of other tests started failing when I started returning the Validate errors, and I am not certain if all of those failures are expected.

(in the future, not this pr) I think we should change the behavior so that lula actually reports an error when there is an error (the errors we swallow include: invalid schema and no confirmation for execution, both things which I believe should result in errors when lula is running in a pipeline).

Related Issue

Fixes #800

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Other (security config, docs update, etc)

Checklist before merging

@mildwonkey mildwonkey added this to the API Domain Maturation milestone Nov 14, 2024
@mildwonkey mildwonkey marked this pull request as ready for review November 15, 2024 19:12
@mildwonkey mildwonkey requested a review from a team as a code owner November 15, 2024 19:12
src/pkg/domains/api/http_request.go Outdated Show resolved Hide resolved
src/pkg/domains/api/spec.go Show resolved Hide resolved
@mildwonkey mildwonkey force-pushed the mildwonkey/api-post branch 4 times, most recently from 0ff60d4 to ac4a643 Compare November 21, 2024 13:20
@meganwolf0
Copy link
Collaborator

With a couple tests I did, the raw and response appear to have the same data... Is there a scenario where they'd have different information? I guess I'm not totally tracking the reason for both. I think also, if this is the final structure, the api-domain.md output schema part will need to be updated

@mildwonkey
Copy link
Contributor Author

@meganwolf0 they do have the same data; though the raw message hasn't been unmarshalled into json. I'm getting ahead of myself. It's more part of a long-term vision that anything that has meaning right now. I've been playing with the idea of letting users supply a basic json/openapi "schema" for the API output (the idea is to support things like storing a token in a variable lula can use in the next call) and that's where having the raw output would be valuable for debugging.

It's not adding anything at the moment though so I can definitely remove it. I think I dropped it in now to support my argument that the "response" key is useful: I worry about collisions in the API domain's DomainResources if we put the entire HTTP response at the same level. I could put everything that isn't the HTTP response under a different key though, so maybe something like this instead? (we could still end up with a collision on "metadata" though, so I think a key for the response body is the right idea, but I'm open to different shapes)

{ 
  "name":  {
    "metadata": {
         "status": 200,
         //  "raw": <json.RawMessage copy of the original API response> but will leave it out for now
    < unmarshalled json here>,
  }
}

@meganwolf0
Copy link
Collaborator

Definitely agree on having the response key since we added status! And ok cool the vision 100% makes sense, I was thinking that you said a reason but couldn't quite recall, so thank you for documenting it here! I could see either case for leaving raw in right now vs deferring to later, so no strong opinion on that.

brandtkeller
brandtkeller previously approved these changes Nov 22, 2024
Copy link
Member

@brandtkeller brandtkeller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finished testing a variety of scenarios with httpbin in order to familiarize myself with the outcomes. Appreciate the patience involved in review.

@brandtkeller brandtkeller merged commit 24f02ea into main Nov 22, 2024
10 checks passed
@brandtkeller brandtkeller deleted the mildwonkey/api-post branch November 22, 2024 23:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

Add Post + IsExecutable
3 participants