Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test: Using Orchestration Grounding module #228

Merged
merged 15 commits into from
Dec 10, 2024
Merged

Test: Using Orchestration Grounding module #228

merged 15 commits into from
Dec 10, 2024

Conversation

Jonas-Isr
Copy link
Member

@Jonas-Isr Jonas-Isr commented Dec 5, 2024

Context

AI/ai-sdk-java-backlog#125.

Add an end-to-end test of the orchestration grounding module.

Feature scope:

  • Add end-to-end test.

Definition of Done

  • Functionality scope stated & covered
  • Tests cover the scope above
  • Error handling created / updated & covered by the tests above
  • Aligned changes with the JavaScript SDK
  • Documentation updated
  • Release notes updated

@Jonas-Isr Jonas-Isr added the please-review Request to review a pull-request label Dec 5, 2024
Copy link
Contributor

@CharlesDuboisSAP CharlesDuboisSAP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add documentation
The grounding app endpoint is broken, the index.html is not updated
I would also like to see a bit more assertions on the e2e test.
And a unit test to assert on all fields of the response, because we don't know if the generated classes can access all fields of the JSON response.

@Jonas-Isr Jonas-Isr removed the please-review Request to review a pull-request label Dec 6, 2024
@Jonas-Isr Jonas-Isr self-assigned this Dec 6, 2024
@Jonas-Isr Jonas-Isr added the please-review Request to review a pull-request label Dec 9, 2024
MatKuhr
MatKuhr previously approved these changes Dec 9, 2024
Copy link
Member

@MatKuhr MatKuhr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor notes, lgtm

Co-authored-by: Matthias Kuhr <52661546+MatKuhr@users.noreply.github.com>
@Jonas-Isr Jonas-Isr dismissed stale reviews from MatKuhr and CharlesDuboisSAP via b5bd89d December 10, 2024 09:08
var llmChoice =
((LLMModuleResultSynchronous) result.getOrchestrationResult()).getChoices().get(0);

assertThat(result.getModuleResults().getGrounding().getData().toString())
Copy link
Contributor

@newtork newtork Dec 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Minor/Comment)

I think AssertJ also has ).hasToString( as replacement for .toString()).isEqualTo(.

Do we actually need the string check when we compare the values anyway? AFAIK we didn't do this in other places.

Copy link
Member Author

@Jonas-Isr Jonas-Isr Dec 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot compare the values directly here, as getModuleResults().getGrounding().getData() returns an Object so that I cannot access "grounding_query" or "grounding_result" via the generated code. This is why I compare it as a String.

Also, I use .toString()).startsWith(... now to not need to provide the complete rather long expected String.

If you feel there is a better way to do that, please let me know :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Object is somewhat misleading, actually this will always be a Map<String, Object>. (Ideally, our generator would be improved in that regard).

So you could cast to that and use .get("grounding_query")

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Casting this adds a warning, though. Is that a problem?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it's just a test assertion, so it's okay if this ever fails (which would mean we've changed our generator but didn't update tests)

@Jonas-Isr Jonas-Isr removed the please-review Request to review a pull-request label Dec 10, 2024
@Jonas-Isr Jonas-Isr added the please-review Request to review a pull-request label Dec 10, 2024
@Jonas-Isr Jonas-Isr requested a review from MatKuhr December 10, 2024 11:41
@Jonas-Isr Jonas-Isr merged commit a174ba8 into main Dec 10, 2024
5 checks passed
@Jonas-Isr Jonas-Isr deleted the grounding-test branch December 10, 2024 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
please-review Request to review a pull-request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants