-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test: Using Orchestration Grounding module #228
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add documentation
The grounding app endpoint is broken, the index.html
is not updated
I would also like to see a bit more assertions on the e2e test.
And a unit test to assert on all fields of the response, because we don't know if the generated classes can access all fields of the JSON response.
...le-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OrchestrationController.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
minor notes, lgtm
orchestration/src/test/java/com/sap/ai/sdk/orchestration/OrchestrationUnitTest.java
Outdated
Show resolved
Hide resolved
Co-authored-by: Matthias Kuhr <52661546+MatKuhr@users.noreply.github.com>
b5bd89d
orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationModuleConfig.java
Show resolved
Hide resolved
var llmChoice = | ||
((LLMModuleResultSynchronous) result.getOrchestrationResult()).getChoices().get(0); | ||
|
||
assertThat(result.getModuleResults().getGrounding().getData().toString()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Minor/Comment)
I think AssertJ also has ).hasToString(
as replacement for .toString()).isEqualTo(
.
Do we actually need the string check when we compare the values anyway? AFAIK we didn't do this in other places.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I cannot compare the values directly here, as getModuleResults().getGrounding().getData()
returns an Object
so that I cannot access "grounding_query" or "grounding_result" via the generated code. This is why I compare it as a String.
Also, I use .toString()).startsWith(...
now to not need to provide the complete rather long expected String.
If you feel there is a better way to do that, please let me know :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Object
is somewhat misleading, actually this will always be a Map<String, Object>
. (Ideally, our generator would be improved in that regard).
So you could cast to that and use .get("grounding_query")
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Casting this adds a warning, though. Is that a problem?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, it's just a test assertion, so it's okay if this ever fails (which would mean we've changed our generator but didn't update tests)
Context
AI/ai-sdk-java-backlog#125.
Add an end-to-end test of the orchestration grounding module.
Feature scope:
Definition of Done
Error handling created / updated & covered by the tests aboveAligned changes with the JavaScript SDKRelease notes updated