Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Orchestration Convenience] Typed model parameters #180

Merged
merged 13 commits into from
Nov 29, 2024

Conversation

newtork
Copy link
Contributor

@newtork newtork commented Nov 21, 2024

Would be helpful for demo purposes and general usability.

+ import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.Parameter.*;

OrchestrationAiModel customGPT4O =
    OrchestrationAiModel.GPT_4O
-       .withParams(
-           Map.of(
-               "max_tokens", 50,
-               "temperature", 0.1,
-               "frequency_penalty", 0,
-               "presence_penalty", 0))
+       .withParam(MAX_TOKENS, 50)
+       .withParam(TEMPERATURE, 0.1)
+       .withParam(FREQUENCY_PENALTY, 0)
+       .withParam(PRESENCE_PENALTY, 0)
        .withVersion("2024-05-13");

image

@newtork newtork marked this pull request as draft November 21, 2024 15:31
Copy link
Contributor Author

@newtork newtork left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed in todays (after) daily..

  • We want to offer API (via fields, methods or classes) for the available parameter keys.
  • We can't reasonably maintain the full mapping of LLM provider / model parameters.
  • We can carefully(?) assume a very small subset of keys to be available for all LLM providers.

How to translate that to code, needs to be aligned still.

@newtork newtork added the please-review Request to review a pull-request label Nov 27, 2024
@newtork newtork marked this pull request as ready for review November 27, 2024 14:56
jjtang1985
jjtang1985 previously approved these changes Nov 27, 2024
Copy link
Contributor

@jjtang1985 jjtang1985 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API looks good!
Very helpful.

Copy link
Member

@MatKuhr MatKuhr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise LGTM. Maybe we can add the link to https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/harmonized-api somewhere in the Javadoc / sample code / docs

Copy link
Member

@MatKuhr MatKuhr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@MatKuhr MatKuhr enabled auto-merge (squash) November 29, 2024 07:26
@MatKuhr MatKuhr merged commit e352343 into main Nov 29, 2024
5 checks passed
@MatKuhr MatKuhr deleted the orchestration-convenience/typed-model-parameters branch November 29, 2024 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
please-review Request to review a pull-request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants