feat: [Orchestration] Spring AI integration (#278)

* WiP * WiP 2 * finito * WiP * WiP 2 * finito * Formatting * docs * repo in parent pom * Formatting * Update docs * Aligned headers * Added no args constructor * Removed spring-ai-app * Added images in index.html * Alex's review comments * Shorter * Added instructions * M5 * fixed copy * Formatting * removed warnings * green CI * Fixed controller, added service * Removed chat Memory * Alex's review * Better documentation * Added images * Better images * mb --------- Co-authored-by: SAP Cloud SDK Bot <cloudsdk@sap.com>
SAP · Jan 17, 2025 · 98facc1 · 98facc1
1 parent 1b6d286
commit 98facc1
Show file tree

Hide file tree

Showing 25 changed files with 1,271 additions and 76 deletions.
diff --git a/README.md b/README.md
@@ -131,10 +131,11 @@ Please refer to [this documentation](docs/guides/ORCHESTRATION_CHAT_COMPLETION.m
 
 For more detailed information and advanced usage, please refer to the following:
 
-- [Connecting to AI Core](docs/guides/CONNECTING_TO_AICORE.md)
-- [Orchestration Chat Completion](docs/guides/ORCHESTRATION_CHAT_COMPLETION.md)
-- [OpenAI Chat Completion](docs/guides/OPENAI_CHAT_COMPLETION.md)
-- [AI Core Deployment](docs/guides/AI_CORE_DEPLOYMENT.md)
+- [<img src="sample-code/spring-app/src/main/resources/static/BTP-Cockpit-Logo.png"/> Connecting to AI Core](docs/guides/CONNECTING_TO_AICORE.md)
+- [<img src="sample-code/spring-app/src/main/resources/static/Orchestration-Logo.png" width="16"/> Orchestration Chat Completion](docs/guides/ORCHESTRATION_CHAT_COMPLETION.md)
+- [<img src="sample-code/spring-app/src/main/resources/static/Open-AI-Logo.svg" width="16"/> OpenAI Chat Completion](docs/guides/OPENAI_CHAT_COMPLETION.md)
+- [<img src="https://spring.io/favicon-32x32.png" width="16"/> Spring AI Integration](docs/guides/SPRING_AI_INTEGRATION.md)
+- [🧰 AI Core Deployment](docs/guides/AI_CORE_DEPLOYMENT.md)
 
 For updating versions, please refer to the [**Release Notes**](docs/release-notes/release-notes-0-to-14.md).
 

diff --git a/docs/guides/AI_CORE_DEPLOYMENT.md b/docs/guides/AI_CORE_DEPLOYMENT.md
@@ -66,7 +66,7 @@ In addition to the prerequisites above, we assume you have already set up the fo
       ```
       </details>
 
-### Create a Deployment
+## Create a Deployment
 
 Use the following code snippet to create a deployment in SAP AI Core:
 
@@ -84,7 +84,7 @@ AiExecutionStatus status = deployment.getStatus();
 
 Refer to the [DeploymentController.java](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/DeploymentController.java) in our Spring Boot application for a complete example.
 
-### Delete a Deployment
+## Delete a Deployment
 
 ```java
 AiDeploymentCreationResponse deployment; // provided

diff --git a/docs/guides/OPENAI_CHAT_COMPLETION.md b/docs/guides/OPENAI_CHAT_COMPLETION.md
@@ -87,7 +87,7 @@ In addition to the prerequisites above, we assume you have already set up the fo
 
       </details>
 
-### Simple chat completion
+## Simple chat completion
 
 ```java
 var result =
@@ -98,7 +98,7 @@ var result =
 String resultMessage = result.getContent();
 ```
 
-### Using a Custom Resource Group
+## Using a Custom Resource Group
 
 ```java
 var destination = new AiCoreService()
@@ -107,7 +107,7 @@ var destination = new AiCoreService()
 OpenAiClient.withCustomDestination(destination);
 ```
 
-### Message history
+## Message history
 
 ```java
 var systemMessage =
@@ -124,7 +124,7 @@ String resultMessage = result.getContent();
 
 See [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OpenAiController.java)
 
-### Chat Completion with Specific Model Version
+## Chat Completion with Specific Model Version
 
 By default, when no version is specified, the system selects one of the available deployments of the specified model, regardless of its version.
 To target a specific version, you can specify the model version along with the model.
@@ -134,7 +134,7 @@ OpenAiChatCompletionOutput result =
     OpenAiClient.forModel(GPT_35_TURBO.withVersion("1106")).chatCompletion(request);
 ```
 
-### Chat completion with Custom Model
+## Chat completion with Custom Model
 
 You can also use a custom OpenAI model for chat completion by creating an `OpenAiModel` object. 
 
@@ -145,11 +145,11 @@ OpenAiChatCompletionOutput result =
 
 Ensure that the custom model is deployed in SAP AI Core.
 
-### Stream chat completion
+## Stream chat completion
 
 It's possible to pass a stream of chat completion delta elements, e.g. from the application backend to the frontend in real-time.
 
-#### Asynchronous Streaming
+### Asynchronous Streaming
 
 This is a blocking example for streaming and printing directly to the console:
 
@@ -168,7 +168,7 @@ try (Stream<String> stream = client.streamChatCompletion(msg)) {
 }
 ```
 
-#### Aggregating Total Output
+### Aggregating Total Output
 
 The following example is non-blocking and demonstrates how to aggregate the complete response.
 Any asynchronous library can be used, such as the classic Thread API.
@@ -205,12 +205,10 @@ Integer tokensUsed = totalOutput.getUsage().getCompletionTokens();
 System.out.println("Tokens used: " + tokensUsed);
 ```
 
-#### Spring Boot example
-
 Please find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OpenAiController.java). It shows the usage of Spring
 Boot's `ResponseBodyEmitter` to stream the chat completion delta messages to the frontend in real-time.
 
-### Embedding
+## Embedding
 
 Get the embeddings of a text input in list of float values:
 

diff --git a/docs/guides/ORCHESTRATION_CHAT_COMPLETION.md b/docs/guides/ORCHESTRATION_CHAT_COMPLETION.md
@@ -85,7 +85,7 @@ var config = new OrchestrationModuleConfig()
 
 Please also refer to [our sample code](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OrchestrationController.java) for this and all following code examples.
 
-### Chat Completion
+## Chat Completion
 
 Use the Orchestration service to generate a response to a user message:
 
@@ -100,7 +100,7 @@ String messageResult = result.getContent();
 In this example, the Orchestration service generates a response to the user message "Hello world! Why is this phrase so famous?".
 The LLM response is available as the first choice under the `result.getOrchestrationResult()` object.
 
-### Chat completion with Templates
+## Chat completion with Templates
 
 Use a prepared template and execute requests with by passing only the input parameters:
 
@@ -117,7 +117,7 @@ var result = client.chatCompletion(prompt, configWithTemplate);
 
 In this case the template is defined with the placeholder `{{?language}}` which is replaced by the value `German` in the input parameters.
 
-### Message history
+## Message history
 
 Include a message history to maintain context in the conversation:
 
@@ -134,7 +134,7 @@ var prompt = new OrchestrationPrompt(message).messageHistory(messagesHistory);
 var result = new OrchestrationClient().chatCompletion(prompt, config);
 ```
 
-### Chat completion filter
+## Chat completion filter
 
 Apply content filtering to the chat completion:
 
@@ -165,7 +165,7 @@ var configWithFilter = config.withInputFiltering(filterStrict).withOutputFilteri
 var result =
     new OrchestrationClient().chatCompletion(prompt, configWithFilter);
 ```
-#### Behavior of Input and Output Filters
+### Behavior of Input and Output Filters
 
 - **Input Filter**:
   If the input message violates the filter policy, a `400 (Bad Request)` response will be received during the `chatCompletion` call.
@@ -178,7 +178,7 @@ var result =
 
 You will find [some examples](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OrchestrationController.java) in our Spring Boot application demonstrating response handling with filters.
 
-### Data masking
+## Data masking
 
 Use the data masking module to anonymize personal information in the input:
 
@@ -201,7 +201,7 @@ var result =
 
 In this example, the input will be masked before the call to the LLM and will remain masked in the output.
 
-### Grounding
+## Grounding
 
 Use the grounding module to provide additional context to the AI model. 
 
@@ -232,11 +232,11 @@ Use the grounding module to provide additional context to the AI model.
 
 In this example, the AI model is provided with additional context in the form of grounding information. Note, that it is necessary to provide the grounding input via one or more input variables.
 
-### Stream chat completion
+## Stream chat completion
 
 It's possible to pass a stream of chat completion delta elements, e.g. from the application backend to the frontend in real-time.
 
-#### Asynchronous Streaming
+### Asynchronous Streaming
 
 This is a blocking example for streaming and printing directly to the console:
 
@@ -253,12 +253,10 @@ try (Stream<String> stream = client.streamChatCompletion(prompt, config)) {
 }
 ```
 
-#### Spring Boot example
-
 Please find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OrchestrationController.java).
 It shows the usage of Spring Boot's `ResponseBodyEmitter` to stream the chat completion delta messages to the frontend in real-time.
 
-### Set model parameters
+## Set model parameters
 
 Change your LLM configuration to add model parameters:
 
@@ -272,7 +270,7 @@ OrchestrationAiModel customGPT4O =
         .withVersion("2024-05-13");
 ```
 
-### Using a Configuration from AI Launchpad
+## Using a Configuration from AI Launchpad
 
 In case you have created a configuration in AI Launchpad, you can copy or download the configuration as JSON and use it directly in your code:
 

diff --git a/docs/guides/SPRING_AI_INTEGRATION.md b/docs/guides/SPRING_AI_INTEGRATION.md
@@ -0,0 +1,52 @@
+# Spring AI Integration
+
+## Table of Contents
+
+- [Introduction](#introduction)
+- [Orchestration Chat Completion](#orchestration-chat-completion)
+- [Orchestration Masking](#orchestration-masking)
+
+## Introduction
+
+This guide provides examples of how to use our Spring AI integration with our clients in SAP AI Core
+for chat completion tasks using the SAP AI SDK for Java.
+
+## Orchestration Chat Completion
+
+The Orchestration client is integrated in Spring AI classes:
+
+```java
+ChatModel client = new OrchestrationChatModel();
+OrchestrationModuleConfig config = new OrchestrationModuleConfig().withLlmConfig(GPT_35_TURBO);
+OrchestrationChatOptions opts = new OrchestrationChatOptions(config);
+
+Prompt prompt = new Prompt("What is the capital of France?", opts);
+ChatResponse response = client.call(prompt);
+```
+
+Please
+find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/SpringAiOrchestrationController.java).
+
+## Orchestration Masking
+
+Configure Orchestration modules withing Spring AI:
+
+```java
+ChatModel client = new OrchestrationChatModel();
+OrchestrationModuleConfig config = new OrchestrationModuleConfig().withLlmConfig(GPT_35_TURBO);
+
+val masking =
+    DpiMasking.anonymization()
+        .withEntities(DPIEntities.EMAIL, DPIEntities.ADDRESS, DPIEntities.LOCATION);
+
+val opts = new OrchestrationChatOptions(config.withMaskingConfig(masking));
+val prompt =
+    new Prompt(
+        "Please write 'Hello World!' to me via email. My email address is foo.bar@baz.ai",
+        opts);
+
+ChatResponse response = client.call(prompt);
+```
+
+Please
+find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/SpringAiOrchestrationController.java).
diff --git a/docs/release-notes/release_notes.md b/docs/release-notes/release_notes.md
@@ -17,6 +17,7 @@
 - Orchestration supports images as input in newly introduced `MultiChatMessage`.
 - `MultiChatMessage` also allows for multiple content items (text or image) in one object.
 - Grounding input can be masked with `DPIConfig`.
+- [Integrate the Orchestration client in Spring AI.](../guides/ORCHESTRATION_CHAT_COMPLETION.md#spring-ai-integration)
 
 ### 📈 Improvements
 

diff --git a/orchestration/pom.xml b/orchestration/pom.xml
@@ -31,11 +31,11 @@
   </developers>
   <properties>
     <project.rootdir>${project.basedir}/../</project.rootdir>
-    <coverage.complexity>70%</coverage.complexity>
-    <coverage.line>87%</coverage.line>
-    <coverage.instruction>88%</coverage.instruction>
-    <coverage.branch>65%</coverage.branch>
-    <coverage.method>65%</coverage.method>
+    <coverage.complexity>76%</coverage.complexity>
+    <coverage.line>90%</coverage.line>
+    <coverage.instruction>91%</coverage.instruction>
+    <coverage.branch>69%</coverage.branch>
+    <coverage.method>70%</coverage.method>
     <coverage.class>100%</coverage.class>
   </properties>
 
@@ -54,7 +54,11 @@
       <groupId>com.sap.cloud.sdk.cloudplatform</groupId>
       <artifactId>connectivity-apache-httpclient5</artifactId>
     </dependency>
-
+    <dependency>
+      <groupId>org.springframework.ai</groupId>
+      <artifactId>spring-ai-core</artifactId>
+      <optional>true</optional>
+    </dependency>
     <dependency>
       <groupId>org.apache.httpcomponents.core5</groupId>
       <artifactId>httpcore5</artifactId>

diff --git a/orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationClient.java b/orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationClient.java
@@ -1,24 +1,20 @@
 package com.sap.ai.sdk.orchestration;
 
-import static com.sap.ai.sdk.core.JacksonConfiguration.getDefaultObjectMapper;
+import static com.sap.ai.sdk.orchestration.OrchestrationJacksonConfiguration.getOrchestrationObjectMapper;
 
 import com.fasterxml.jackson.core.JsonProcessingException;
 import com.fasterxml.jackson.databind.JsonNode;
 import com.fasterxml.jackson.databind.ObjectMapper;
-import com.fasterxml.jackson.databind.module.SimpleModule;
 import com.fasterxml.jackson.databind.node.ObjectNode;
 import com.google.common.annotations.Beta;
 import com.sap.ai.sdk.core.AiCoreService;
 import com.sap.ai.sdk.core.DeploymentResolutionException;
 import com.sap.ai.sdk.core.common.ClientResponseHandler;
 import com.sap.ai.sdk.core.common.ClientStreamingHandler;
 import com.sap.ai.sdk.core.common.StreamedDelta;
-import com.sap.ai.sdk.orchestration.model.ChatMessagesInner;
 import com.sap.ai.sdk.orchestration.model.CompletionPostRequest;
 import com.sap.ai.sdk.orchestration.model.CompletionPostResponse;
-import com.sap.ai.sdk.orchestration.model.LLMModuleResult;
 import com.sap.ai.sdk.orchestration.model.ModuleConfigs;
-import com.sap.ai.sdk.orchestration.model.ModuleResultsOutputUnmaskingInner;
 import com.sap.ai.sdk.orchestration.model.OrchestrationConfig;
 import com.sap.cloud.sdk.cloudplatform.connectivity.ApacheHttpClient5Accessor;
 import com.sap.cloud.sdk.cloudplatform.connectivity.HttpDestination;
@@ -41,25 +37,7 @@
 public class OrchestrationClient {
   private static final String DEFAULT_SCENARIO = "orchestration";
 
-  static final ObjectMapper JACKSON;
-
-  static {
-    JACKSON = getDefaultObjectMapper();
-
-    // Add mix-ins
-    JACKSON.addMixIn(LLMModuleResult.class, JacksonMixins.LLMModuleResultMixIn.class);
-    JACKSON.addMixIn(
-        ModuleResultsOutputUnmaskingInner.class,
-        JacksonMixins.ModuleResultsOutputUnmaskingInnerMixIn.class);
-
-    final var module =
-        new SimpleModule()
-            .addDeserializer(
-                ChatMessagesInner.class,
-                PolymorphicFallbackDeserializer.fromJsonSubTypes(ChatMessagesInner.class))
-            .setMixInAnnotation(ChatMessagesInner.class, JacksonMixins.NoneTypeInfoMixin.class);
-    JACKSON.registerModule(module);
-  }
+  static final ObjectMapper JACKSON = getOrchestrationObjectMapper();
 
   @Nonnull private final Supplier<HttpDestination> destinationSupplier;