Access via Destination

[sagar-git-hub]

Ask the Question

Hello Team,
Our models and the application consuming the model runs in different accounts. I have created the destination where the application is hosted and have added Oauth2 credentials.
Do you have any documentation on how to use the SDK to call the emdebbing model (using destination) and get the embeddings and store it in Hana vector DB?

[MatKuhr]

Hey, yes this is a use case we have in mind and have documented in the readme.

However, that example is for using the DeploymentApi and currently doesn't translate well to the OpenAiClient. We are working to make it a bit easier with this PR: #201 If you like you can already check out and install the SDK from that branch locally to try it out.

[sagar-git-hub]

Ok thanks. When will the ai-sdk-java be GA?

[MatKuhr]

Version 1.0.0 is available now: https://central.sonatype.com/artifact/com.sap.ai.sdk/core/versions

To use a destination from destination service, you can use:

var destination = DestinationAccessor.getDestination("myDestination").asHttp();
var service = new AiCoreService().withBaseDestination(destination);
var openAiDestination = service.getInferenceDestination().forModel(OpenAiModel.TEXT_EMBEDDING_3_LARGE);
var client = OpenAiClient.withCustomDestination(openAiDestination);

Please run this code each time you want to use the client, because the destination will expire when the attached token expires. This is only needed when you use a destination from BTP destination service.

[sagar-git-hub]

Hi @MatKuhr ,
I have not set any env variable. I have created a destination, but I get below error:

	at java.base/java.lang.Thread.run(Thread.java:840) ~[na:na]
	Suppressed: com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationNotFoundException: Could not find environment variable for name 'destinations'.
		at com.sap.cloud.sdk.cloudplatform.connectivity.EnvVarDestinationLoader.getJsonNodes(EnvVarDestinationLoader.java:138) ~[cloudplatform-connectivity-5.14.0.jar:na]
		at com.sap.cloud.sdk.cloudplatform.connectivity.EnvVarDestinationLoader.extractDestination(EnvVarDestinationLoader.java:162) ~[cloudplatform-connectivity-5.14.0.jar:na]
		at com.sap.cloud.sdk.cloudplatform.connectivity.EnvVarDestinationLoader.lambda$tryGetDestination$79e22d5b$1(EnvVarDestinationLoader.java:116) ~[cloudplatform-connectivity-5.14.0.jar:na]
		at io.vavr.control.Try.of(Try.java:75) ~[vavr-0.10.5.jar:na]
		at com.sap.cloud.sdk.cloudplatform.connectivity.EnvVarDestinationLoader.tryGetDestination(EnvVarDestinationLoader.java:116) ~[cloudplatform-connectivity-5.14.0.jar:na]
		at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationLoaderChain.tryGetDestination(DestinationLoaderChain.java:89) ~[cloudplatform-connectivity-5.14.0.jar:na]
		... 135 common frames omitted

[MatKuhr]

That is only a suppressed exception and not necessarily the root cause. I guess you may have not bound the BTP destination service to your application

[sagar-git-hub]

Hi @MatKuhr ,
I was able to pass the destination details (local setup)

DestinationProperties baseProperties = DefaultDestination.builder()
				.property("Name", "model")
				.property("Type", "HTTP")
				.property("URL", "url")
				.property("Authentication", "OAuth2ClientCredentials")
				.property("ProxyType", "Internet")
				.property("ClientID", "clientID")
				.property("ClientSecret", "secret")
				.property("TokenServiceURL", "tokenURL")
				.build();
		DefaultHttpDestination dest = DefaultHttpDestination.fromProperties(baseProperties).build();
		var service = new AiCoreService().withBaseDestination(dest);
		var openAiDestination = service.getInferenceDestination("resource_group").usingDeploymentId("deploymentID");
		var client = OpenAiClient.withCustomDestination(openAiDestination);

I need to use Oauth2 credentials. But I don't see any Authorization header in the request which is made to the model. Not sure if the property name that I set is correct.
I see these logs while debugging

No proxy URI or host and port specified. Continuing without proxy configuration.
No valid JSON primitives 'User' and 'Password' defined. Continuing without basic credentials

[MatKuhr]

Using DefaultDestination or DefaultHttpDestination does not work. If you want to build an OAuth destination locally, you can use OAuth2DestinationBuilder.