Skip to content

Commit

Permalink
[OPIK-683] TypeScript SDK batch support (#1197)
Browse files Browse the repository at this point in the history
* [OPIK-683] WIP: Implementing batch system for TS SDK

* Implement batch queues instead of direct calls to the API

* Improve batch mechanism + update test script

* Add better coverage with the batch mechanism and fix batching

* Improve batching system + add better coverage

* Upgrade to Fern TS 0.48.5

* Improve coding in the BatchQueue + add more coverage

* Add limit to batch size

* WIP: track decorator

* [OPIK-683] Upgrade TS SDK Fern version to 0.48.5

* Remove zone.js (and drop browser support) in favor of async_hooks (only Node/Deno/Bun)

* Add track decorator support

* Add support for track decorator in TypeScript

* Split spans in batches by parentSpanId (BE issue if not)

* Fix build for decorators

* Improve edge-cases of spans without parent + batch queue dependencies improvement

* Fix issue with the BE on update span with parent span ID

* Improve track decorator to support new (experimental) and old decorators in TypeScript

* Bump patch version

* Add simple README file

* Improve README

* Add input/outputs to the README example

* Bump version
  • Loading branch information
Fernando Carril authored Feb 3, 2025
1 parent 7d0580e commit eb1dd56
Show file tree
Hide file tree
Showing 19 changed files with 1,029 additions and 217 deletions.
81 changes: 81 additions & 0 deletions sdks/typescript/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
<h1 align="center" style="border-bottom: none">
<div>
<a href="https://www.comet.com/site/products/opik/?from=llm&utm_source=opik&utm_medium=github&utm_content=header_img&utm_campaign=opik"><picture>
<source media="(prefers-color-scheme: dark)" srcset="/apps/opik-documentation/documentation/static/img/logo-dark-mode.svg">
<source media="(prefers-color-scheme: light)" srcset="/~https://github.com/comet-ml/opik/blob/HEAD/apps/opik-documentation/documentation/static/img/opik-logo.svg">
<img alt="Comet Opik logo" src="/~https://github.com/comet-ml/opik/blob/HEAD/apps/opik-documentation/documentation/static/img/opik-logo.svg" width="200" />
</picture></a>
<br>
Opik
</div>
Open source LLM evaluation framework<br>
</h1>

## Installation

You can install the `opik` package using your favorite package manager.

```bash
npm install opik
```

## Usage

```typescript
import { Opik } from "opik";

// Create a new Opik client with your configuration
const client = new Opik({
apiKey: "<your-api-key>",
host: "https://www.comet.com/opik/api",
projectName: "<your-project-name>",
workspaceName: "<your-workspace-name>",
});

// Log 10 traces
for (let i = 0; i < 10; i++) {
const someTrace = client.trace({
name: `Trace ${i}`,
input: {
prompt: `Hello, world! ${i}`,
},
output: {
response: `Hello, world! ${i}`,
},
});

// For each trace, log 10 spans
for (let j = 0; j < 10; j++) {
const someSpan = someTrace.span({
name: `Span ${i}-${j}`,
type: "llm",
input: {
prompt: `Hello, world! ${i}:${j}`,
},
output: {
response: `Hello, world! ${i}:${j}`,
},
});

// Some LLM work
await new Promise((resolve) => setTimeout(resolve, 100));

// Mark the span as ended
someSpan.end();
}

// Mark the trace as ended
someTrace.end();
}

// Flush the client to send all traces and spans
await client.flush();
```

## Contributing

Contributions are welcome! If you have any suggestions or improvements, please feel free to open an [issue](/~https://github.com/comet-ml/opik/issues) or submit a [pull request](/~https://github.com/comet-ml/opik/pulls).

## License

This project is licensed under the [Apache License 2.0](/~https://github.com/comet-ml/opik/blob/main/LICENSE).
28 changes: 17 additions & 11 deletions sdks/typescript/examples/manual-trace.js
Original file line number Diff line number Diff line change
@@ -1,16 +1,22 @@
import { Opik } from "opik";

const client = new Opik();
const someTrace = await client.trace({
name: "test123",
startTime: new Date(),
});

const someSpan = await someTrace.span({
name: "test123 span",
type: "llm",
startTime: new Date(),
});
for (let i = 0; i < 10; i++) {
const someTrace = client.trace({
name: `Trace ${i}`,
});

await someSpan.end();
await someTrace.end();
for (let j = 0; j < 10; j++) {
const someSpan = someTrace.span({
name: `Span ${i}-${j}`,
type: "llm",
});

someSpan.end();
}

someTrace.end();
}

await client.flush();
23 changes: 23 additions & 0 deletions sdks/typescript/examples/track-decorator.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
import { track, trackOpikClient } from "opik";

class TestClass {
@track({ type: "llm" })
async llmCall() {
return "llm result";
}

@track({ name: "translate" })
async translate(text: string) {
return `translated: ${text}`;
}

@track({ name: "initial", projectName: "track-decorator-test" })
async execute() {
const result = await this.llmCall();
return this.translate(result);
}
}

const test = new TestClass();
await test.execute();
await trackOpikClient.flush();
Loading

0 comments on commit eb1dd56

Please sign in to comment.