This release includes major improvements for middleware support.
- If Handler: New handler that conditionally executes a handler based on a predicate.
- Noop Handler: New handler that does nothing, useful for testing and debugging.
- Logging Middleware: New middleware for logging handler execution.
- ThreadFlow: Originally intended to manage middleware chains, it has been removed in favor of a more flexible approach.
- Retry Middleware: Added retry criteria and refactored to use new middleware pattern.
- OpenAI Provider: Fixed issue with system prompt not being set correctly. Added "Name" option.
- Sequence Handler: Fixed issue where thread context changes were not propagated to the next handler.
- OpenTelemetry Support: Integrated tracing via
go.opentelemetry.io/otel. - New Handlers:
- MetadataEquals: Enables conditional routing based on metadata keys.
- PolicyValidator: Validates thread content using an LLM with customizable result processing.
- New Middleware:
- Retry with Custom Criteria: Enhanced retry mechanism with configurable backoff and retry conditions.
- New ThreadFlow Features:
- Support for Scoped Middleware Groups, allowing different processing paths.
- Sequential Execution Enhancements: Improved execution order and thread state handling.
- ThreadFlow: Middleware can now be applied at different scopes within a group.
- Handlers:
- Must, First, and Sequential now provide better concurrency and error handling.
- For Handler: Improved iteration control with optional continuation functions.
- Retry Middleware:
- Added support for configurable retry criteria, backoff strategies, and context propagation.
- Deprecated
UseMiddleware Method: Fully replaced by the newThreadFlowmiddleware pattern.
- ThreadContext:
- Added
SetKeyValue, changed to copy on write strategy. - Added
MergetoMetadatawith merging strategies. AppendMessagesno longer returns ThreadContext; mutates in place.
- Added
- ThreadFlow: Added
ThreadFlowscoped middleware handler and handler chains.
- Gemini/OpenAI providers: Default to retryable HTTP client.
- Must Handler: Now accepts result aggregator.
- First, For, Range and Sequential: Removed
Usemiddleware method in lieu ofThreadFlowmiddleware pattern.
ThreadFlow manages multiple handlers and their middleware chains. Handlers are executed sequentially, with each handler receiving the result from the previous one. Middleware is scoped and applied to handlers added within that scope, allowing different middleware combinations for different processing paths.
Example:
validate := NewValidationHandler()
process := NewProcessingHandler()
seq := Sequential("main", validate, process)
// Create flow and add global middleware
flow := NewThreadFlow("example")
flow.Use(NewLogging("audit"))
// Add base handlers
flow.Handle(seq)
// Add handlers with scoped middleware
flow.Group(func(f *ThreadFlow) {
f.Use(NewRetry("retry", 3))
f.Use(NewTimeout("timeout", 5))
f.Handle(NewContentProcessor("content"))
f.Handle(NewValidator("validate"))
})
result, err := flow.HandleThread(initialContext, nil)
if err != nil {
log.Fatalf("Error in flow: %v", err)
}
fmt.Println("Result:", result.Messages().Last().Content)
Retry creates a middleware that automatically retries failed handler executions with configurable backoff and retry criteria.
The middleware supports customization through options including:
- Number of retry attempts
- Backoff strategy between attempts
- Retry criteria based on error evaluation
- Timeout propagation control
If no options are provided, the middleware uses default settings:
- 3 retry attempts
- No delay between attempts
- Retries on any error
- Timeout propagation enabled
Parameters:
- name: Identifier for the middleware instance
- opts: Optional configuration using retry.Option functions
Example usage:
flow.Use(Retry("api-retry",
retry.WithAttempts(5),
retry.WithBackoff(retry.DefaultBackoff(time.Second)),
retry.WithRetryCriteria(func(err error) bool {
return errors.Is(err, io.ErrTemporary)
}),
))
The middleware stops retrying if:
- An attempt succeeds
- The maximum number of attempts is reached
- The retry criteria returns false
- Context cancellation (if timeout propagation is enabled)
Returns:
- A middleware that implements the retry logic around a handler
- Added new handler: switch
- Renamed ThreadContext
AppendMessagetoAppendMessages(...Message)
Switch creates a new Switch handler that executes the first matching case's handler. If no cases match, it executes the default handler. The name parameter is used for debugging and logging purposes.
The SwitchCase struct pairs a SwitchCondition interface with a handler.
When the condition evaluates to true, the corresponding handler is executed.
Example:
The MetadataEquals condition checks if the metadata key "type" equals "question"
metadata := MetadataEquals{Key: "type", Value: "question"}
questionHandler := SomeQuestionHandler()
defaultHandler := DefaultHandler()
sw := Switch("type-switch",
defaultHandler,
SwitchCase{metadata, questionHandler},
)- Added the new handlers: For, First, Must, Policy, Range.
- Greatly simplified Response interface (for now). More changes coming.
The First handler executes all provided handlers in parallel and
returns when the first handler succeeds. If all handlers fail, it returns all
errors.
The For handler repeats the provided handler for a specified
number of iterations or infinitely. The handler can be used to repeat a handler
a fixed number of times or until a condition is met.
The Must handler executes all provided handlers in parallel and
returns when all handlers succeed. If any handler fails, it cancels execution of
any remaining handlers.
The Policy handler uses the LLM to validate thread content against a given policy.
The result is processed by the optional result function. If the result function is nil,
the handler defaults to checking the Valid field of the validation result.
The Range handler processes a thread with a series of values.
For each value in the provided list, the handler executes with the value stored in the thread context's metadata under the key "range_value". An optional middleware handler can be used to wrap each iteration.