Fix llm client retry

[danielchalef]

🚀	This description was created by Ellipsis for commit 461780c

Summary:

This PR introduces RateLimitError and updates LLM client retry logic to handle rate limit errors using exponential backoff.

Key points:

• Introduces RateLimitError in graphiti_core/llm_client/errors.py.
• Updates graphiti_core/llm_client/__init__.py to export RateLimitError.
• Modifies graphiti_core/llm_client/client.py to retry on RateLimitError using tenacity with exponential backoff.
• Updates graphiti_core/llm_client/anthropic_client.py, groq_client.py, and openai_client.py to raise RateLimitError on rate limit exceptions.

---

Generated with ❤️ by ellipsis.dev

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to a053889 in 24 seconds

More details

• Looked at 187 lines of code in 6 files
• Skipped 0 files when reviewing.
• Skipped posting 5 drafted comments based on config settings.

1. graphiti_core/llm_client/anthropic_client.py:43

• Draft comment:
  The max_retries parameter in AsyncAnthropic is set to 1, which might conflict with the retry logic in LLMClient. Consider aligning these settings to avoid unexpected behavior.
• Reason this comment was not posted:
  Confidence changes required: 80%
  The retry logic in the LLMClient class is set to retry up to 4 times with exponential backoff. However, the max_retries parameter in the AsyncAnthropic client is set to 1, which might conflict with the retry logic in LLMClient. This could lead to unexpected behavior if the AsyncAnthropic client does not allow retries beyond its max_retries setting.

2. graphiti_core/llm_client/client.py:37

• Draft comment:
  Consider extending is_server_or_retry_error to handle other retryable errors like network timeouts or connection errors for more robust retry logic.
• Reason this comment was not posted:
  Confidence changes required: 70%
  The is_server_or_retry_error function checks for RateLimitError and HTTP 5xx errors. However, it does not handle other potential retryable errors like network timeouts or connection errors, which could also benefit from retry logic.

3. graphiti_core/llm_client/anthropic_client.py:66

• Draft comment:
  Consider handling JSONDecodeError in _generate_response to manage cases where the response is not valid JSON.
• Reason this comment was not posted:
  Confidence changes required: 70%
  The json.loads call in _generate_response methods assumes the response is always valid JSON. If the response is malformed, this will raise a JSONDecodeError, which is not currently handled.

4. graphiti_core/llm_client/groq_client.py:63

• Draft comment:
  Consider handling JSONDecodeError in _generate_response to manage cases where the response is not valid JSON.
• Reason this comment was not posted:
  Confidence changes required: 70%
  The same issue with json.loads exists in the groq_client.py file, where the response is assumed to be valid JSON without handling potential JSONDecodeError.

5. graphiti_core/llm_client/openai_client.py:63

• Draft comment:
  Consider handling JSONDecodeError in _generate_response to manage cases where the response is not valid JSON.
• Reason this comment was not posted:
  Confidence changes required: 70%
  The same issue with json.loads exists in the openai_client.py file, where the response is assumed to be valid JSON without handling potential JSONDecodeError.

Workflow ID: wflow_Mra9qL90kfp8c7nb

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

👍 Looks good to me! Incremental review on 461780c in 18 seconds

More details

• Looked at 13 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 1 drafted comments based on config settings.

1. graphiti_core/llm_client/client.py:66

• Draft comment:
  The after parameter in the retry decorator should directly assign the lambda function without the conditional expression. The current implementation may lead to unexpected behavior if retry_state.attempt_number is 1. Consider simplifying the assignment.
• Reason this comment was not posted:
  Confidence changes required: 80%
  The lambda function in the after parameter of the retry decorator should be directly assigned without the conditional expression. This ensures that the warning is logged only when necessary, avoiding potential issues when retry_state.attempt_number is 1.

Workflow ID: wflow_H9J6qn3oHBSbqyDm

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.