slice bounds out of range - Frequent Ingress Controller restarts #12870
Labels
needs-kind
Indicates a PR lacks a `kind/foo` label and requires one.
needs-priority
needs-triage
Indicates an issue or PR lacks a `triage/foo` label and requires one.
Bug Description:
Our Kubernetes cluster has been running for the past 8 months, handling 10,000 to 30,000 requests per second, primarily for a chat module(web socket). There have been no sudden changes in load, but the Ingress Controller has started restarting frequently with the following error:
panic: runtime error: slice bounds out of range [:65283] with capacity 14918
This issue is affecting production stability, and we are looking for guidance on the possible cause and resolution.
Environment Details:
Ingress Controller Version:
NGINX Ingress Controller
nginx version: nginx/1.25.3
Repository: /~https://github.com/kubernetes/ingress-nginx
Build: 4fb5aac
Release: v1.10.1
Kubernetes Version:
v1.30.1
Steps to Reproduce:
The cluster runs with a stable load (10K–30K requests/sec).
No significant changes were made to the ingress configuration or deployment.
The Ingress Controller pod restarts frequently with a panic error.
Actual Behavior:
The Ingress Controller crashes frequently with a runtime panic.
An out-of-bounds slice access.
Relevant Logs:
panic: runtime error: slice bounds out of range [:65283] with capacity 14918
Troubleshooting Steps Taken:
Checked cluster resource utilization (CPU/memory) – no anomalies.
No recent configuration changes or version upgrades.
No sudden spike in traffic beyond normal range.
Observed that the issue occurs randomly without external triggers.
The text was updated successfully, but these errors were encountered: