Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: enabled integration with HA-valkey via network changes and documentation #245

Merged
merged 12 commits into from
Dec 3, 2024
4 changes: 4 additions & 0 deletions charts/config/templates/redis-peerauthentication.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,8 @@ spec:
# Redis/Valkey exception to support GitLab 'dependency' init containers
{{ .Values.redis.port }}:
mode: PERMISSIVE
{{- if .Values.redis.sentinel.enabled }}
{{ .Values.redis.sentinel.port }}:
mode: PERMISSIVE
{{- end }}
{{- end }}
30 changes: 25 additions & 5 deletions charts/config/templates/uds-package.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,11 @@ spec:
remoteNamespace: {{ .Values.redis.namespace | quote }}
remoteSelector:
{{ .Values.redis.selector | toYaml | nindent 10 }}
port: {{ .Values.redis.port }}
ports:
- {{ .Values.redis.port }}
{{- if .Values.redis.sentinel.enabled }}
- {{ .Values.redis.sentinel.port }}
{{- end }}
{{- else }}
remoteGenerated: Anywhere
{{- end }}
Expand All @@ -294,7 +298,11 @@ spec:
remoteNamespace: {{ .Values.redis.namespace | quote }}
remoteSelector:
{{ .Values.redis.selector | toYaml | nindent 10 }}
port: {{ .Values.redis.port }}
ports:
- {{ .Values.redis.port }}
{{- if .Values.redis.sentinel.enabled }}
- {{ .Values.redis.sentinel.port }}
{{- end }}
{{- else }}
remoteGenerated: Anywhere
{{- end }}
Expand Down Expand Up @@ -334,7 +342,11 @@ spec:
remoteNamespace: {{ .Values.redis.namespace | quote }}
remoteSelector:
{{ .Values.redis.selector | toYaml | nindent 10 }}
port: {{ .Values.redis.port }}
ports:
- {{ .Values.redis.port }}
{{- if .Values.redis.sentinel.enabled }}
- {{ .Values.redis.sentinel.port }}
{{- end }}
{{- else }}
remoteGenerated: Anywhere
{{- end }}
Expand Down Expand Up @@ -374,7 +386,11 @@ spec:
remoteNamespace: {{ .Values.redis.namespace | quote }}
remoteSelector:
{{ .Values.redis.selector | toYaml | nindent 10 }}
port: {{ .Values.redis.port }}
ports:
- {{ .Values.redis.port }}
{{- if .Values.redis.sentinel.enabled }}
- {{ .Values.redis.sentinel.port }}
{{- end }}
{{- else }}
remoteGenerated: Anywhere
{{- end }}
Expand Down Expand Up @@ -429,7 +445,11 @@ spec:
remoteNamespace: {{ .Values.redis.namespace | quote }}
remoteSelector:
{{ .Values.redis.selector | toYaml | nindent 10 }}
port: {{ .Values.redis.port }}
ports:
- {{ .Values.redis.port }}
{{- if .Values.redis.sentinel.enabled }}
- {{ .Values.redis.sentinel.port }}
{{- end }}
{{- else }}
remoteGenerated: Anywhere
{{- end }}
Expand Down
3 changes: 3 additions & 0 deletions charts/config/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,9 @@ redis:
app.kubernetes.io/name: valkey
namespace: valkey
port: 6379
sentinel:
enabled: false
port: 26379
postgres:
password: ""

Expand Down
83 changes: 83 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,89 @@ With this override definition one can then provide the IAM role ARNs to the depl

GitLab uses Redis as a key value store for caching, job queueing and more and supports external providers (such as Elasticache) as well as the [UDS Valkey](/~https://github.com/defenseunicorns/uds-package-valkey/) package to provide the service.

### Valkey HA Configuration

The [Valkey UDS Package](/~https://github.com/defenseunicorns/uds-package-valkey) supports the HA replicated architecture ([as of v8.0.1-uds.1](/~https://github.com/defenseunicorns/uds-package-valkey/releases/tag/v8.0.1-uds.1)) where there is one write node (called a primary), multiple read nodes, and sentinels as side-cars who will elect a new primary in the event the existing primary goes down.
This configuration is further [documented in the Valkey repo](/~https://github.com/defenseunicorns/uds-package-valkey/blob/main/docs/configuration.md#high-availability). All configuration changes required to connect an HA Valkey to GitLab will be performed at the _bundle_ level. To connect the HA Valkey to Gitlab:

1. Perform the [configuration changes](/~https://github.com/defenseunicorns/uds-package-valkey/blob/main/docs/configuration.md#configuration-changes) to configure the Valkey Package to deploy an HA instance in your bundle.

2. Change the `global.redis.host` value to be the _name_ of the primary node's role. By default, that is `mymaster`. This value is no longer to be the address for redis.

> [!WARNING]
> This may seem unintuitive until you consider that GitLab will be using the sentinel to find the redis address, but needs to know the name of the primary's role. This value is still key info required in finding the redis host, but the value ends up _not_ being the redis host address.

```yaml
packages:
- name: gitlab
overrides:
gitlab:
gitlab:
values:
- path: global.redis.host
value: mymaster
```

3. _At the bundle level_, override the `global.redis.sentinels` path in the GitLab chart with a list of the valkey sentinel headless addresses, shown below.

```yaml
packages:
- name: gitlab
overrides:
gitlab:
gitlab:
values:
# See https://docs.gitlab.com/charts/charts/globals.html#redis-sentinel-support
# for more details on this section of GitLab's chart.
- path: global.redis.sentinels
value:
- host: valkey-node-0.valkey-headless.<valkey namespace>.svc.cluster.local
port: 26379
- host: valkey-node-1.valkey-headless.<valkey namespace>.svc.cluster.local
port: 26379
- host: valkey-node-2.valkey-headless.<valkey namespace>.svc.cluster.local
port: 26379
```

4. Set `redis.sentinel.enabled` to `true` in `uds-gitlab-config` chart. This will cause the GitLab UDS Package to include add network policies allowing the GitLab services to access the sentinel's port in addition to the read/write ports.

```yaml
packages:
- name: gitlab
overrides:
gitlab:
uds-gitlab-config:
values:
- path: redis.sentinel.enabled
value: true
```

5. Make sure GitLab and Valkey agree on whether auth is required for normal valkey, and whether authentication is required for the sentinel.

```yaml
# The values in the valkey chart
packages:
- name: valkey
overrides:
valkey:
valkey:
namespace: gitlab-valkey
values:
- path: auth.enabled
value: true
- path: auth.sentinel
value: true
- name: gitlab
overrides:
gitlab:
gitlab:
values:
- path: global.redis.auth.enabled
value: true
- path: global.redis.sentinelAuth.enabled
value: true
```

### Manual Keystore Connection

You can use the following Helm overrides to configure a connection to Redis / Valkey:
Expand Down
4 changes: 4 additions & 0 deletions values/common-values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,10 @@ global:
enabled: true
secret: gitlab-redis
key: password
sentinelAuth:
enabled: false
secret: gitlab-redis
key: password

minio:
enabled: false
Expand Down
Loading