From fb525a2b704928a8b6dde0be2ef62f10c97a978d Mon Sep 17 00:00:00 2001 From: Ruslan Khizhnyak Date: Thu, 26 Jun 2025 02:10:36 +0300 Subject: [PATCH 1/5] Add extraDeploy and annotations for auth secret --- charts/redis-ha/Chart.yaml | 2 +- charts/redis-ha/templates/redis-auth-secret.yaml | 3 +++ charts/redis-ha/values.yaml | 6 +++++- 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/charts/redis-ha/Chart.yaml b/charts/redis-ha/Chart.yaml index 3db74db7..2cced456 100644 --- a/charts/redis-ha/Chart.yaml +++ b/charts/redis-ha/Chart.yaml @@ -5,7 +5,7 @@ keywords: - redis - keyvalue - database -version: 4.33.7 +version: 4.33.8 appVersion: 7.2.7 description: This Helm chart provides a highly available Redis implementation with a master/slave configuration and uses Sentinel sidecars for failover management icon: https://upload.wikimedia.org/wikipedia/en/thumb/6/6b/Redis_Logo.svg/1200px-Redis_Logo.svg.png diff --git a/charts/redis-ha/templates/redis-auth-secret.yaml b/charts/redis-ha/templates/redis-auth-secret.yaml index a1fd6311..2855e55f 100644 --- a/charts/redis-ha/templates/redis-auth-secret.yaml +++ b/charts/redis-ha/templates/redis-auth-secret.yaml @@ -4,6 +4,9 @@ kind: Secret metadata: name: {{ template "redis-ha.fullname" . }} namespace: {{ .Release.Namespace | quote }} + {{- with .Values.authSecretAnnotations }} + annotations: {{ . | toYaml | nindent 4 }} + {{- end }} labels: {{ include "labels.standard" . | indent 4 }} {{- range $key, $value := .Values.extraLabels }} diff --git a/charts/redis-ha/values.yaml b/charts/redis-ha/values.yaml index e857f158..00fe4873 100644 --- a/charts/redis-ha/values.yaml +++ b/charts/redis-ha/values.yaml @@ -813,7 +813,8 @@ podDisruptionBudget: {} auth: false # -- (string) A password that configures a `requirepass` and `masterauth` in the conf parameters (Requires `auth: enabled`) redisPassword: ~ - +# -- Annotations for auth secret +authSecretAnnotations: {} ## Use existing secret containing key `authKey` (ignores redisPassword) ## Can also store AWS S3 or SSH secrets in this secret # -- An existing secret containing a key defined by `authKey` that configures `requirepass` and `masterauth` in the conf @@ -1008,3 +1009,6 @@ splitBrainDetection: interval: 60 # -- splitBrainDetection resources resources: {} + +# Array of extra objects to deploy with the release +extraDeploy: [] From 8cfdab4d28f8b5668fbe0da83daab5a09520df68 Mon Sep 17 00:00:00 2001 From: Aaron Layfield Date: Sat, 13 Sep 2025 08:29:03 +0900 Subject: [PATCH 2/5] fix: merge auth and authSecretAnnotations for redis auth secret Signed-off-by: Aaron Layfield --- charts/redis-ha/templates/redis-auth-secret.yaml | 3 ++- charts/redis-ha/values.yaml | 3 +++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/charts/redis-ha/templates/redis-auth-secret.yaml b/charts/redis-ha/templates/redis-auth-secret.yaml index 2855e55f..1553fb3b 100644 --- a/charts/redis-ha/templates/redis-auth-secret.yaml +++ b/charts/redis-ha/templates/redis-auth-secret.yaml @@ -1,10 +1,11 @@ {{- if and .Values.auth (not .Values.existingSecret) -}} +{{ $fullAnnotations := mustMerge .Values.redis.annotations .Values.authSecretAnnotations }} apiVersion: v1 kind: Secret metadata: name: {{ template "redis-ha.fullname" . }} namespace: {{ .Release.Namespace | quote }} - {{- with .Values.authSecretAnnotations }} + {{- with $fullAnnotations }} annotations: {{ . | toYaml | nindent 4 }} {{- end }} labels: diff --git a/charts/redis-ha/values.yaml b/charts/redis-ha/values.yaml index 975c5fb7..b634f19d 100644 --- a/charts/redis-ha/values.yaml +++ b/charts/redis-ha/values.yaml @@ -819,10 +819,13 @@ podDisruptionBudget: {} # -- Configures redis with AUTH (requirepass & masterauth conf params) auth: false + # -- (string) A password that configures a `requirepass` and `masterauth` in the conf parameters (Requires `auth: enabled`) redisPassword: ~ + # -- Annotations for auth secret authSecretAnnotations: {} + ## Use existing secret containing key `authKey` (ignores redisPassword) ## Can also store AWS S3 or SSH secrets in this secret # -- An existing secret containing a key defined by `authKey` that configures `requirepass` and `masterauth` in the conf From d706d3997a6f27ab0ce769632ff41ff801081cfc Mon Sep 17 00:00:00 2001 From: Aaron Layfield Date: Sat, 13 Sep 2025 08:30:55 +0900 Subject: [PATCH 3/5] Updating helm-docs & Removing extraDeploy Signed-off-by: Aaron Layfield --- charts/redis-ha/README.md | 15 ++++++++++----- charts/redis-ha/values.yaml | 3 --- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/charts/redis-ha/README.md b/charts/redis-ha/README.md index daf2e6ff..25f41ac6 100644 --- a/charts/redis-ha/README.md +++ b/charts/redis-ha/README.md @@ -67,6 +67,7 @@ The following table lists the configurable parameters of the Redis chart and the | `affinity` | Override all other affinity settings for the Redis server pods with a string. | string | `""` | | `auth` | Configures redis with AUTH (requirepass & masterauth conf params) | bool | `false` | | `authKey` | Defines the key holding the redis password in existing secret. | string | `"auth"` | +| `authSecretAnnotations` | Annotations for auth secret | object | `{}` | | `configmap.labels` | Custom labels for the redis configmap | object | `{}` | | `configmapTest.image` | Image for redis-ha-configmap-test hook | object | `{"repository":"koalaman/shellcheck","tag":"v0.10.0"}` | | `configmapTest.image.repository` | Repository of the configmap shellcheck test image. | string | `"koalaman/shellcheck"` | @@ -173,6 +174,7 @@ The following table lists the configurable parameters of the Redis chart and the | `serviceLabels` | Custom labels for redis service | object | `{}` | | `splitBrainDetection.interval` | Interval between redis sentinel and server split brain checks (in seconds) | int | `60` | | `splitBrainDetection.resources` | splitBrainDetection resources | object | `{}` | +| `splitBrainDetection.retryInterval` | | int | `10` | | `sysctlImage.command` | sysctlImage command to execute | list | `[]` | | `sysctlImage.enabled` | Enable an init container to modify Kernel settings | bool | `false` | | `sysctlImage.mountHostSys` | Mount the host `/sys` folder to `/host-sys` | bool | `false` | @@ -464,11 +466,14 @@ Should your Pod require additional egress rules, define them in a `egressRules` Under not entirely known yet circumstances redis sentinel and its corresponding redis server reach a condition that this chart authors call "split brain" (for short). The observed behaviour is the following: the sentinel switches to the new re-elected master, but does not switch its redis server. Majority of original discussion on the problem has happened at the . The proposed solution is currently implemented as a sidecar container that runs a bash script with the following logic: -1. At intervals defined by `splitBrainDetection.interval`, the sidecar checks which node is recognized as master by Sentinel. -2. If the current pod is the master according to Sentinel, it verifies that the local Redis server is also running as master. -3. If the current pod is not the master, it ensures the local Redis server is replicating from the correct master node. -4. If any of these checks fail, the sidecar will retry the check at intervals defined by `splitBrainDetection.retryInterval`. -5. If the checks continue to fail after the retry attempts, the sidecar triggers a reinitialization: it regenerates the Redis configuration and instructs the Redis server to shut down. Kubernetes will then automatically restart the container. + +1. Every `splitBrainDetection.interval` seconds a master (as known by sentinel) is determined +1. If it is the current node: ensure the redis server's role is master as well. +1. If it is not the current node: ensure the redis server also replicates from the same node. + +If any of the checks above fails - the redis server reinitialisation happens (it regenerates configs the same way it's done during the pod init), and then the redis server is instructed to shutdown. Then kubernetes restarts the container immediately. + +# Change Log ## 4.14.9 - ** POTENTIAL BREAKING CHANGE. ** Introduced the ability to change the Haproxy Deployment container pod diff --git a/charts/redis-ha/values.yaml b/charts/redis-ha/values.yaml index b634f19d..626c40fb 100644 --- a/charts/redis-ha/values.yaml +++ b/charts/redis-ha/values.yaml @@ -1021,6 +1021,3 @@ splitBrainDetection: retryInterval: 10 # -- splitBrainDetection resources resources: {} - -# Array of extra objects to deploy with the release -extraDeploy: [] From ff0a4a6a2a47e0617380b5eb742dbda27ec2b1a4 Mon Sep 17 00:00:00 2001 From: Aaron Layfield Date: Sat, 13 Sep 2025 08:32:32 +0900 Subject: [PATCH 4/5] chore: bump version to 4.34.7 in Chart.yaml Signed-off-by: Aaron Layfield --- charts/redis-ha/Chart.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/charts/redis-ha/Chart.yaml b/charts/redis-ha/Chart.yaml index aef9b856..53e8414b 100644 --- a/charts/redis-ha/Chart.yaml +++ b/charts/redis-ha/Chart.yaml @@ -5,7 +5,7 @@ keywords: - redis - keyvalue - database -version: 4.34.6 +version: 4.34.7 appVersion: 8.2.1 description: This Helm chart provides a highly available Redis implementation with a master/slave configuration and uses Sentinel sidecars for failover management icon: https://img.icons8.com/external-tal-revivo-shadow-tal-revivo/24/external-redis-an-in-memory-data-structure-project-implementing-a-distributed-logo-shadow-tal-revivo.png From a14697db8131d44621bbc40ad9f6dbfce9da5ea9 Mon Sep 17 00:00:00 2001 From: Aaron Layfield Date: Sat, 13 Sep 2025 08:39:55 +0900 Subject: [PATCH 5/5] Fixing README instructions for split-brain Signed-off-by: Aaron Layfield --- charts/redis-ha/README.md | 12 ++++++------ charts/redis-ha/README.md.gotmpl | 12 ++++++------ 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/charts/redis-ha/README.md b/charts/redis-ha/README.md index 25f41ac6..40c3e580 100644 --- a/charts/redis-ha/README.md +++ b/charts/redis-ha/README.md @@ -463,15 +463,15 @@ Should your Pod require additional egress rules, define them in a `egressRules` ## Sentinel and redis server split brain detection -Under not entirely known yet circumstances redis sentinel and its corresponding redis server reach a condition that this chart authors call "split brain" (for short). The observed behaviour is the following: the sentinel switches to the new re-elected master, but does not switch its redis server. Majority of original discussion on the problem has happened at the . +Under not entirely known yet circumstances redis sentinel and its corresponding redis server reach a condition that this chart authors call "split brain" (for short). The observed behaviour is the following: the sentinel switches to the new re-elected master, but does not switch its redis server. Majority of original discussion on the problem has happened at the #121. The proposed solution is currently implemented as a sidecar container that runs a bash script with the following logic: -1. Every `splitBrainDetection.interval` seconds a master (as known by sentinel) is determined -1. If it is the current node: ensure the redis server's role is master as well. -1. If it is not the current node: ensure the redis server also replicates from the same node. - -If any of the checks above fails - the redis server reinitialisation happens (it regenerates configs the same way it's done during the pod init), and then the redis server is instructed to shutdown. Then kubernetes restarts the container immediately. +1. At intervals defined by splitBrainDetection.interval, the sidecar checks which node is recognized as master by Sentinel. +2. If the current pod is the master according to Sentinel, it verifies that the local Redis server is also running as master. +3. If the current pod is not the master, it ensures the local Redis server is replicating from the correct master node. +4. If any of these checks fail, the sidecar will retry the check at intervals defined by splitBrainDetection.retryInterval. +5. If the checks continue to fail after the retry attempts, the sidecar triggers a reinitialization: it regenerates the Redis configuration and instructs the Redis server to shut down. Kubernetes will then automatically restart the container. # Change Log diff --git a/charts/redis-ha/README.md.gotmpl b/charts/redis-ha/README.md.gotmpl index dedbf282..28266e3e 100644 --- a/charts/redis-ha/README.md.gotmpl +++ b/charts/redis-ha/README.md.gotmpl @@ -214,15 +214,15 @@ Should your Pod require additional egress rules, define them in a `egressRules` ## Sentinel and redis server split brain detection -Under not entirely known yet circumstances redis sentinel and its corresponding redis server reach a condition that this chart authors call "split brain" (for short). The observed behaviour is the following: the sentinel switches to the new re-elected master, but does not switch its redis server. Majority of original discussion on the problem has happened at the . +Under not entirely known yet circumstances redis sentinel and its corresponding redis server reach a condition that this chart authors call "split brain" (for short). The observed behaviour is the following: the sentinel switches to the new re-elected master, but does not switch its redis server. Majority of original discussion on the problem has happened at the #121. The proposed solution is currently implemented as a sidecar container that runs a bash script with the following logic: -1. Every `splitBrainDetection.interval` seconds a master (as known by sentinel) is determined -1. If it is the current node: ensure the redis server's role is master as well. -1. If it is not the current node: ensure the redis server also replicates from the same node. - -If any of the checks above fails - the redis server reinitialisation happens (it regenerates configs the same way it's done during the pod init), and then the redis server is instructed to shutdown. Then kubernetes restarts the container immediately. +1. At intervals defined by splitBrainDetection.interval, the sidecar checks which node is recognized as master by Sentinel. +2. If the current pod is the master according to Sentinel, it verifies that the local Redis server is also running as master. +3. If the current pod is not the master, it ensures the local Redis server is replicating from the correct master node. +4. If any of these checks fail, the sidecar will retry the check at intervals defined by splitBrainDetection.retryInterval. +5. If the checks continue to fail after the retry attempts, the sidecar triggers a reinitialization: it regenerates the Redis configuration and instructs the Redis server to shut down. Kubernetes will then automatically restart the container. # Change Log