Skip to content

Merge upstream Saremox/main (135 commits) - #32

Closed
usiegj00 wants to merge 136 commits into
mainfrom
chore/merge-upstream-saremox-2026-10
Closed

usiegj00 wants to merge 136 commits into
mainfrom
chore/merge-upstream-saremox-2026-10

Conversation

@usiegj00

Copy link
Copy Markdown
Collaborator

Syncs the fork with Saremox/main. The last sync was 2026-09-19; this brings
in 135 commits from 2026-10-01 to 2026-10-20. After it, main is 0 behind
upstream.

Most of the merge is upstream taking our work back

Upstream adopted the two largest pieces of this fork, then improved them:

Upstream also brings the autoscaler, the soak tester, a status subresource,
and a large documentation and lint pass.

What stays different, and why

Difference Reason
The instance manager is wired in Upstream builds the binary but does not use it. Here an init container copies it to a shared volume, the Redis container runs it as PID 1, and the liveness and readiness probes are HTTP probes against its health server.
No preStop hook in operator-managed mode Upstream still runs a synchronous SAVE there. It is redundant with the save points and the manager's SIGTERM forwarding, it eats the grace period before the manager's own 25s window starts, and against an instance already in a BGSAVE it never returns. A production replica stopped this way.
The operator image stays Alpine The init container above runs cp, and the distroless image upstream moved to has no cp.
ghcr.io/buildio, not ghcr.io/saremox The images, the chart defaults, the kustomize overlays and the release checks all point at this fork.

The fork wiring now lives in one file,
operator/redisfailover/service/generator_instance_manager.go, so the next
sync has one place to look instead of a diff spread through generator.go.

Conflicts

25 files. cmd/instance, checker.go, generator.go, go.mod and the
integration tests took upstream's side, because upstream's version contains
ours. types.go and defaults.go took upstream's documentation plus our
InstanceManagerImage field. The Dockerfile, the chart values and the
manifests kept this fork's choices, each with a comment saying why.

Generated files

The CRD and the DeepCopy code are regenerated with controller-gen v0.20.1.
Only the instanceManagerImage description changed; the k8s 0.37.0 → 0.37.1
bump changed no schema.

The golden files are rewritten. Only the Redis StatefulSets changed in them,
which is exactly what the instance manager touches, and the Sentinel-mode
golden files keep their preStop hook while the sentinel-less ones lose it.

Verification

go build ./...          exit 0
go vet ./...            no output
gofmt -l .              no output
go test ./...           all packages ok, except service/redis
make helm-test          chart templating ok; check-version needs bash 4

service/redis fails the same 9 tests on a clean checkout of upstream/main
on this machine, so the failures come from the environment, not the merge.
check-version uses mapfile, which macOS bash 3.2 does not have. Both run
on CI.

Version

Chart, manifests and examples move to 4.2.0-rc2 with upstream.
DefaultInstanceManagerImage stays at ghcr.io/buildio/redis-operator:4.1.0,
the last image this fork published, because the init container pulls it at
runtime. Bump it when the fork cuts its next release.

Left for later

test/soak still pulls the upstream OCI charts, because buildio publishes
none. The soak workflow is workflow_dispatch only and never runs on a pull
request.

dependabot Bot and others added 30 commits October 1, 2026 10:28
Bumps [k8s.io/api](https://github.com/kubernetes/api) from 0.37.0 to 0.37.1.
- [Commits](kubernetes/api@v0.37.0...v0.37.1)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-version: 0.37.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.37.0 to 0.37.1.
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](kubernetes/client-go@v0.37.0...v0.37.1)

---
updated-dependencies:
- dependency-name: k8s.io/client-go
  dependency-version: 0.37.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Valkey can't load the RDB format of Redis 7.4 and later. Changing the
image from those versions leaves the first Valkey replica unable to
sync, so the rollout never completes while the master stays on Redis.
Found by the soak tester on Redis 7.4.11 and 8.10.2 to Valkey 7.2, 8.1
and 9.1.


Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

Co-authored-by: Claude <noreply@anthropic.com>
…remox#215)

minimum.yaml had no spec, which the CRD requires, and
bootstrapping-with-port.yaml set bootstrapNode.port as a number where
the field is a string. The API server rejected both.

A new test decodes every RedisFailover in example/redisfailover the way
the API server does, without type coercion and rejecting unknown
fields, and runs Validate on it.


Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

Co-authored-by: Claude <noreply@anthropic.com>
…x#210)

In operator-managed mode, a scale-down that removes the master could
lose acknowledged writes. GetNumberMasters still counted the old master,
so the reconcile took the one-master branch. The StatefulSet (Parallel
pod management) then marked the master pod for deletion, and
CheckMasterHealth's GetMasterIP, which skips terminating pods, found no
master and reported it unhealthy with an empty IP. The branch promoted a
replica at once, while the old master kept taking writes on existing
connections until its clients were disconnected 2-3s later. Those
writes were lost when it was demoted or deleted.

The no-master branch already guards against this with
masterPodStopping. Apply the same guard in the unhealthy-master branch
when no master was found, so the operator waits for the stopping master
to exit before promoting. An unreachable master on a live pod still
fails over as before.


Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

Co-authored-by: Claude <noreply@anthropic.com>
…aremox#207)

The CRD had no status subresource (`subresources: {}`), so
UpdateRedisFailoverStatus's merge patch went to the main resource and
every status change bumped metadata.generation. A fresh RF was already
at generation 3-4 with no spec change, and anything that follows
generation to detect spec changes (GitOps tools, kubectl wait, the soak
tester's convergence window) saw a new generation on every status
transition.

Enable the status subresource on the CRD (+kubebuilder:subresource:status)
and send the same merge patch, still with every status field, to the
"status" subresource. The finalizers patch stays on the main resource,
which is the only other RF write the operator makes. Grant the operator
redisfailovers/status in the chart, kustomize and example ClusterRoles.

The CRD manifests carry only the subresource change: a full
`make generate-crd` also pulls in unrelated schema drift from the
k8s.io/api bump. The README notes that the CRD and the operator must be
upgraded together: an older operator's status writes are dropped
against the new CRD, and the new operator's status patches 404 against
the old one. Existing RFs keep their stored status.


Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

Co-authored-by: Claude <noreply@anthropic.com>
* fix(redis): report a pod rollout stalled on the same pod

UpdateRedisesPods holds the rollout while a replica is not synced with
the master, and until the pods already replaced are ready. That is the
right gate, but nothing told the user when it never opens: a new image
that cannot load the master's data, e.g. Valkey on a Redis 7.4 or 8
RDB, retries the full sync forever with its link down, and the
RedisFailover stayed Healthy with an empty message.

Track in memory what the rollout waits on (the pod, by UID) and since
when, as the per-RF password state does. Once it has waited on the same
pod for more than 10 minutes, the status message names the pod and the
reason, e.g. "rollout waiting on pod rfr-x-1 for more than 10m: not
synced with the master". A replica that is not synced only counts while
a rollout is pending. The timer starts over when the pod is recreated
and is dropped once the rollout moves on, so the message clears.

The state stays Healthy, like the other deliberate waits that only set
a message (maxmemory kept, an in-place resize waiting): the master keeps
serving on the old version. The bound is the operator's own replication
check (SlaveIsReady), so it holds whether or not the pod's readiness
reflects the replication link, and is set above a full sync of a
dataset of tens of GB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* test(redis): cover the stalled rollout report's other waits and errors

Share the rollout test's setup and add cases for the waits in
redisPodsSettled (a replaced pod not ready, a pod terminating, a pod
missing), a message already set (maxmemory) that the report appends to,
and a failed update revision or pod lookup, which is returned and does
not restart the wait.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
…ox#205)

* fix(redis): make a replica without the master's data not ready

ready.sh failed a replica only while a full sync was in progress or while
it still pointed at the 127.0.0.1 placeholder. Between full-sync retries
neither holds, so a replica that can never load the master's data stayed
Ready with master_link_status:down and an empty dataset, and served empty
reads behind rfrs-<name>. Changing the image from Redis 7.4/8 to Valkey
does this: Valkey can't read RDB format 12/15 and retries the sync
forever.

Redis 7.2, 7.4 and 8 and Valkey 8 and 9 all report
master_link_down_since_seconds while the link is down: -1 when it hasn't
been up since the server started, otherwise the seconds since it dropped.
Failed sync attempts don't reset it. A replica whose link is down is now
ready only if that value is between 0 and 60, so it never counts as
ready before its first sync, and it stays ready through a failover.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(redis): size the readiness link-down window from the failover timing

ready.sh kept a replica ready for a fixed 60s after its link to the
master dropped. With Sentinel's down-after-milliseconds or
failover-timeout set above that in spec.sentinel.customConfig, or a long
spec.sentinel.failoverTimeout in operator mode, every replica went
NotReady and rfrs emptied before the failover had even started.

The window is now generated per RedisFailover: 60s plus failoverTimeout
without Sentinel, or plus the effective down-after-milliseconds and
failover-timeout with it (the operator's defaults of 5000 and 10000 unless
customConfig sets them, parsed the way the operator applies them). The
defaults give 70s and 75s. EnsureRedisReadinessConfigMap updates the
ConfigMap when its data changes, and the pods mount it without subPath,
so a changed setting reaches running pods.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
It only runs when the wait is logged, which the other tests do with the
dummy logger, so it was the one uncovered function in the package.


Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

Co-authored-by: Claude <noreply@anthropic.com>
…bes (Saremox#217)

* fix(redis): bound redis-cli in the readiness script

A frozen (SIGSTOPped) Redis still accepts TCP connections into the
kernel backlog, so every redis-cli call in ready.sh waits for a reply
forever. The probe's 5s timeout doesn't stop it on containerd: the
runtime kills only the probe's top-level shell and then waits for the
exec output to drain, which the orphaned redis-cli, xargs and grep
children keep open. kubelet gives up only at the CRI call deadline,
the probe timeout plus two minutes, so each probe takes about 125s and
three failures are needed. The frozen master's pod stays Ready for
over six minutes, and operator-managed failover, which waits while the
master pod is Ready, is blocked for that time.

Run every redis-cli call under `timeout 2`. A frozen server now fails
the probe on its own in about 4s, within the probe timeout, on any
runtime. timeout ships next to tr, which the script already needs, in
coreutils and busybox, and is in the official Redis and Valkey images.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(redis): keep ready.sh working on images without timeout

A custom redis image that has the script's other tools but no timeout
binary failed every readiness check with "timeout: not found", so all
Redis pods went unready. Prefix redis-cli with "timeout 2" only when
command -v finds timeout; without it the script keeps the previous,
unbounded behaviour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(redis): bound redis-cli in the default liveness probes

The default Redis liveness probe, `redis-cli ... ping | grep PONG`,
hangs on a frozen server just like ready.sh did: containerd kills only
the probe's sh at the 5s timeout and then waits for the orphaned
redis-cli and grep to release the exec output. kubelet gives up at the
CRI deadline, so each probe took about 125s and six failures were
needed, about 12.5 minutes before a frozen pod was restarted.

The Sentinel liveness probe has the same unbounded redis-cli. Busybox
sh execs it, so the runtime timeout works on Alpine images, but dash
forks it, which leaves the same orphan on Debian images.

Run redis-cli under `timeout 2` in both, with the same fallback to the
bare command as ready.sh when the image has no timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
…x#221)

docker/app/Dockerfile built the published operator image from the mutable
tags golang:1.26-alpine and alpine:latest. A re-pointed or compromised
upstream tag would flow straight into the operator, which runs in-cluster
with its ClusterRole.

Pin both stages to a tag plus its multi-arch index digest, the same pins
the soak tester image uses. alpine:3.24.2 is the latest 3.x patch and the
same index alpine:latest resolves to today, so the runtime is unchanged.
Add a dependabot docker entry for /docker/app so the pins get bumped.


Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

Co-authored-by: Claude <noreply@anthropic.com>
…aster check fails (Saremox#219)

In operator-managed mode, checkAndHealOperatorManagedMode returned the
error from masterPodStopping as is, both with no master counted (case 0)
and when the counted master is no longer found (case 1, added in Saremox#210).
rf.Status still held the HealthyState that CheckAndHeal initialised, so
the deferred status update published a healthy RedisFailover although
the operator couldn't tell whether an election was safe.

Set NotHealthyState with "unable to check whether the master is
stopping" before returning the error, as Sentinel mode does (Saremox#209).


Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

Co-authored-by: Claude <noreply@anthropic.com>
…x#218)

The default Sentinel readiness probe pipes two redis-cli calls through
head and grep with no limit. A frozen sentinel still accepts the
connection, so redis-cli waits forever, and the orphaned pipeline
keeps the exec output open past the probe's 5s timeout on containerd,
as Saremox#217 found for the Redis probes.

Run both calls under `timeout 2` when the image has timeout, with the
same fallback to the bare command as the other default probes. A
frozen sentinel now fails the probe in about 2s.


Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

Co-authored-by: Claude <noreply@anthropic.com>
…pp (Saremox#222)

Bumps golang from 1.26-alpine to 1.27-alpine.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.27-alpine
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…#208)

* fix(crd): regenerate the CRD manifests for k8s.io/api 0.37

The CRD embeds the schema of the core types RedisFailover references
(probes, containers, volumes, volume mounts, PVC templates). The bump
from k8s.io/api 0.35.0 to 0.37.0 in Saremox#156 changed those types, but the
manifests were never regenerated, so the API server prunes fields such
as httpGet.protocol, grpc.mode, volumeMounts[].bindMountOptions,
emptyDir.mode and the volume user/defaultUser fields from RedisFailovers.
The 0.37.0 -> 0.37.1 bump (Saremox#197) does not change the output.

Regenerated with `make generate-crd` and controller-gen v0.20.1. The
change is additive: no property, required entry, type, format or enum is
removed or changed; the rest are description updates, including the
sentinel.enabled default that types.go already documents.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* ci(codegen): verify the generated CRD manifests too

verify-codegen only checked DeepCopy, so the CRD manifests went stale
after a k8s.io/api bump without CI noticing. It now also runs
generate-crd and fails on a diff in any of the three CRD copies.

controller-gen is pinned to v0.20.1 (CONTROLLER_GEN_VERSION, installed
with `make install-controller-gen`) instead of @latest, since the output
is compared byte for byte and records the generator version. With a
fixed version and go.mod the output is deterministic. The pre-commit
hook also runs on go.mod changes now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
…xt (Saremox#230)

A partial pod securityContext kept the default user, group, runAsNonRoot
and fsGroup, but dropped the RuntimeDefault seccomp profile.


Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

Co-authored-by: Claude <noreply@anthropic.com>
* docs(readme): fix drift from the code on main

State the CI test matrix, the operator's security-context, annotation,
label and service-account defaults, and the resources it creates. Fix
the topologySpreadConstraints and customConfig field names. Reword the
minimal kustomize overlay and the bootstrap replica-priority note.

Add caveats: the kubectl manifest needs the Prometheus Operator CRDs for
its monitors, a custom redis.command must pass the auth flags itself, and
the default readiness probe does not check replication while a pod still
has the old password.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: show the shell wrapper for an auth custom command, and the bootstrap Sentinel gate

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: eval <eval@example.invalid>
* docs: fix Go comments that drifted from the code

Correct doc comments that name the wrong identifier or resource, or
describe behaviour that changed. Only comments change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: state that no PDB spec field is defaulted

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: eval <eval@example.invalid>
* docs: fix drift in the logic, development and agent guides

- docs/logic.md: add the pipeline steps before Ensure, the missing
  resources, the quorum rule, and the operator-managed, Sentinel and
  bootstrap modes.
- docs/development.md: fix the folder list and the test targets.
- .github/copilot-instructions.md: operator-managed failover is the
  default; use make generate-api and verify-codegen; list all CI checks.
- kind-cluster skill: subnet octets must be 2 apart; kind-up.sh does
  not pass --retain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: name the PDB and service account gates and the quorum limit

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: eval <eval@example.invalid>
…NGELOG with migration guides (Saremox#225)

* docs(crd): document the limits of command, maxMemory and inPlaceResize

A custom redis command replaces the default one, which passes the
auth.secretPath password to redis-server. Without --requirepass and
--masterauth from $REDIS_PASSWORD, Redis runs with no password.

maxMemory needs a memory limit of at least 64Mi, uses the smallest pod
limit and forces replica-ignore-maxmemory yes. inPlaceResize needs
Kubernetes 1.33, and 1.35 to lower a memory limit; otherwise the pods
are recreated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs(chart): correct the monitoring and CRD upgrade hook comments

monitoring.enabled creates a metrics Service and, with serviceMonitor, a
ServiceMonitor, not a PodMonitor. The CRD upgrade hook Job has its own
ServiceAccount and ClusterRole, not the operator's. An empty hook image
tag uses v1.36.2, which the template pins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs(changelog): list the changes on main after 4.2.0-rc2

The newest entry was v1.1.0-rc.3 from 2022, and no release notes cover
the changes merged after 4.2.0-rc2. Saremox#207 and Saremox#208 need a CRD re-apply,
and Saremox#207 needs get and patch on redisfailovers/status, so the upgrade
notes come first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs(changelog): add Saremox#230

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: replace CHANGELOG.md with migration guides

The GitHub releases list the changes in each release. CHANGELOG.md
stopped at v1.1.0-rc.3 and duplicated the release notes, so remove it.

Manual upgrade steps go to docs/migrations/<version>.md. The guide for
4.2.0 takes the upgrade notes from the old Unreleased section: the CRD
re-apply for Saremox#207 and Saremox#208, the redisfailovers/status RBAC, and the
pod restart from Saremox#230. Each step is a numbered procedure with the
reason. The README links to the release notes and to the guides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* ci(release): warn in the release notes when a migration guide exists

If docs/migrations/<version>.md exists at the tag, the release notes
start with a warning that links to the guide. The version is the tag
without the -rcN suffix, so the guide for 4.2.0 also applies to
4.2.0-rc3. The step prepends to release-notes.md, which the release
action reads through body_path, so the generated notes stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: eval <eval@example.invalid>
…aremox#223)

With serviceAccount.name set and create true, the Deployment used the
name, but the chart created the ServiceAccount and the binding subject
as the fullname. The pod then referred to a ServiceAccount that did
not exist. Use the chart.serviceAccountName helper for both.

The values.yaml comment now states that create false also skips the
ClusterRole and the ClusterRoleBinding.


Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

Co-authored-by: Claude <noreply@anthropic.com>
…me (Saremox#229)

* fix(redis): reach Sentinel from the shutdown script by its Service name

The default shutdown script found Sentinel only through the service-link
variables RFS_<NAME>_SERVICE_HOST and RFS_<NAME>_SERVICE_PORT_SENTINEL.
Since Saremox#148 the redis pods set enableServiceLinks: false, so these
variables are empty. Thus the script never reached Sentinel: on a
graceful delete of the master it did not ask for a failover, and
Sentinel moved the master only after down-after-milliseconds. In
operator-managed mode the script also spent about 9s on three failed
attempts before SAVE.

The script now uses the Sentinel Service name rfs-<NAME> and port
26379. Service links stay off, because Saremox#148 disabled them on purpose.
When the RedisFailover runs no Sentinels (SentinelsAllowed is false),
the script has no Sentinel part and only runs SAVE. The custom shutdown
example and the README use the Service name too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(redis): end the shutdown script inside the grace period

After the failover request, the shutdown script ran "sleep 31". This is
more than the default terminationGracePeriodSeconds of 30. Thus the
kubelet stopped the hook (FailedPreStopHook), and the SAVE in the script
did not run.

The script now asks Sentinel for the master about once per second and
stops when Sentinel reports a different master, or at a deadline 15s
after the script starts. Each sentinel call has a 2s limit through
"timeout", if the image has it, so a frozen Sentinel cannot hold the
hook. The retry sleeps are 1s. The worst case for the Sentinel part is
18s: the lookup and failover loops take at most 8s + 9s = 17s, and the
last poll starts before 15s and takes at most 1s + 2s. This leaves at
least 12s of the default grace period for SAVE and the redis shutdown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(redis): pause the writes on the master during the shutdown failover

A default script that reaches Sentinel can lose acknowledged writes on a
graceful master delete. Until Sentinel promotes a replica, the old
master still accepts writes, and the promotion loses those that did not
reach the replica. The Sentinel shutdown test measured 3 and 162 lost
writes.

When the pod is the master, the script now pauses the writes with
CLIENT PAUSE WRITE before it asks Sentinel to fail over. When Sentinel
reports the new master, the script makes the old master a replica of
it, and then ends the pause. The clients that still reach the old
master then get READONLY and not an acknowledged write that is lost.
When the failover request fails or the poll reaches its deadline, the
script only ends the pause. CLIENT PAUSE WRITE needs Redis 6.2; on an
earlier version the script continues without the pause.

Sentinel has no password, so only the local calls send REDIS_PASSWORD
(REDISCLI_AUTH). redis-cli stops when its AUTH fails.

The deadline is 12s and the failover request stops at it too. The last
failover request or poll starts before 12s and ends before 15s;
REPLICAOF and CLIENT UNPAUSE add a maximum of 4s. Thus the Sentinel part
ends before 19s, which leaves more than 11s of the default 30s grace
period for SAVE and the redis shutdown. The pause stops 6s after the
deadline (before 19s), also if the script stops.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
…aster (Saremox#220)

* fix(failover): skip terminating and prefer Ready pods in SetOldestAsMaster

SetOldestAsMaster sorted every StatefulSet pod by creation time and made
the first one master. It did not look at DeletionTimestamp or readiness,
so it could promote a pod that is being deleted (scale-down, rolling
replacement), whose master role then vanishes right after the election,
or a pod that is not Ready while a Ready one exists. It runs as the
fallback when GetBestReplicaForPromotion fails and in the Sentinel-mode
no-master paths (single replica, no Sentinel quorum, localhost master).

Drop terminating pods from the candidates and order the rest Ready
first, then oldest first. With no Ready pod, as on first boot, the
oldest non-terminating pod still wins. If every pod is terminating, the
candidate list is empty and the existing "number of redis pods are 0"
error is returned. Terminating pods are no longer repointed as replicas,
matching PromoteBestReplica.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(failover): don't elect a master while the old one is stopping

GetNumberMasters skips pods with a DeletionTimestamp, so in Sentinel
mode a Ready master that is terminating counts as no master. With no
Sentinel quorum the heal then calls SetOldestAsMaster, which now drops
terminating pods and would promote another pod while the old master
still takes writes, leaving two writable masters.

SetOldestAsMaster now returns without promoting when a terminating pod
is still Ready and labelled master, the same check masterPodStopping
uses, so the next reconcile elects once it has gone. A terminating
master that is not Ready (lost node) does not block the election. The
operator-managed fallback already ran masterPodStopping, so the guard
is a no-op there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: rewrite SetOldestAsMaster comments in ASD-STE100

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: shorten SetOldestAsMaster comments

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: describe SetOldestAsMaster choice

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
…decrease (Saremox#204)

* fix(redis): recreate a pod at once when the kubelet refuses a memory decrease

On Kubernetes 1.35 the kubelet refuses to lower a memory limit below the
pod's or container's current usage, which includes the page cache. It
reports this as the PodResizeInProgress condition with reason Error and
the message "cannot decrease memory limits: attempting to set ... memory
limit (N) below current usage (M)", and retries with the same result
while the usage stays. The operator treated it like any other resize
error and waited the full 5 minute timeout per pod before recreating it,
so lowering the memory of a 3-pod instance took over 18 minutes.

The condition's type and reason are shared with every other resize
error, so the refusal is recognised by the "below current usage" part of
the kubelet's message, and the pod is recreated right away. Other
deferred or failing resizes keep the timeout. The managed maxmemory
gate still runs before any pod is resized or recreated, so a pod only
gets the smaller limit once the data fits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(redis): recreate only for a refusal of the requested memory limit

A "below current usage" refusal can be left from a superseded resize,
e.g. a lower limit the user raised again, until the kubelet reports on
the new request. podCondition only drops conditions whose
ObservedGeneration is set and older than the pod's, so without one the
stale refusal recreated the pod at once.

The kubelet sets ObservedGeneration on the resize conditions (the
PodObservedGenerationTracking gate is GA and locked on in 1.35), so a
refusal podCondition keeps with one set is current. Without one, the
refused limit in the message must be the requested redis container
limit or the pod's total; otherwise the existing timeout applies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(redis): match a refused memory decrease for every container and the whole pod

Without an observed generation, a "below current usage" refusal is tied
to the current request by the limit it names. That only checked the
redis container's limit and a pod limit summed over the template's
containers. The kubelet's pod limit also counts containers injected
outside the template (e.g. a service mesh sidecar), native sidecars,
init containers and the RuntimeClass overhead, so a refusal of the pod
limit alone waited out the 5 minute timeout, and a refusal of the
exporter's limit did too, unlike with a generation.

Every container the update resizes is now matched by its own limit, and
the pod limit is derived from the pod's own containers as the kubelet's
PodLimits does, with the desired limits applied. A pod with pod-level
resources or a container without a memory limit has no pod form to
match and keeps the timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: rewrite the in-place resize refusal docs in simplified technical English

Split the README condition sentence into a list and keep the page-cache
reason. Shorten the comments on the memory refusal check to the reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
* fix(failover): wait for a stopping master in Sentinel mode

GetNumberMasters skips pods that are being deleted, so while the master
pod terminates but is still Ready and taking writes, the Sentinel-mode
heal sees no master. With a single replica it then calls
SetOldestAsMaster straight away, and without a Sentinel quorum it picks
a master itself as well, while the old master still accepts writes.
Those writes are lost once the old master exits.

Operator-managed mode already waits for this case via masterPodStopping.
Apply the same guard at the top of the Sentinel-mode no-master branch.
It only blocks while the stopping master pod is still Ready, so first
boot and a master on a lost node are unaffected, and it leaves the
failover the shutdown script requests from Sentinel to Sentinel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(failover): report NotHealthy when the stopping-master check fails

When listing the Redis pods failed, the Sentinel-mode no-master guard
returned the error with rf.Status still at the Healthy state CheckAndHeal
starts from, so the deferred status update published Healthy although no
master was found and the operator couldn't tell whether electing one was
safe. Set NotHealthy with a message before returning the error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: state the why of the Sentinel-mode stopping-master guard in STE

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
…emox#203)

* fix(failover): promote a Ready replica before a restarted old master

GetBestReplicaForPromotion only counted a replica as ready when its
replication link was up and not syncing. When the master pod is deleted
gracefully in operator-managed mode (rolling update, kill), no replica has
its link up, so the selection fell back to the highest offset among all
replicas. The old master's pod restarts from the RDB it saved on shutdown,
holds the highest offset and won, although its readiness probe kept it out
of rfrm-<name> for its initial delay: clients had no master for 42-46s
while a Ready, synced replica was available.

Record whether each replica's pod is Ready and rank candidates by pod
readiness first, then by a synced link, and by the highest offset among
equally ranked ones. With no Ready pod (e.g. all pods restarting) the
selection is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(failover): rank a higher offset above pod readiness

The first version ranked a Ready pod above a higher replication offset.
If the master's container restarts in place, it keeps writes the lagging
replicas never got and starts NotReady; a replica still Ready would be
promoted and the old master would full-sync from it, dropping
acknowledged writes main would have kept.

Rank candidates by a synced link, then by the highest offset, and use pod
readiness only to break a tie between equal offsets. After a graceful
shutdown the restarted old master's offset equals its replica's, and the
strict comparison let the first pod in the list, the old master, win;
the tie now goes to the Ready replica.

Rename ReplicaInfo.IsReady to Synced so it isn't confused with PodReady,
compare candidates pairwise instead of recomputing a rank, log every
candidate once per election, and correct the comment in checker.go
that described the selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: rewrite promotion-ranking comments in Simplified Technical English

Keep the reasons for the rank order (data safety first, then a Ready
pod for the master Service) and remove text that repeats the code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
…or it (Saremox#211)

* fix(failover): replace an unsynced stale replica instead of waiting for it

UpdateRedisesPods waited for every replica to be synced with the master
before it replaced any stale pod. After a failed image change is
reverted, e.g. Redis 7.4 -> Valkey 8 -> Redis 7.4, the Valkey replica
can't load the master's RDB and never syncs, but it is also the one
stale pod left. The rollout waited on the very pod it had to replace and
stayed stuck until someone deleted that pod by hand.

A replica that isn't synced and is on a stale revision has no data the
master lacks and is due for replacement anyway, so replace it, before
any synced stale replica. The other rules stay: the master is never
replaced this way (nor anything while the master is unknown), one pod at
a time, the last replacement has to settle first, and a replica syncing
on the update revision is still waited for. The replacement is on the
update revision, so a pod is replaced at most once and a slow sync can't
cause a replace loop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* test(failover): cover the unsynced stale replica selection

Covers the error returns while picking an unsynced stale replica
(update revision, pod list, pod revision), an unsynced replica without a
listed pod (no IP or terminating), which is waited for, and the
bootstrapping path, where no master is known but a stale unsynced
replica is still replaced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(failover): re-check the role of an unsynced stale replica before replacing it

A master has no upstream link, so it fails the sync check too. If
sentinel promotes a pod after GetMasterIP returned and before the loop
reaches it, and that pod is on a stale revision, it was collected as an
unsynced stale replica and deleted through the replica path, which skips
the master gate. The old path got its candidates from
GetRedisesSlavesPods, which reads each pod's current role.

Right before deleting an unsynced stale candidate, read the roles again
with GetRedisesSlavesPods and only delete the pod if it is still a
replica. If it is a master now, or a role can't be read, wait for the
next reconcile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: note the unsynced stale replica in the pod update order

The "Pod updates" section from Saremox#206 said every replica must be synced
before a pod is replaced. A replica that isn't synced and is on a stale
revision is now replaced first instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: rewrite the unsynced stale replica docs in Simplified Technical English

Keep the reasons in the README and the comments, and remove text that
repeats the code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix: wait for an unsynced stale replica while bootstrapping

While bootstrapping, every pod replicates from the external bootstrap
node. If that node is not available, an unsynced pod can hold the only
copy of the data, so the rollout waits as on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
…er (Saremox#231)

* docs: describe write loss in a Sentinel failover with a live old master

Sentinel does not fence the old master, so writes that it acknowledges
between the promotion and its demotion are lost. Users need the size of
this window, which paths it affects, and the Redis settings that reduce
it. The default shutdown script does not reach Sentinel, because the
pods get no service-link variables, so a graceful master deletion does
not open the window. The note also records that the wait for a stopping
master cannot see a pod that Sentinel promotes while it stops.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: describe WAIT as a check, not a guarantee

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: describe the fenced shutdown failover from Saremox#229

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
…emox#228)

* fix(release): make a release tag deploy the operator of that tag

The raw manifests and the kustomize overlays at a release tag deployed
operator 4.0.0-rc2 and v1.4.0. The release workflow runs after the tag
exists, so it cannot change the files at the tag. A new verify job now
stops the release when the example manifests, the kustomize base and
version component, or Chart.yaml do not carry the tag version.
scripts/release.sh set-version writes the version before the tag, and
helm-tests.sh checks that the files agree on every pull request.

The release sed over values.yaml also replaced the repository and tag
of crds.upgradeHook.image, so the published chart ran the CRD upgrade
hook with the operator image, which has no kubectl. The substitution
now changes only the top-level image block, and helm-tests.sh asserts
that the hook image does not change.

The release kustomize step set the image for the name redis-operator,
which matches no image, so the redis-operator.yaml release asset also
deployed v1.4.0. It now uses the image name of the base deployment.

The files now carry 4.2.0-rc2, the newest release on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(release): fail the version check when a version field is missing

A substitution in with_version that matches nothing changes nothing, so
check-version passed when a version field was removed or the image
repository was renamed. The release then pinned no operator version.
check-version and set-version now fail when a required field is not in
its file. helm-tests.sh renames the image in a copy of the files and
asserts that the check fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: eval <eval@example.invalid>
…stalled master (Saremox#213)

* fix(failover): wait spec.sentinel.failoverTimeout before replacing a stalled master

spec.sentinel.failoverTimeout was documented as how long operator-managed
failover waits before promoting a replica, but nothing read it. The
operator promoted on the first reconcile that saw the master unhealthy,
so a short stall (GC pause, slow fork, network hiccup) cost a failover:
a resync of every replica and possibly lost writes.

The operator now remembers, per RedisFailover, when the master was first
seen not answering, and promotes only once that is failoverTimeout ago.
The wait applies while the master's pod is still there:

- one master was counted but failed the health check that followed;
- no master answered and the pod labelled master is running, not
  deleting and not ready.

A ready pod that doesn't answer already blocks promotion (Saremox#193); the
clock starts there too, so once Kubernetes marks the pod not ready the
operator promotes as soon as the timeout has run out instead of waiting
it out again. With no master pod at all, or one being deleted (handled
by the existing wait for it to exit), there is nothing to wait for and
the election stays immediate.

The time resets when the master answers again or a replica was
promoted, and is dropped when the RedisFailover is deleted. While
waiting, status.message reads "master unreachable for 4s, failing over
after 10s", and the RedisFailover is queued again for when the timeout
runs out, since no pod event may come before then. 0s keeps the old
immediate failover.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* test(failover): cover the master pod listing error in the no-master branch

When the pods can't be listed while checking for an unreachable master
pod, the reconcile returns the error and starts no wait. No test reached
that path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(failover): don't delay the election when the health check finds no master

CheckMasterHealth returns no master IP when its own scan finds no
master: the master counted just before restarted or became a replica
in between, or listing the pods failed. That was treated as a master
that stopped answering and waited out failoverTimeout, delaying the
immediate election for a missing master.

Only a master the health check found, which then didn't answer, gets
the timeout now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(failover): keep the failover deadline across operator restarts

When the master was first seen not answering lived only in the
operator's memory. An operator restart or a leader change during an
outage started failoverTimeout over, so repeated restarts could put
the failover off indefinitely.

The operator now records that time on the master pod, in the
redisfailovers.databases.spotahome.com/unreachable-since annotation
(RFC3339), when the wait starts, and reads it back on every later
reconcile, also after a restart. The annotation is cleared once the
master answers again or a replica was promoted. In memory the operator
only remembers that no pod carries the annotation any more, so a
healthy reconcile doesn't list the pods each time; after a restart the
first healthy reconcile looks once and clears a leftover annotation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(failover): report NotHealthy when the failover timeout can't be tracked

The failoverTimeout wait added API calls whose errors were returned
while the status still said Healthy: looking up the master pod that
doesn't answer, and recording on it when it stopped answering. Like the
stopping-master lookup since Saremox#219, they now set NotHealthy first, with
"unable to look up the master pod" or "unable to record when the master
became unreachable".

Clearing the unreachable-since annotation runs once the master answers
again or was replaced, so the RedisFailover is healthy then. A failure
there no longer fails the reconcile or makes it unhealthy: it is logged,
the RedisFailover isn't marked clear, and the next reconcile tries again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: rewrite failoverTimeout docs and comments in ASD-STE100

Keep the reasons for the failoverTimeout wait, the unreachable-since
annotation and the immediate election; remove text that repeats the
code. The CRD descriptions follow the new FailoverTimeout doc comment
(controller-gen v0.20.1 output, only that field's description).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* docs: shorten failoverTimeout README text and test comments

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

* fix(failover): skip the unreachable-since annotation when failoverTimeout is 0s

waitForFailover wrote the unreachable-since annotation on the master pod
before it compared the elapsed time with failoverTimeout. With 0s there is
no wait, but a failed PATCH (API throttling, a webhook that rejects it)
still stopped the failover: the RedisFailover went NotHealthy with "unable
to record when the master became unreachable" and no replica was promoted.
Each 0s failover also cost a PATCH, and later a LIST and a PATCH to clear
the annotation.

Without a wait there is no deadline to keep, so waitForFailover now fails
over at once and does not read or write the annotation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE

---------

Co-authored-by: Claude <noreply@anthropic.com>
Saremox and others added 26 commits October 16, 2026 01:30
)

Bump golang.org/x/net from v0.59.0 to v0.60.0 in go.mod and go.sum, and the same in test/soak. govulncheck reported 5 called vulnerabilities in v0.59.0 (GO-2026-6603, GO-2026-6610, GO-2026-6611, GO-2026-6612, GO-2026-6617) through the client-go HTTP/2 transport. It reports none in both modules after the bump. No other dependency changes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…ns (Saremox#316)

The caller context reaches Handle, the finalizer patches, the status writes, the conflict requeue and the lease release. The writes and the release use context.WithoutCancel, because they must finish on shutdown, so the cancellation behavior stays as before at every site. CheckAndHeal keeps its signature and calls the new checkAndHeal(ctx, rf).

The Sentinel and replica counts are parsed with ParseUint (64 bit) with the existing limit of 65536, so gosec sees a proven int32 range. A count above 2^63 now gives the "more than 65536" error and the SENTINEL_TOO_MANY metric instead of MISC. The redis command uses CommandContext with a context without cancel and a #nosec comment with the reason for G204 and G702. The health listener uses ListenConfig. The soak metrics server shutdown keeps the values of the run context.

Strict lint: contextcheck 10 to 2, noctx 2 to 0, gosec 6 to 0. Left for the owner: the soak request struct that carries the verify context over a channel, and two test fixtures that store a context.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…not the pod IP (Saremox#313)

The metric redis_operator_controller_redis_operations_total has the labels namespace and pod_name instead of IP. A pod IP changes each time Kubernetes creates the pod again, and each IP gave other series in Prometheus. The check metrics redis_operator_controller_redis_checks_total and redis_operator_controller_sentinel_checks_total use the label pod_name instead of instance. The value is the pod name or NA. The name pod_name does not collide with the labels pod and instance that a ServiceMonitor adds.

GetStatefulSetPods and GetDeploymentPods give the kind and the IP of each listed pod to the metrics (Recorder.SetPodIP), so the recorder finds the pod name without an extra API call. When two pods of one kind share an IP (host network), the series get the IP as pod_name and NA as namespace; the operator does not guess a pod. The 5-minute GC deletes the series of a removed pod and forgets the IPs that no pod list showed. The operation SET_SENTINEL_TO_MONITOR_REDIS_WITH_GIVEN_PORT now has the kind SENTINEL, not REDIS. The example Grafana dashboard and the 4.2.0 migration guide are updated.

Breaking for dashboards and alerts that filter redis_operations_total by IP, the check metrics by instance, or the kind of the operation above. Counters start again under the new label values. Not tested: a scrape by a real Prometheus.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…Saremox#317)

The client-go errors in service/k8s carry the verb, kind and object ("get statefulset ns/name: ..."). The go-redis errors in service/redis carry the Redis command ("INFO replication: ...", "SENTINEL MASTER: ..."). %w keeps apierrors.IsNotFound and the other helpers, and the metric classification still reads the raw error with strings.Contains or errors.As. The informer cache, EndpointSlice, ControllerRevision and log level errors are wrapped too.

The new redis.ReplyHasPrefix replaces the prefix checks on err.Error() in the handover and heal code, because a wrapped reply no longer starts with "ERR ...", "NOPERM" or "LOADING". redis.IsNoSuchMasterError reads the wrapped reply, so a Sentinel without mymaster still gets a new monitor. Four cmd.Result() checks that cannot fail are removed, because Process returns the error of the command.

Only error and log text changes. There are no API, CRD or metric changes. Strict lint: wrapcheck 101 to 1.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…x#315)

The soak tester drives and judges mode Auto of the memory autoscaler. New instances auto (operator mode) and auto-sent (Sentinel mode) start at 128Mi with request equal to the limit, maxMemory 35 percent with noeviction and memoryMax 384Mi. They need an operator with mode Auto and Kubernetes 1.33 or later.

Seven autoscale mutation kinds: a memory load to 97 percent of maxmemory, a master or replica kill while a step is Applying, and the user paths limit, memoryMax, mode and lost state. Each kind tells what it may do to the pods (recreate, hand over, shrink). A kill kind names the killed pod, and a shrink is allowed only when the target is lower.

Judges for each step: the size by the rule of the operator (25 percent, rounded up to 64Mi, capped at memoryMax, exactly one target), the Applying timeout, cooldown, budget and back-off, and no restart, recreate, handover, shrink or maxmemory above the limit without a user path. The judge reads every status field with a defined outcome (a finding or a tested default), tracks adopted phases and steps in the same minute, and ends every settle window at the step timeout. A violation is a finding, and the metric redis_soak_autoscale_steps_total counts the steps. The tester can delete the state ConfigMap, and the full e2e profile leaves the autoscale instances out.

Not tested: a kind or e2e run on a cluster.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…aremox#318)

The soak judge of the autoscaler step size no longer compares the limit before a step with itself. In the reconcile that starts a step, the operator writes the state (phase Applying, the new lastStep), and the status still shows the old effectiveMemory, or none when it equals the spec limit. The template and the status get the target one reconcile later. The soak run of main 740f2c9 raised a false autoscale_step_size finding at every step, although the operator steps were correct.

At the start of a step the judge only remembers the limit before the step, the spec limit and memoryMax. It checks the size in Applying as soon as effectiveMemory differs from that limit, and at Done or Held for a step that is first seen late. A Reverted step gets no check, and a change of the spec limit or of memoryMax during the step drops the check. A wrong size is still a finding, and each step is counted once. The other judges do not read effectiveMemory in the first round of a step.

Not tested: a kind or e2e run.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
… the cleanup command (Saremox#319)

Tests pin the behavior before the refactor: the error texts of the config validation, the defaults, the metric descriptions, the order of the API calls of Delete, fetch, rotateOffline and collect, golden plans of newPlan and planAutoscale, the window of execute, and the output of the cleanup command. The golden files were written on the base code.

Mutations.validate, Config.setDefaults, Config.validate, Mutator.execute, Mutator.fetch, newPlan, planResources, planAutoscale, stuckChange, rotateOffline, Observer.apply, Observer.collect, Instance.Delete, metrics.New, run and runCleanup are split into helpers, so that each has a cyclomatic complexity of at most 15, a cognitive complexity of at most 20 and at most 80 lines. The reads of the instance move to mutator/fetch.go and the reads of the pods to observer/pods.go (moved code unchanged), so that each file has fewer than 800 lines. TestDeleteOrder ends its waits after a number of calls, so that it does not depend on the speed of the machine.

No change of logic, messages or the order of API calls. Left over: validateVersions, Chaos.validate, validateChain, Observer.observeMaster and validateAutoscale are still above 15. Not tested: the rest of run, which needs a cluster.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…long builders (Saremox#320)

generator.go (1597 lines) is split into generator_service.go, generator_configmap.go, generator_statefulset.go, generator_deployment.go and generator_exporter.go. The declarations move unchanged, so each file has one object kind. generateRedisStatefulSet (202 lines) and generateSentinelDeployment (179 lines) are split into builders for the pod spec, the containers and the probes, so that each function is below 80 lines. The Sentinel part of shutdown.sh and ready.sh moves to constants, and the script text stays byte-identical. One builder serves the Redis master and replica Services, which removes a dupl clone.

The golden tests pass with unchanged golden files. A harness compared the JSON of all 12 generated objects (and the input after the calls) for 6062 RedisFailover variants on main and on this branch: the output is byte-identical. No behavior change. Not tested: a cluster run.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…emox#321)

ResizePodInPlace, inPlaceResources and keepsTemplate (inplace.go) are split into helpers, so that each function has a cyclomatic complexity of at most 15 and a cognitive complexity of at most 20. The closures become methods of podResize. EnsureRedisMaxMemory and redisMemoryLimits (maxmemory.go) are split in the same way, with the closures as methods of maxMemoryRun. HandOverMaster (handover.go) is split into startFailover, awaitFailover and handoverResult. The label and replica steps of PromoteBestReplica, SetOldestAsMaster and SetMasterOnAll (heal.go), and the password change of a pod in ApplyPassword (password.go), are extracted. Tests for the error branches of the healer and for keepsTemplate were added before the refactor.

No behavior change: the API and Redis calls, the log lines and the results stay the same and in the same order. A deterministic simulation (seeds 9000-9199, 20000-20099, and 212 new seeds of the review) gives identical output on main and on this branch, and an independent review ran about 2.2 million randomized differential cases of the ten old functions against the new ones with every call recorded in order: 0 differences. Not tested: a real cluster and a real Redis.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…pic (Saremox#322)

UpdateRedisesPods, checkAndHeal, checkAndHealOperatorManagedMode, checkAndHealBootstrapMode, applyPassword, checkAndHealSentinels and readSentinelFailovers are split into helpers, so that each function has a cyclomatic complexity of at most 15, a cognitive complexity of at most 20 and at most 80 lines. The copies of the stopping master check, the oldest pod election, the config and rollout step, the Sentinel quorum check and the status of a failed promotion are replaced by one helper each. The declarations of checker.go move without a change to seven topic files, each below 800 lines.

TestOperatorElectedText of the soak module reads the status text from the checker files. It now reads all checker*.go files, and the CI path filter of the soak job is operator/redisfailover/checker*.go, so that the soak job runs when a split file changes.

No behavior change. A deterministic simulation gave identical traces of every API and Redis call on 402 seeds, and an independent review extended it with patch bodies, logs, metrics, requeues and a Sentinel failover model: identical on 212 new seeds. Not tested: a cluster run.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
… the metrics and the Redis and Kubernetes services (Saremox#323)

autoscale (cyclomatic 40), recordAutoscaleRollout (24), Observe (24) and Plan (23) are split into helpers, and autoscaler.go into three topic files. Validate (22) and validateAutoscaling (21) of the API move their steps to helpers. NewRecorder (125 lines) builds the metric vectors with two helpers, and removeStaleMetrics (cognitive 28) is split. In service/redis the two Sentinel count reads are shared, the INFO replication parse moves out of GetReplicationInfo, and the 1238-line client.go is split by topic. In service/k8s the CreateOrUpdate steps and the create/update result of the kinds are shared, and each PVC is updated in its own method. CheckSentinelsCannotFailover moves its per-Sentinel check to a helper, and the 980-line check.go is split by topic.

No change of behavior: error, log and metric texts, API call order, status texts and generated objects stay the same. One observable difference: the src=file:line field of log lines of moved or extracted code changes (for example, the debug lines for a successful Create or Update of five Kubernetes kinds report util.go:102).

Characterization tests pin the error texts, the call and log order, the metric descriptions and the defaults. A deterministic simulation gives identical output on main and on this branch for 402 seeds of the author and 212 new seeds of the review. The review ran differential tests of the old and new code (3,000,000 random cases and 60,000 trajectories of Observe and Plan, 1,000,000 random specs, 20,000 seeds of CreateOrUpdate for five kinds): identical. Not tested: a real cluster.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…pected state (Saremox#324)

The soak run of main 740f2c9 raised RedisSoakNotWritable on op-noevict (operator mode, policy noeviction) for about 85 seconds without any lost write. The operator behaves as documented: with a non-allkeys policy it never sets maxmemory below the used memory it reads, and on a raised limit it sets the requested percent. With noeviction, Redis refuses writes as soon as the used memory reaches maxmemory, and nothing frees memory until the user deletes data or changes the policy, percent or limit.

The tester treats an OOM refusal from a full noeviction master as an expected state. Full means policy noeviction and used_memory minus mem_not_counted_for_evict at or above maxmemory minus a margin (1 percent, at least 1Mi), read from the INFO call that the observer already makes. A write probe counts as expected only when its own SET failed with OOM and the single master is full. New metrics: redis_soak_expected_not_writable and redis_soak_expected_not_writable_seconds_total. The alert RedisSoakNotWritable gets one unless clause for them, with a new rule test. The state is bounded by observer.noevictionFullLimit (default 30m), after which it counts the finding noeviction_full; a round without a single master keeps the start time, and only a round that sees a master that is not full ends the state. The new finding maxmemory_below_used counts a pod whose maxmemory decreased below its counted memory plus the margin under a non-allkeys policy. Both finding series start at 0 for every instance. README and metrics golden file are updated.

Not run locally: the promtool check of the alert rules.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
… the root module (Saremox#327)

Receivers: logger and dummy use pointer receivers, and the in-place helpers of the autoscaler State become functions with a *State parameter, so that all State methods have value receivers. The exported method set and the JSON form do not change. health.go uses strings.IndexByte instead of a local helper, the DNS policy names follow the Go naming rules, and the modernize hints of 41 test files are applied. The recreateAfter schema test checks the length before the regexp and each value once: it takes 6 seconds instead of 20 to 26 under the race detector, with the same values and assertions. Each exported constant has a comment that starts with its name, and one long sentence of the 4.2.0 migration guide is split.

Strict profile of the root module: 614 to 415 findings. Not done on purpose: nilnil in checker_rollout.go (a nil *rolloutWait means no wait on the core reconcile path), the containedctx test wrapper and the ALL_CAPS metric names. No change of behavior: the simulation harness gives byte-identical output on base and head.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…f the soak tester (Saremox#326)

Verify takes the turn of the verifier and verifies with the context of its caller, so that no struct stores a context any more. One verification runs at a time, a failover that arrives during a requested verify waits and is verified after it as its own event, and only the caller's context cancels a verify. The mutator test fixture returns the context of its run instead of a field. planAutoscale gets a default case with the same skipped plan. Chaos and Version keep value receivers only, and their setters become functions. validateVersions, validateChain, Chaos.validate, validateAutoscale, Data.verify, Data.Load, Judge.Observe, Judge.checkPods, tracker.update and observeMaster are split into steps, below cyclomatic 15 and cognitive 20. newTracker uses config.MinDwell, its only value. Mechanical strict-lint fixes (prealloc, gocritic, revive, modernize, perfsprint, errname, intrange, goprintffuncname) touch about 30 files, and the README sentences of more than 25 words are split.

Strict profile of test/soak: 69 findings before, 12 after. No change of the config schema, the metric names, the findings, the deploy files or the e2e filter. Characterization tests were committed first and pass on main. Not tested: the renames in ensureInstances and authSources of cmd/soak, which need a cluster.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
The operator logs each change of the managed maxmemory at Info. A master line has the old and the new value, the used memory and the policy that the operator read, and the policy in force at the write: the read one before a policy change (a raise writes maxmemory before the policy), the managed one once the master accepted it. The restore of the old value on the master after a lower that does not fit also logs a line. A replica gets a line only when the value that it sets differs from the master's value that was read, and the line states only that master value, because the operator does not read the replicas. An unchanged value logs nothing, since the operator sets maxmemory at each reconcile.

The end of a maxmemory message and the end of a rollout hold are logged each on its own, also when the other part of the outcome stays, and removing spec.redis.maxMemory ends a stored message and hold with an Info line. The log lines claim only what the operator read or set. An OOM refusal under noeviction can come from a lowered maxmemory or from the writes, and the soak run could not tell them apart. The README says where to find the lines and that noeviction refuses writes when the used memory reaches maxmemory.

The Redis calls and their results do not change: a deterministic simulation gives the same API calls, Redis calls and log lines on main and on this branch, apart from the new lines. Not tested: a real cluster.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…ls (Saremox#328)

Time each observer round, log and measure slow reads with their phase (dial, reply or context), register the Go runtime metrics of the GC and the scheduler, and detect stalls of the tester process. A failed API read keeps its time in the round log. The error text of a read timeout stays the same.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
… and docs (Saremox#329)

Resolve the cheap strict-lint findings in tests, comments and docs: unused test parameters, shadowed names, test style fixes, a test context without an embedded field, %w for the Kubernetes configuration error cause, README sentence splits, doc comments, and the observer round log with the round context. The text of every error stays the same.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…aremox#331)

In Sentinel mode, the rollout deletes the stale master pod only after the Sentinel Deployment has finished its rollout and no Sentinel pod is terminating. The shutdown script of the old master sends SENTINEL FAILOVER through the Sentinel Service, and in two soak events it reached the last old Sentinel, which the Deployment controller stopped 0.3 seconds later. The wait ends after 10 minutes with a Warning, so a Sentinel rollout that does not finish cannot hold the master for ever. A restart of the operator starts the 10 minutes again.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…ves the master role (Saremox#330)

In Sentinel mode, the operator reads the master address from each running Sentinel and counts the answers per address before it moves a master label, a replica or a Sentinel. It waits when an address other than the Redis master has a quorum of the answers, counting unreadable answers as possible votes. A Sentinel leader can promote a replica and stop before it tells the other Sentinels. The operator then gave that replica the master label, and a second Sentinel failover discarded its writes. Two soak events lost 9 acknowledged writes each. The wait ends after the largest down-after plus the largest failover-timeout of the Sentinels, 15 seconds by default, and the known limits are in docs/logic.md.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…aremox#332)

CreateOrUpdateDeployment keeps the stored annotations of the Sentinel Deployment. The API server raises the generation of a Deployment when its annotations change. The operator updated the Sentinel Deployment in each reconcile and removed the revision annotation each time, so the generation changed in each reconcile. The wait of the master replacement for the Sentinel rollout is tracked per generation, so it started again in each reconcile and its 10-minute bound was never reached. A desired annotation replaces a stored annotation with the same key. Annotations that other tools add to the Sentinel Deployment now stay.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…f the RedisFailover (Saremox#335)

Give the Sentinel master name one source for sentinel.conf, the readiness probe and the shutdown script, with no change to the generated objects. In Sentinel mode, read SENTINEL MASTER and SENTINEL SENTINELS from each running Sentinel and report a peer IP that is no Sentinel pod, and a master IP that is no Redis pod, of the RedisFailover. The report needs two checks at least 90 seconds apart and ignores IPs that a pod of the RedisFailover had in the last 5 minutes. It gives a Warning at the start, an Info at the end and two gauges, and the operator changes no status, role or Sentinel. All RedisFailovers use the master name mymaster, so the Sentinels of two RedisFailovers can merge into one group, as in a soak event.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…ed the master (Saremox#333)

Do not wait for the Sentinels after the operator elected the master itself. The election needs Sentinels that know no replica pod and monitor no pod, so they cannot fail over the old address, and the wait of the Sentinel agreement only delayed the repair of the replicas and the Sentinels by about 14 seconds in a soak event. The operator records the election, also when the promotion fails only on a replica or on the master label, and skips the wait while the Redis master is the elected one. The skip ends at the bound (down-after-milliseconds plus failover-timeout, counted from the election), when the Redis master changes, when the Sentinels give the elected master, after a reconcile that skips the check, on Forget and on an operator restart. The error text of a failed master label gets the prefix "promotion succeeded but master label failed: ".

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…mox#334)

Bumps alpine/helm from 4.3.0 to 4.3.1.

---
updated-dependencies:
- dependency-name: alpine/helm
  dependency-version: 4.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…emox#336)

TestRunStopsOnSIGTERM waits until the metrics handler registers its metrics on the test registry before it sends SIGTERM. Run builds the handler in a goroutine that it does not wait for, and promhttp.Handler reads prometheus.DefaultRegisterer there. The test cleanup writes that global, so under load the race detector reported the pair and the test failed. The production code does not change.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
…he only Redis pod as master (Saremox#337)

With redis.replicas set to 1, the operator records its election of the only Redis pod, as it records the promotion when no Sentinel can fail over, so the wait of the Sentinel agreement is skipped with the same bound and end rules. Other pods stay only while the StatefulSet removes them, and the election already gives the master label to the elected pod, so a wait does not protect its writes. In 4 soak events on a single-pod instance the Sentinels gave the address of the deleted pod, could not fail over, and the wait reached its bound each time. SetOldestAsMaster returns the IP of the master that it set, or an empty string when it set none; the elections after a lost Sentinel quorum and after a start from 127.0.0.1 record nothing.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G4mmyo3sqLwf23m5FE2nBE
The last sync was 2026-09-19. Upstream took the two largest pieces of this
fork into its own tree in the meantime, so most of the merge is theirs:

- The instance manager (Saremox#236, then Saremox#238, Saremox#239, Saremox#316, Saremox#319, Saremox#327, Saremox#329).
  Upstream's copy fixes a real race: the zombie reaper and exec.Cmd.Wait
  both called wait4, so the reaper could take the Redis exit status and
  leave Wait with ECHILD. All of cmd/instance is upstream's version now.
- Operator-managed failover, the master handover and the replica quorum
  gate, in the split checker (Saremox#322) and checker_operator_mode.go.

What stays different in this fork, and why:

- The instance manager is wired in. Upstream builds the binary but does
  not use it; here an init container copies it to a shared volume, the
  Redis container runs it as PID 1, and the probes are HTTP probes
  against its health server. The wiring is now in one file,
  operator/redisfailover/service/generator_instance_manager.go, so the
  next sync has one place to look.
- No preStop hook in operator-managed mode. Upstream still runs a
  synchronous SAVE there. That SAVE is redundant with the save points and
  the manager's own SIGTERM forwarding, it eats the grace period before
  the manager's 25s window starts, and against an instance already in a
  BGSAVE it never returns. A production replica stopped this way.
- The operator image stays Alpine. The init container above runs cp, and
  the distroless image that upstream moved to has no cp.
- The images, the charts and the release checks point at ghcr.io/buildio,
  not ghcr.io/saremox.

Also here: the CRD and the DeepCopy code are regenerated with
controller-gen v0.20.1, and the golden files are rewritten. Only the Redis
StatefulSets changed in them, which is what the instance manager touches.

Chart and manifests move to 4.2.0-rc2 with upstream.
DefaultInstanceManagerImage stays at 4.1.0, the last image this fork
published; bump it when the fork cuts its next release.

Not done, and not needed for this merge: test/soak still pulls the
upstream OCI charts, because buildio publishes none. That workflow is
workflow_dispatch only and never runs on a pull request.
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.56294% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
cmd/instance/run/cmd.go 96.82% 1 Missing and 1 partial ⚠️
operator/redisfailover/autoscaler.go 99.30% 1 Missing and 1 partial ⚠️
cmd/instance/run/health.go 90.00% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@usiegj00

Copy link
Copy Markdown
Collaborator Author

Superseded by the rebase of the fork onto upstream/main. Closing in favour of that branch.

@usiegj00 usiegj00 closed this Oct 22, 2026
@usiegj00
usiegj00 deleted the chore/merge-upstream-saremox-2026-10 branch October 22, 2026 03:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants