Skip to content

secret: values are only pulled at startup, so rotated credentials need a restart #994

Description

@Michad

Secrets are resolved once at startup and never again. docs/operation/modules/ROOT/pages/configuration/secret/aws_secrets_manager.adoc describes the consequence:

Currently secrets are only pulled at once at startup, however in the future this might be changed to re-pull periodically or upon configuration changes. Therefore it is advised to think through how often your secrets might rotate when configuring the TTL value, even though currently it only need to be a small value to span the initial start-up of the application.

For an operator with rotating credentials this is a real constraint: a rotated upstream API key or database password doesn't take effect until the process restarts. Hot reload covers it only if something independently triggers a reload, and the config file isn't what changed.

Add periodic refresh. Points to work through:

  • Where the refreshed value takes effect. Secrets are substituted into config values that are then consumed by already-constructed entities, so a refreshed secret likely means rebuilding the affected entities, which is the machinery hot reload already has (pkg/config/config.go, internal/server/server.go). Reusing the existing generation swap is probably better than threading live secret lookups into providers.
  • Whether refresh is opt-in per secret backend or a global interval, and how it interacts with the existing per-backend TTL cache.
  • What happens when a refresh fails. The running generation should keep working on the last known good values rather than tearing down, consistent with how a failed config reload is handled today.

This also affects the Secreter interface in pkg/entities/secret, which 1.0 freezes. Even if the implementation lands later, the interface should be checked now for whether it can accommodate a backend that needs periodic or push-based refresh. That check is the part with a 1.0 deadline.

Update the doc text quoted above once resolved, per #993.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions