A small (3MB uncompressed docker image), efficient (via inotify) sidecar to trigger application reloads when configuration changes.
Applications using this method include:
In Kubernetes, the “recommended” / usual way of managing configuration is instead to:
- Have a ConfigMap (or Secret) holding the configuration
- When it changes, trigger a rolling-upgrade of the matching Deployment or DaemonSet
While this method is great to ensure configuration changes are highly visible and ensures all replicas use the same config (see immutable infrastructure), it is best for stateless apps which can easily handle rolling upgrades.
Stateful apps, by contrast, might be better off reloading their config so as to not disrupt long-lived open connections, or not to incur a long restart time.
In Kubernetes, an upgrade to a ConfigMap or Secret is eventually (note: see gotchas) propagated to the running Pods, meaning we can watch configuration loaded via ConfigMaps or Secrets and send a signal to the main app when it changes, triggering a hot reload.
config-reloader-sidecar exists specifically for that use case!
How it works
config-reloader-sidecar uses Go’s fsnotify package to watch one (or more) configuration folders, and send a signal to a process when any change is detected within that folder. This includes file created, file updated, file renamed & file deleted, but excludes file permissions changes.
config-reloader-sidecar needs to run, as the name implies, as a separate container in the same Pod as the application you want to reload, i.e. a sidecar.
In addition, you’ll need to set
shareProcessNamespace: true on your Pod to send signals across containers.
config-reloader-sidecar is then configured through the following env vars:
CONFIG_DIR: comma-separated list of configuration directories to watch (mandatory)
PROCESS_NAME: process to send the signal to (mandatory)
RELOAD_SIGNAL: signal to send (optional, defaults to
Example Pod configuration
apiVersion: v1 kind: Pod metadata: name: auto-reloading-pgbouncer spec: shareProcessNamespace: true containers: - name: pgbouncer image: bitnami/pgbouncer:latest command: - pgbouncer args: - /etc/pgbouncer/pgbouncer.ini volumeMounts: - mountPath: /etc/pgbouncer/ name: secret-volume - name: config-reloader-sidecar image: some-private-repo.example.com/config-reloader-sidecar # Note: this isn't yet available on the Docker hub! env: - name: CONFIG_DIR value: /etc/pgbouncer/ - name: PROCESS_NAME value: pgbouncer volumeMounts: - mountPath: /etc/pgbouncer/ name: secret-volume volumes: - name: secret-volume secret: secretName: pgbouncer-secret
Share Process Namespace
In order for the sidecar to find which process to send the signal to, the Pod needs to be configured to Share Process Namespace with
You might noticed when editing a Secret or ConfigMap that your process isn’t being reloaded immediately.
This is because the projected values of ConfigMaps and Secrets are not updated exactly when the underlying object changes, but instead they’re updated periodically according to the
syncFrequency argument to the kubelet config. This defaults to 1 minute.
Config files mounted via
subPath are never updated
This is a long-standing Kubernetes issue: ConfigMap and Secrets mounted as files with a
subPath key do not get updated by the kubelet. See issue #50345 on Github.
The (pretty ugly) workaround involves mounting the secret/configmap without subPath in a different folder and manually creating a symlink from an initContainer ahead of time to that folder, or if possible at all switching to not using