update-ksops-secrets

Update and generate the encrypted secrets preparing for Kustomize SOPS (KSOPS)

Overview

The update-ksops-secrets provides a declarative configuration for managing the secrets encryption and also generates the integrated manifests that could be rendered the Secret resource by kustomize with Kustomize SOPS plugin.

Kustomize SOPS manages the Kubernetes secret resources via the GitOps paradigm by integrating with kustomize and SOPS.

FunctionConfig

We use UpdateKSopsSecrets custom resource to configure the update-ksops-secrets function. The desired values are provided as a schema.

apiVersion: fn.kpt.dev/v1alpha1
kind: UpdateKSopsSecrets
metadata:
  name: string
  annotations:
    string: string
  labels:
    string: string
secret:
  type: string
  references:
    - string
  items:
    - string
recipients:
  - type: string
    recipient: string

apiVersion

The API version of the configuration resource

  • fn.kpt.dev/v1alpha1

kind

The resource kind refers to this configuration

  • UpdateKSopsSecrets

metadata

The metadata for describing the generated Secret resource

Field Description Example
name The generated Secret resource name runtime-secrets
annotations The secret annotations fn.kpt.dev/generator: update-ksops-secrets
labels The secret labels fn.kpt.dev/generated-by: kpt

secret

Field Description Example
type Type of the generated Secret resource –Opaque (default)–kubernetes.io/dockerconfigjson... kubernetes.io/dockerconfigjson
references The list of unencrypted secret resources that the update-ksops-secrets will look up and generates encrypted files unencrypted-secretsunencrypted-secrets-config-txt
items The list of secret keys for data look up in the referenced secret resources
recipients The list of recipients who could decrypt the generated encrypted files

recipients

Field Description Example
type The type of the recipient that supported by SOPS, eg. age, pgp age
recipient The recipient id–age: public key–pgp: fingerprint id age1x7pzjx4r05ar95pulf20knx0mkscaxa0zhtqr948wza3863fvees8tzaaaF532DA10E563EE84440977A19D0470BDA6CDC457

update-ksops-secrets function performs the following steps when invoked:

  1. Pass unencrypted secrets manifests referred by the configuration to the mutators pipeline.
  2. Generate the encrypted secrets from the listed items in the configuration using SOPS,
    • Unavailable secrets would be skipped.
    • Existing encrypted files of unavailable secrets will be processed with untouch.
  3. Generate or update the kustomization and KSOPS secrets resources.

Examples

Generate kustomization manifests with encrypted files

Let’s start with the input resource in a package, see the Note for how to prepare an unencrypted-secrets resource file

# unencrypted-secrets.yaml
apiVersion: v1
kind: Secret
metadata:
  name: unencrypted-secrets
dataString:
  test2: test2
  UPPER_CASE: upper_case
data:
  test: dGVzdA==

# unencrypted-secrets-config-txt.yaml
apiVersion: v1
kind: Secret
metadata:
  name: unencrypted-secrets-config-txt
data:
  config.txt: Y29uZmlnLnR4dAo=

Declare the new desired values for setters in the functionConfig file.

# update-ksops-secrets.yaml
apiVersion: fn.kpt.dev/v1alpha1
kind: UpdateKSopsSecrets
metadata:
  name: test-update-ksops-secrets
secret:
  type: Opaque
  references:
    - unencrypted-secrets
    - unencrypted-secrets-config-txt
  items:
    - test
    - test2
    - UPPER_CASE
    - config.txt
recipients:
  - type: age
    recipient: age1x7pzjx4r05ar95pulf20knx0mkscaxa0zhtqr948wza3863fvees8tzaaa

# Kptfile
apiVersion: kpt.dev/v1
kind: Kptfile
metadata:
  name: update-ksops-secrets
pipeline:
  mutators:
    - image: ghcr.io/neutronth/kpt-update-ksops-secrets:0.2
      configPath: update-ksops-secrets.yaml

Invoke the function:

$ kpt fn render

Alternatively, invoke function directly without the Kptfile

$ kpt fn eval \
    --image=ghcr.io/neutronth/kpt-update-ksops-secrets:0.2 \
    --fn-config=update-ksops-secrets.yaml

If you encountered the error with the PGP/GPG recipients encryption, see the Note to understand the limitation and working solution.

The above command will add files to your directory, which you can view

$ kpt pkg tree
Package "example"
├── [Kptfile]  Kptfile update-ksops-secrets
├── [kustomization.yaml]  Kustomization
├── [secrets.yaml]  Secret test-update-ksops-secrets
├── [update-ksops-secrets.yaml]  UpdateKSopsSecrets test-update-ksops-secrets
└── generated
    ├── [ksops-generator.yaml]  ksops ksops-generator-test-update-ksops-secrets-config.txt
    ├── [ksops-generator.yaml]  ksops ksops-generator-test-update-ksops-secrets-test
    ├── [ksops-generator.yaml]  ksops ksops-generator-test-update-ksops-secrets-test2
    ├── [ksops-generator.yaml]  ksops ksops-generator-test-update-ksops-secrets-upper_case
    ├── [secrets.config.txt.enc.yaml]  Secret test-update-ksops-secrets
    ├── [secrets.test.enc.yaml]  Secret test-update-ksops-secrets
    ├── [secrets.test2.enc.yaml]  Secret test-update-ksops-secrets
    └── [secrets.upper_case.enc.yaml]  Secret test-update-ksops-secrets

The generated files in the package could be rendered by the kustomization

$ kustomize build --enable-alpha-plugins .

apiVersion: v1
data:
  UPPER_CASE: dXBwZXJfY2FzZQ==
  config.txt: Y29uZmlnLnR4dAo=
  test: dGVzdA==
  test2: dGVzdDI=
kind: Secret
metadata:
  annotations:
    kustomize.config.k8s.io/behavior: merge
  name: test-update-ksops-secrets
type: Opaque

Note

Create unencrypted secrets

We could manually create an unencrypted secrets file with your favorite text editor.

Or alternatively, create with kubectl as following command

$ kubectl create secret generic unencrypted-secrets --output=yaml --dry-run=client \
  --from-literal="test=test" \
  --from-literal="test2=test2" \
  --from-file=config.txt \
  | tee unencrypted-secrets.yaml

This created file must not stored in the repository, it was used only for data preparation that kept in local storage. Make sure to add the .gitignore for those files exclusion.

# .gitignore
unencrypted-*

GPG receive-keys requires network to work properly

The limitation of Kpt for security reason, the running container network is not allowed by default. The user should explicitly instruct the Kpt to enable network.

There is no options to enable network within render function as described above. Only the alternative command works with additional parameter --network

Hence, if the encrypted files recipients include the PGP/GPG fingerprints, the kpt-update-ksops-secrets requires network to work properly as following command,

$ kpt fn eval \
    --image=ghcr.io/neutronth/kpt-update-ksops-secrets:0.2 \
    --fn-config=update-ksops-secrets.yaml \
    --network

GitHub

View Github