cluster-machine-approver

This controller provides the following functionality:

  • Watch the CSR endpoint for CSR requests
  • Decide if the CSR should be allowed or denied
  • Approve or deny and update CSR status

Introduction

Kubernetes includes support for TLS
bootstrapping

for Nodes, which OpenShift makes use of.

Kubelet needs two certificates for its normal operation:

When a new host is provisioned, kubelet will start and communicates to the CSR
(Certificate Signing Request) API endpoint to request signed client and server
certificates. It issues this request using bootstrap
credentials

that it finds on its local filesystem.

At this point, these CSRs must be approved. They can be manually approved
through the API using
kubectl
,
or kube-controller-manager can be configured to approve
them
.
Alternatively, some custom component could be built to approve CSRs through the
API, which is what OpenShift has done.

OpenShift and CSRs

OpenShift includes a custom component to approve CSRs: the
cluster-machine-approver. The cluster-machine-approver is used to
automatically approve CSRs, but with more strict criteria than what was
supported in kube-controller-manager.

Note that the cluster-machine-approver only kicks in post-install. During
the cluster bootstrapping phase, the approve-csr service on the bootstrap
node

automatically approves all CSRs. This bootstrap service will end up approving
the CSRs for the control plane nodes, while cluster-machine-approver will
take over for future new CSRs from worker nodes.

Disabling Node Client CSR Approvals

It is possible to disable node client CSR approvals completely. This is done
using a ConfigMap resource, as shown in this PR
comment
.

apiVersion: v1
kind: ConfigMap
metadata:
  name: machine-approver-config
  namespace: machine-approver
data:
  config.yaml: |-
    nodeClientCert:
      disabled: true

This may be useful if you explicitly want to only allow manual CSR approvals
for new nodes.

Node Client CSR Approval Workflow

CSR approval details can be found in csr_check.go. Assuming
this has not been disabled, the following criteria must be met for the client
CSR to be approved:

  • The user must be the node bootstrapper
    • The username in the CSR must be
      system:serviceaccount:openshift-machine-config-operator:node-bootstrapper
    • The groups in the CSR must be
      system:serviceaccounts:openshift-machine-config-operator,
      system:serviceaccounts, and system:authenticated.
  • A Node object must not yet exist for the node that created the CSR.
  • The Machine API is used to do a sanity check. A Machine must exist with
    a NodeInternalDNS address in its Status that matches the future name of
    the Node, as found in the CSR.
  • This Machine must not have a NodeRef set.
  • The CSR creation timestamp must be close to the Machine creation timestamp
    (currently within 2 hours)
  • The CSR is for node client auth.

Node Server CSR Approval Workflow

Details of this workflow can be found in the same file as the client workflow,
csr_check.go.

For this workflow, it is assumed that the Node is now up and running, and the
Node object exists in the API. Validation for the server CSR is different
than the client case and is based primarily on matching addresses between
associated Node and Machine objects.

First, there must be a Machine object with a NodeRef field set to the
Node that sent this CSR. The NodeRef is set by a Node controller under
the machine-api-operator.

Once a NodeMachine pair has been identified, validation is done on all of
the Addresses in the Status field of the Machine. The CSR requests a
certificate with the SAN (Subject Alternate Names)
extension
. The resulting
certificate will be valid for every address or hostname listed on the Node
resource to validate this request, the cluster-machine-approver ensures that
every DNS name or IP address in the CSR matches a (NodeInternalDNS,
NodeExternalDNS, NodeHostName) or (NodeInternalIP, NodeExternalIP)
address on the corresponding Machine object.

Requirements for Cluster API Providers

As discussed in previous sections, cluster-machine-approver imposes some
requirements on each Cluster API provider used with the
machine-api-operator.
This section serves as a summary of those requirements.

  • A Machine must have a NodeInternalDNS set in Status.Addresses that
    matches the name of the Node. The NodeInternalDNS entry must be
    present, even before the Node resource is created.
  • A Machine must also have matching NodeInternalDNS, NodeExternalDNS,
    NodeHostName, NodeInternalIP, and NodeExternalIP addresses as those
    listed on the Node resource. All of these addresses are placed in the CSR
    and are validated against the addresses on the Machine object.

GitHub

View Github