04 - Kubernetes Certificates API & TLS Bootstrapping

kubernetes/security kubernetes/networking

Core Idea

Skip manual signing on the master: the Certificates API turns certificate issuance into a CSR workflow (request, approve, download), and kubelet TLS bootstrapping automates node enrollment with it.

• How the Certificates API works through the CertificateSigningRequest custom resource.
• Worked example onboarding user Jane: generate key and CSR, create the CSR, review and approve, extract and decode, configure kubeconfig, test.
• Worker node auto-enrollment: kubelet TLS bootstrapping and its key components.

Instead of manually generating certificates on the master node, you can use the Kubernetes Certificates API to request, approve, and download certificates. This is useful for onboarding new users or automating node additions.

For manual creation patterns, see 03 - Manual Certificate Generation.

1. How the Certificates API Works

The Certificates API uses a Kubernetes custom resource named CertificateSigningRequest (CSR).

Worked Example: Onboarding User “Jane”

This hands-on example walks through creating a certificate for a user named jane who needs developer access to the cluster.

Step 1: Jane Generates Key and CSR Locally

Jane creates a private key and a CSR. The CSR identifies her as CN jane, with group system:authenticated (default for users).

# Done on Jane's machine
openssl genrsa -out jane.key 2048
openssl req -new -key jane.key -subj "/CN=jane" -out jane.csr

Step 2: Create the Kubernetes CSR Resource

The cluster administrator takes jane.csr, base64 encodes it, and creates a Kubernetes CertificateSigningRequest manifest.

1. Get the Base64 representation of the CSR:

   # Remove newlines to get a single continuous base64 string
   cat jane.csr | base64 | tr -d '\n'

2. Create the manifest file jane-csr.yaml:

   apiVersion: certificates.k8s.io/v1
   kind: CertificateSigningRequest
   metadata:
     name: jane-developer
   spec:
     request: <BASE64_STRING_FROM_PREVIOUS_STEP>
     signerName: kubernetes.io/kube-apiserver-client
     usages:
     - digital signature
     - key encipherment
     - client auth

Kubernetes v1.36 Spec: CertificateSigningRequest

NOTE

signerName is required in the stable v1 API (standard in v1.36). The signer kubernetes.io/kube-apiserver-client is used for client authentication certificates. If you were requesting a server certificate (e.g., for a web service), you would use kubernetes.io/kubelet-serving or a custom signer.

3. Apply the resource to the cluster:

   kubectl apply -f jane-csr.yaml

Step 3: Review and Approve the CSR

The administrator lists all CSRs in the cluster.

1. Check status:

   kubectl get csr

   Expected Output:

   NAME             AGE   SIGNERNAME                            REQUESTOR          CONDITION
   jane-developer   12s   kubernetes.io/kube-apiserver-client   kubernetes-admin   Pending

2. Approve the certificate:

   kubectl certificate approve jane-developer

3. Re-check status:

   kubectl get csr jane-developer

   Expected Output:

   NAME             AGE   SIGNERNAME                            REQUESTOR          CONDITION
   jane-developer   45s   kubernetes.io/kube-apiserver-client   kubernetes-admin   Approved,Issued

Step 4: Extract and Decode the Certificate

Once the status changes to Approved,Issued, the signed certificate is embedded back in the resource’s status.certificate field as a base64-encoded string.

1. Extract and decode the certificate to a file:

   kubectl get csr jane-developer -o jsonpath='{.status.certificate}' | base64 --decode > jane.crt

2. Verify Jane’s new certificate:

   openssl x509 -in jane.crt -text -noout

   Notice that the issuer is the Kubernetes Cluster CA and the Subject CN is jane.

Step 5: Configure Jane’s Kubeconfig

Jane can now access the cluster using her private key and the new signed certificate.

# Add cluster details to kubeconfig
kubectl config set-cluster my-k8s-cluster \
  --certificate-authority=/etc/kubernetes/pki/ca.crt \
  --embed-certs=true \
  --server=https://192.168.1.100:6443 \
  --kubeconfig=jane.kubeconfig
 
# Add Jane's credentials
kubectl config set-credentials jane \
  --client-certificate=jane.crt \
  --client-key=jane.key \
  --embed-certs=true \
  --kubeconfig=jane.kubeconfig
 
# Set context
kubectl config set-context jane-context \
  --cluster=my-k8s-cluster \
  --user=jane \
  --kubeconfig=jane.kubeconfig
 
# Test authentication
kubectl --kubeconfig=jane.kubeconfig get pods

3. Worker Node Auto-Enrollment: Kubelet TLS Bootstrapping

Generating client certificates manually for every new worker node is operations-heavy. Kubernetes solves this via Kubelet TLS Bootstrapping:

sequenceDiagram
    participant Node as Kubelet Node
    participant API as Kube-APIServer
    participant RBAC as Auto-Approver Controller
    participant Signer as Cert-Signing Controller

    Node->>Node: Boots up with Bootstrap Token
    Node->>API: Authenticates as system:bootstrappers
    Node->>API: Submits CSR for Node Client Cert (CN=system:node:<name>)
    RBAC->>API: Auto-approves CSR (if RBAC rules permit)
    Signer->>API: Signs CSR using Control Plane CA key
    Node->>API: Polls and downloads signed certificate
    Node->>Node: Writes kubelet.conf (uses new certificate)
    Node->>Node: Deletes Bootstrap Token config

Key Components of TLS Bootstrapping:

1. Bootstrap Token: A short-lived shared secret (token-id.token-secret) stored as a Kubernetes Secret in the kube-system namespace. It allows the new Kubelet to authenticate to the API server as a member of the group system:bootstrappers.
2. Kubelet Configuration: The node starts with a bootstrap kubeconfig (/etc/kubernetes/bootstrap-kubelet.conf).
3. Auto-Approval: To prevent administrators from manually running kubectl certificate approve for every node joining the cluster, RBAC permissions are configured to allow the controller manager to auto-approve node client cert CSRs submitted by the system:bootstrappers group.
4. Certificate Rotation: Under the hood, the Kubelet configuration contains:

   rotateCertificates: true

   This tells the Kubelet to automatically detect when its node client certificate is nearing expiration (typically 70-80% of its lifetime), generate a new key pair, submit a CSR to the API server, download the signed cert, and reload its configuration without restarting the service.

🌐 External Resources & Notes

• KodeKloud CKS Certificates API