Nimendra's Notes 🪴

What Is Mutual TLS (mTLS), Why Do We Need It, And How Do We Get It.

6 min read

kubernetes/networking networking videonote

What Is Mutual TLS (mTLS), Why Do We Need It, And How Do We Get It?

Speaker: Viktor Farcic (DevOps & AI Toolkit)
[!video]- What Is Mutual TLS (mTLS), Why Do We Need It, And How Do We Get It?

Tldr

  • Bi-Directional Verification: Standard TLS only authenticates the server to the client. mTLS forces both sides to cryptographically prove their identity, establishing a robust foundation for Zero Trust.

  • Zero Trust Foundation: In modern dynamic networks (like Kubernetes), internal service-to-service communication cannot be trusted by default; mTLS ensures only validated endpoints can connect.

  • Automated Internal CA: Rather than relying on public entities like Let’s Encrypt, a service mesh’s control plane acts as an internal Certificate Authority to dynamically issue and rotate short-lived certificates.

  • Application Obliviousness: By using sidecar proxies (e.g., in Linkerd or Istio), mTLS encryption and handshake processes happen entirely out-of-band. The application code remains completely untouched.

  • mTLS is Only the First Step: Encrypted transport secures connections, but it must be combined with strict Network Policies and micro-segmentation to control exact authorization paths.

1. The Architectural Shift: Monoliths vs. Microservices

A common misconception is that monolithic systems are simple and microservices are complex. In reality:

  • Even simple monoliths quickly expand to include frontends, backends, databases, pipeline tools (Jenkins, Tekton, Argo Workflows), and third-party helpers.
  • As monolithic systems split or scale into microservices, the volume of internal network communication increases exponentially.
  • The Scaling Principle: Securing 100 applications does not require 10 times more effort than securing 10 applications, provided the process is automated.

Quote

“Scaling in quantity of applications does not require equal scaling in effort. Remember that.”

2. One-Way TLS vs. Mutual TLS (mTLS)

One-Way TLS (Standard Browser Model)

  1. The client establishes a connection to the server.
  2. The server presents its TLS certificate.
  3. The client validates the certificate using a trusted external Certificate Authority (CA) and encrypts the session.
  4. The Gap: The server has no cryptographic proof of who the client is. This model works well for public web browsing but is inadequate for internal microservice networks where identity authorization goes both ways.

Mutual TLS (mTLS)

mTLS ensures bi-directional authentication where both parties confirm their identities before exchanging traffic. This is a critical component of a Zero Trust architecture, where no component is trusted by default.

sequenceDiagram
    autonumber
    participant App A as Client Sidecar
    participant App B as Server Sidecar
    participant CA as Mesh Control Plane (Internal CA)

    App A->>App B: 1. Initiate Connection
    App B-->>App A: 2. Send Server TLS Certificate
    Note over App A: Verifies Server Cert via Internal CA
    App A-->>App B: 3. Send Client TLS Certificate
    Note over App B: Verifies Client Cert via Internal CA
    Note over App A, App B: Handshake complete & session keys exchanged
    App A->>App B: 4. Secure Encrypted Traffic

3. Threat Vectors Mitigated by mTLS

Attack VectorExplanationHow mTLS Resolves It
On-Path (Man-in-the-Middle)An attacker runs a malicious process/container inside the cluster to intercept and sniff internal traffic.All transport traffic is strongly encrypted. Intercepted data is completely unreadable without the corresponding private keys.
Spoofing (Impersonation)A rogue pod masquerades as a legitimate application to send requests or steal incoming data.The spoofing pod cannot obtain a valid identity certificate signed by the internal CA, failing the handshake.
Credential StuffingAn attacker steals access keys or database passwords to gain unauthorized access from within.Connection attempts are rejected at the transport layer (network level) because the attacker lacks a valid client certificate.

4. Automation via Service Meshes and Sidecar Proxies

Manually managing and rotating certificates for dozens of microservices is impossible. Service meshes (such as Linkerd or Istio) automate this using the Sidecar Pattern.

The Sidecar “Dance”

  1. Application A sends a standard, unencrypted HTTP request (e.g., http://silly-demo:8080).
  2. Proxy A (intercepting all outbound traffic) catches the request and initiates an mTLS handshake with Proxy B.
  3. Both sidecars exchange and verify certificates using the service mesh control plane.
  4. Proxy A encrypts the data and sends it over the network.
  5. Proxy B receives the encrypted payload, decrypts it, and forwards a standard plaintext HTTP request locally to Application B.

Application Obliviousness

Applications remain entirely oblivious to the encryption process. Developers write standard HTTP code without security overhead, while the infrastructure transparently handles cryptographic transport.

Service Mesh CA Analogy

“You can think of that certificate authority as being Let’s Encrypt running inside your cluster.”

5. Implementation Walkthrough: Enabling mTLS in Linkerd

Enabling cluster-wide mTLS typically involves adding a single annotation to your namespaces, forcing the mesh to auto-inject helper sidecars.

Step 1: Annotating the Namespace

Instruct Linkerd to automatically inject sidecar proxies into all workloads deployed in the namespace:

apiVersion: v1
kind: Namespace
metadata:
  name: demo
  annotations:
    linkerd.io/inject: enabled

Step 2: Injecting and Verifying Pods

Once annotated, workloads are deployed normally. Checking the running pods reveals two running containers (one being the sidecar):

# Verify running pods show 2/2 containers ready (Application + Proxy)
kubectl get pods -n demo

You can verify the secure status of the network mesh using Linkerd’s CLI tools:

# Check meshed status and security details
linkerd viz -n demo stat pod

Expected output columns:

NAME          MESHED   ...   SECURED
other-app     2/2            âś”
silly-demo    2/2            âś”

Step 3: Transparent mTLS in Action

When calling standard services from a container (e.g., using curl inside an Alpine container):

# Execute curl inside a meshed container to a meshed application
curl http://silly-demo:8080

Even though the command uses http://, checking active traffic flow using CLI metrics (e.g., linkerd viz tap pod) confirms that the packet transfers are dynamically secured using TLS over the wire.

6. Crucial Best Practices & Next Steps

Encrypting transit communication is only the first step in protecting your cloud-native infrastructure.

Quote

“Mutual TLS is only the beginning, not the end of the story of securing your cluster.”

To achieve a true Zero Trust posture, implement the following network controls next:

  • Default-Deny Network Policies: Restrict non-meshed workloads from communicating with secure applications.
  • Micro-segmentation: Establish rigid ingress/egress rules defining exactly which application is authorized to talk to which (e.g., only App A can contact Database B, while App C is completely blocked).

Backlinks

Graph View