kubernetes kubernetes/cka kubernetes/networking
Core Idea
CNI and kube-proxy solve different layers of pod networking: CNI connects containers to the network and hands out IPs, kube-proxy makes Services actually reach those pods.
- CNI is the software interface between the container runtime and the network implementation: a CNCF project with a specification, library,
cnitool, and reference plugins, with the IPAM interface allocating addresses.- kube-proxy watches Services and programs the forwarding rules on each node.
- The note’s comparison table spells out where each layer’s responsibility stops.
CNI
CNI is the software interface between the container runtime and the network implementation.
CNI is a project under cncf that provides essential specification, a library. for CNI integration with applications, a CLI tool called cnitool for CNI plugins and reference plugins.
Kubernetes relies on the CNI project to comply with the following requirements:
- All containers must communicate with each other without NAT.
- Nodes can communicate with containers without NAT.
- A container’s IP address is the same as those outside the container that it sees itself as
CNI Specification
cni/SPEC.md at main · containernetworking/cni · GitHub
The CNI specification covers the following parts:
- Network configuration file format
- Protocol for interaction between container runtime and network plugins
- Execution flow of plugins
- Execution flow when delegating to other plugins
- Data types of execution results returned to the runtime
How CNI Works
The CNI framework consists of:
- CNI Plugins:
- These are executable that handle specific tasks like setting up interfaces, IP allocation, and routing.
- CNI Configuration:
- A JSON configuration file specifies the CNI plugin and parameters like IP ranges, gateway, etc.
- Configuration is passed to the plugin by the container runtime.
- Container Runtime Integration:
- The container runtime (e.g., Docker, CRI-O, containerd) invokes the CNI plugin to set up networking for containers.
CNI Plugins
The CNI plugin has two primary responsibilities: allocate and assign unique IP addresses for pods and ensure that routes exist within Kubernetes to each pod IP address.
- Kubernetes does not include a built-in networking solution but relies on third-party CNI plugins to provide cluster networking.
- Popular CNI plugins for Kubernetes:
- Calico: Network policy enforcement and IP-based routing.
- Flannel: Simple overlay network.
- Weave Net: Secure networking across nodes.
- Cilium: Advanced networking and security with eBPF.
The Kubelet manages a container networking implementation (via the CNI) and a container runtime (via the CRI).
The IPAM Interface
The CNI spec has a second interface, the IP Address Management (IPAM) interface, to reduce duplication of IP allocation code in CNI plugins.
How IPAM Works
- When a Pod starts, the CNI plugin requests an IP from the IPAM.
- The IPAM assigns an available IP to the Pod.
- The CNI plugin configures the Pod’s network interface using the assigned IP.
- When the Pod is deleted, the IP is released back to the pool.
Kube-Proxy
Kube-Proxy is a Kubernetes network component that handles Service networking by managing network rules for traffic forwarding across Pods and Services.
- Manages Service-to-Pod communication.
- Implements load balancing across Pods within a Service.
- Uses iptables, IPVS, or userspace to route traffic to the correct Pod.
How Kube-Proxy Works?
- A Service is created with a ClusterIP.
- Kube-Proxy watches the API Server for Service and Endpoint updates.
- It configures iptables or IPVS rules to route traffic from the Service to the correct Pod.
Comparison: CNI vs. Kube-Proxy
| Feature | CNI | Kube-Proxy |
|---|---|---|
| Purpose | Connects Pods across nodes | Routes traffic to Services |
| Function | Assigns IPs, enforces policies | Handles Service-to-Pod traffic |
| Works at | Pod-to-Pod level | Service-to-Pod level |
| Implements | L3 (Networking) | L4 (Load Balancing) |
| Example Plugins | Calico, Flannel, Cilium | iptables, IPVS |