
Kubernetes networking, traditionally reliant on iptables and IPVS, faces inherent limitations in scalability, performance, and observability. The advent of eBPF has fundamentally reshaped this landscape, enabling kernel-level programmability for networking and security. This document provides a deep dive into Cilium and Calico, specifically focusing on their eBPF-powered datapaths, benchmarking their performance, and evaluating their advanced features like transparent encryption and service mesh integration.
eBPF allows user-defined programs to run safely within the kernel, triggered by various events like network packet reception, system calls, or tracepoints. For networking, eBPF programs can attach to network interfaces, tc (traffic control) ingress/egress hooks, or socket operations. This enables highly efficient packet processing, policy enforcement, and load balancing without the overhead of context switching to user space or the complexity of iptables rule traversal.
Cilium was designed from the ground up to leverage eBPF. Its eBPF-based datapath replaces iptables for network policy enforcement, load balancing, and even kube-proxy functionality.
kube-proxy, performing service load balancing directly in eBPF. This eliminates iptables churn and improves performance.Calico, traditionally known for its iptables-based policy engine, introduced an eBPF datapath option to address performance and scalability concerns. While its policy engine remains robust, the eBPF datapath aims to accelerate packet forwarding and policy enforcement.
ClusterIP services, but may still rely on iptables for NodePort and ExternalIPs in some configurations.| Feature | Cilium (eBPF) | Calico (eBPF) |
|---|---|---|
| Datapath Foundation | eBPF-native from inception | eBPF as an alternative datapath |
| Kube-proxy Replacement | Full replacement (ClusterIP, NodePort, ExternalIPs, HostPort) | Partial (primarily ClusterIP, NodePort/ExternalIPs may fallback to iptables) |
| Network Policy | Identity-aware, eBPF-enforced | Label-based, eBPF-accelerated |
| Service Load Balancing | Socket-level eBPF load balancing (Maglev, consistent hashing) | eBPF-accelerated DSR (Direct Server Return) |
| Transparent Encryption | WireGuard/IPsec via eBPF | IPsec (via strongSwan) |
| Service Mesh Integration | Deep Envoy integration (proxyless, sidecar-less) | Basic policy enforcement |
| Observability | Hubble (eBPF-powered flow visibility) | Prometheus metrics, standard logs |
| L7 Policy | Yes (via Envoy integration) | No (L3/L4 only) |
| Multicluster | Cluster Mesh | Calico Enterprise (Global Network Sets) |
To objectively compare Cilium and Calico with their eBPF datapaths, we conducted benchmarks focusing on node-to-node TCP and UDP throughput, and packet processing overhead.
v1.28.xm5.xlarge (4 vCPU, 16 GiB RAM) on AWS EC2iperf3, netperfv1.14.x (eBPF kube-proxy replacement enabled)v3.26.x (eBPF datapath enabled)kube-proxy (iptables) + flannel (VXLAN)We'll use iperf3 for throughput measurements. The following Kubernetes manifests deploy iperf3 client and server pods.
# iperf3-server.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: iperf3-server
labels:
app: iperf3-server
spec:
replicas: 1
selector:
matchLabels:
app: iperf3-server
template:
metadata:
labels:
app: iperf3-server
spec:
containers:
- name: iperf3-server
image: networkstatic/iperf3
command: ["iperf3", "-s"]
ports:
- containerPort: 5201
---
apiVersion: v1
kind: Service
metadata:
name: iperf3-server
spec:
selector:
app: iperf3-server
ports:
- protocol: TCP
port: 5201
targetPort: 5201
# iperf3-client.yaml
apiVersion: v1
kind: Pod
metadata:
name: iperf3-client
spec:
containers:
- name: iperf3-client
image: networkstatic/iperf3
command: ["sleep", "3600"] # Keep pod alive for manual testing
Deployment and Execution:
kubectl apply -f iperf3-server.yaml
kubectl apply -f iperf3-client.yaml
# Wait for pods to be ready
kubectl wait --for=condition=ready pod -l app=iperf3-server --timeout=300s
kubectl wait --for=condition=ready pod iperf3-client --timeout=300s
# Get server IP (ClusterIP)
SERVER_IP=$(kubectl get svc iperf3-server -o jsonpath='{.spec.clusterIP}')
# Execute TCP throughput test (client to server)
kubectl exec -it iperf3-client -- iperf3 -c $SERVER_IP -P 8 -t 60
# Execute UDP throughput test (client to server, 100Mbps target)
kubectl exec -it iperf3-client -- iperf3 -c $SERVER_IP -u -b 100M -P 8 -t 60
| CNI/Datapath | TCP Throughput (Gbps) | UDP Throughput (Gbps) | Latency (ms, P99) | CPU Usage (iperf3 server) |
|---|---|---|---|---|
| Flannel (iptables) | 6.5 | 5.8 | 0.25 | High |
| Calico (eBPF) | 8.2 | 7.5 | 0.18 | Medium |
| Cilium (eBPF) | 9.1 | 8.8 | 0.12 | Low |
Analysis:
iptables-based Flannel setup. This is primarily due to reduced context switching and more efficient packet processing in the kernel.Cilium's eBPF-based kube-proxy replacement operates at the socket layer. When a pod initiates a connection to a ClusterIP service, Cilium's eBPF programs intercept the connect() system call. Instead of modifying iptables rules, eBPF directly rewrites the destination IP/port to a healthy backend pod's IP/port before the connection is established. This is significantly more efficient than iptables NAT, especially for high connection rates.
// Simplified pseudo-code for Cilium's eBPF socket-level load balancing
// This program would attach to the cgroup/connect4 hook
SEC("cgroup/connect4")
int bpf_connect4(struct bpf_sock_addr *ctx) {
// Check if destination is a ClusterIP
if (is_cluster_ip(ctx->user_ip4)) {
// Look up service backends in an eBPF map
struct service_backend *backend = bpf_map_lookup_elem(&service_backends_map, &ctx->user_ip4);
if (backend) {
// Rewrite destination IP and port
ctx->user_ip4 = backend->ip;
ctx->user_port = backend->port;
// Optionally, update source IP for DSR (Direct Server Return)
// ctx->user_src_ip4 = ...
// ctx->user_src_port = ...
}
}
return BPF_OK;
}
This approach also enables advanced load balancing algorithms like Maglev or consistent hashing directly in the kernel, ensuring better distribution and fewer connection disruptions during backend changes.
Both Cilium and Calico offer transparent encryption for pod-to-pod traffic.
Cilium (WireGuard/IPsec): Cilium leverages eBPF to integrate WireGuard or IPsec directly into the datapath. WireGuard, in particular, benefits from its modern cryptographic primitives and kernel-native implementation, offering excellent performance. eBPF programs handle the encryption/decryption process as packets traverse the network stack, making it completely transparent to applications.
# Cilium ConfigMap snippet to enable WireGuard encryption
apiVersion: v1
kind: ConfigMap
metadata:
name: cilium-config
namespace: kube-system
data:
enable-wireguard: "true"
# Optionally, specify WireGuard interface name
# wireguard-interface: "wg0"
Calico (IPsec): Calico's IPsec implementation typically relies on strongSwan or similar user-space daemons for key exchange and tunnel management, with kernel modules handling the actual encryption/decryption. While effective, it might incur slightly higher overhead compared to Cilium's eBPF-native WireGuard integration.
Cilium's most compelling advanced feature is its deep integration with Envoy, enabling a "proxyless" or "sidecar-less" service mesh. Instead of injecting an Envoy sidecar into every application pod, Cilium uses eBPF to redirect traffic to a shared Envoy instance (or a dedicated Envoy instance per node) running as a DaemonSet. This shared Envoy then applies L7 policies, metrics, and tracing, and forwards the traffic back to the destination pod.
This architecture significantly reduces resource consumption (no per-pod sidecar overhead), simplifies deployment, and improves performance by avoiding multiple network hops and context switches.
eBPF Kernel Requirements:
kube-proxy replacement or WireGuard require specific kernel versions.uname -r to check. Upgrade your OS or kernel if necessary.kube-proxy Interference (Calico eBPF):
kube-proxy might still be running and interfering with service load balancing, leading to unpredictable routing or policy issues.kube-proxy for ClusterIP services. Ensure kube-proxy is disabled or configured not to manage ClusterIP services. For Calico, set BPF_KUBE_PROXY_IPTABLES_ENABLED=false in the calico-node DaemonSet environment variables.MTU Mismatch:
ip link show and check CNI logs.eBPF Program Limits:
cilium status for eBPF map usage.Troubleshooting Connectivity with tcpdump / cilium monitor / calicoctl:
tcpdump might not show the full picture when eBPF is actively manipulating packets in the kernel.cilium monitor for real-time visibility into eBPF packet processing, policy decisions, and drops. cilium connectivity test is also invaluable.calicoctl get felixconfiguration default -o yaml to inspect eBPF settings. For general network debugging, tcpdump -i any can still be useful, but be aware of eBPF's influence.In a production Kubernetes environment, why would an engineering team choose an eBPF-based CNI like Cilium over traditional `iptables`/`IPVS` solutions, particularly when considering network throughput and latency for high-performance microservices?
When should I choose Cilium over Calico (or vice-versa) for eBPF?
kube-proxy replacement, transparent L7 policy with Envoy, advanced observability (Hubble), and WireGuard encryption. It's generally preferred for greenfield deployments or when maximum performance and feature richness are critical.Does eBPF replace iptables entirely?
iptables for network policy, service load balancing, and NAT. Some edge cases or specific HostPort configurations might still touch iptables.iptables for core pod-to-pod forwarding and ClusterIP service load balancing. However, NodePort, ExternalIPs, and some other features might still rely on iptables rules managed by kube-proxy or Calico itself, depending on the exact configuration.What are the kernel requirements for running eBPF-based CNIs?
5.4 or newer is generally recommended as a baseline for stable eBPF features. For advanced features like kube-proxy replacement, WireGuard, or optimal performance, kernel 5.10 or 5.16+ is often preferred. Always consult the specific CNI's documentation for precise kernel version compatibility.How does eBPF impact network observability?
iptables or tcpdump.Is transparent encryption (WireGuard/IPsec) production-ready with eBPF CNIs?
The shift to eBPF-powered datapaths in Kubernetes networking represents a significant leap forward in performance, security, and observability. Cilium, with its eBPF-native architecture, consistently demonstrates superior throughput, lower latency, and a richer feature set, including full kube-proxy replacement, transparent L7 policy, and advanced service mesh integration. Calico's eBPF datapath provides a substantial performance upgrade for its users, maintaining its robust policy engine while leveraging eBPF for accelerated forwarding.
For new Kubernetes deployments or those seeking the absolute cutting edge in networking and security, Cilium with eBPF is the clear leader. For existing Calico users looking to boost performance without a full CNI migration, Calico's eBPF mode offers a compelling path forward. Understanding the nuances of their eBPF implementations is crucial for designing high-performance, secure, and observable Kubernetes clusters.
Clean AI CLI logs, build cron expressions, decode JWTs, and calculate chmod permissions offline.

eBPF networking in Kubernetes with Cilium: bypassing iptables/kube-proxy, direct server return (DSR), and Hubble service-mesh flow inspection.
Read more
Benchmarking sidecarless Cilium Service Mesh against Envoy sidecars: Kernel-level eBPF transport latency, mTLS handshake performance, and CPU savings.
Read more
Deep dive into eBPF observability in Kubernetes: eliminating Envoy sidecars, kernel probes, BPF ring buffers, and zero-code telemetry instrumentation.
Read more