Series: Kubernetes and Cloud Networking · Part 3 of 5
- Kubernetes on EC2 Before EKS: What We Got Wrong (And Right)
- Common Kubernetes Pitfalls on EC2 (And How to Avoid Them)
- Istio Ambient Mode: A Service Mesh Without the Sidecar Tax
- Ingress-NGINX Is Retired: A Practical Migration to the Kubernetes Gateway API
- Ingress, Istio Service Mesh and API Gateway Explained: What Each Does and When to Use It
Contents
The Sidecar Problem
The classic Istio architecture injects an Envoy proxy as a sidecar container into every application pod. Every request in and out of the pod passes through it. That design gave us mTLS, retries, traffic shifting and rich telemetry without changing application code, and it came with a well-known bill:
- Resource overhead. A proxy per pod adds CPU and memory to every pod. With hundreds of pods, that's a meaningful share of cluster capacity.
- Operational friction. Upgrading Istio meant restarting every workload to pick up the new sidecar. Pods had start-up and shutdown ordering problems (the app starting before its proxy was ready, or jobs that never completed because the sidecar never exited).
- All or nothing. You paid for full L7 processing on every hop even when all you wanted was encryption.
This is why, for most small and medium teams, my advice was to skip the mesh entirely.
How Ambient Mode Works
Ambient mode, generally available since Istio 1.24, splits the mesh into two layers:
1. ztunnel: the secure L4 overlay. A lightweight, Rust-based proxy runs once per node as a DaemonSet. It handles mTLS, workload identity, L4 authorisation and TCP telemetry for every pod on that node. Traffic between nodes is tunnelled over HBONE (HTTP-based overlay network encapsulation). Application pods are untouched: no sidecar, no restart.
2. Waypoint proxies: optional L7 processing. When you need HTTP-aware features (header-based routing, retries, L7 authorisation policies, request-level metrics), you deploy a waypoint, an Envoy proxy that serves a namespace or a specific service. Only traffic that needs L7 goes through it.
Pod A ──► ztunnel (node 1) ══ HBONE / mTLS ══► ztunnel (node 2) ──► Pod B
│ ▲
└──► waypoint (L7, optional) ──┘The result is a mesh you can adopt incrementally: encryption and identity everywhere first, L7 features only where they earn their cost.
Getting Started
Install Istio with the ambient profile, then opt namespaces in with a label:
istioctl install --set profile=ambient --skip-confirmation
# Add a namespace to the mesh: no pod restarts required
kubectl label namespace orders istio.io/dataplane-mode=ambientAt this point, all traffic to and from pods in orders is mTLS-encrypted with SPIFFE-based identities. You can already write L4 authorisation policies:
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
name: payments-allow-orders
namespace: payments
spec:
selector:
matchLabels:
app: payments
action: ALLOW
rules:
- from:
- source:
principals: ["cluster.local/ns/orders/sa/orders"]When you need L7 features in a namespace, add a waypoint. Waypoints are defined with the Kubernetes Gateway API, which is a nice sign of where the ecosystem is converging:
istioctl waypoint apply -n payments --enroll-namespaceNow HTTPRoute resources can do weighted canaries and header matching, and authorisation policies can match on HTTP methods and paths.
What You Get
| Capability | ztunnel only | With waypoint |
|---|---|---|
| mTLS and workload identity | Yes | Yes |
| L4 authorisation (who can talk to whom) | Yes | Yes |
| TCP metrics | Yes | Yes |
| HTTP routing, retries, timeouts, fault injection | No | Yes |
| L7 authorisation (method, path, headers) | No | Yes |
| Request-level metrics and tracing | No | Yes |
Trade-offs to Be Honest About
- It's still Istio. The control plane, CRDs and debugging model are the same. You still need someone who understands it.
- Some features differ from sidecar mode. Check the current feature status page for anything you depend on, particularly multi-cluster and VM workloads, before committing.
- Waypoints add a hop. For L7 traffic, requests pass through a shared proxy rather than a local sidecar. For most workloads that's fine; for latency-critical paths, measure it.
- CNI compatibility. Ambient relies on the Istio CNI node agent to redirect traffic. Check compatibility with your CNI (and with EKS-specific networking) in a non-production cluster first.
When I'd Now Consider a Mesh
Ambient mode moves my threshold. I'd consider it when:
- You have a compliance requirement for encryption in transit and service identity inside the cluster, and you'd rather not implement mTLS in every service.
- You have more than a handful of services and want consistent "who can call whom" policies.
- You already use, or plan to use, the Gateway API for ingress, so the routing model is shared.
I still wouldn't adopt it for a five-service startup without a compliance driver. Good client libraries with sensible timeouts, plus OpenTelemetry, cover most of the need there.
The Takeaway
Ambient mode removes most of the reasons teams rejected a service mesh: no sidecars, no restarts to upgrade, and you pay for L7 processing only where you use it. Start with ztunnel for zero-trust encryption across a namespace or two, add waypoints only where you need traffic management, and treat it as a platform capability with a clear owner.
