Kubernetes Basics #10 - Network Policies & Service Mesh

DevOps

Last Edited: 6/18/2025

This blog post introduces network policies and service mesh (& Gateway API) in Kubernetes.

DevOps

So far, we have covered how to set up a secure entrypoint to the cluster with Ingress, access control for secure resource management by developers and services, and a CI/CD pipeline. Although this seems sufficient, it still presents a security vulnerability that any service can use any other service by default, which opens opportunities for malicious users of an application to access private information (like metrics) and manipulate payment services that are not intended for them. Therefore, in this article, we will cover network policies and service mesh, which can reduce this security vulnerability of intra-cluster communication.

Network Policies

In Kubernetes, a container network interface (CNI) is utilized to configure networking between containers, and MiniKube installs Kindnet as its CNI by default. Although performant and reliable, Kindnet doesn't allow us to set network policies to change the default configurations where any service can communicate with any other service. Therefore, we can disable the default CNI and install a CNI that supports network policies, such as Weavenet and Calico. Fortunately, MiniKube allows us to switch to Calico easily by adding --cni calico to minikube start.

# nginx-netpol.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: access-nginx
spec:
  podSelector:
    matchLabels:
      app: nginx # pods' label to apply this policy to
  ingress: # set rules regarding ingress (incoming traffic) to `nginx`
  - from:
    - podSelector:
        matchLabels:
          access: "true" # only allow access when pods have `access:true`
---
# default-netpol.yaml (Calico network policy)
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: default-deny
spec:
  selector: projectcalico.org/namespace != "kube-system"
  types:
  - Ingress
  - Egress

The example above shows a Kubernetes network policy that restricts access to nginx pods from pods without access:true in their labels, and a useful Calico global network policy that doesn't allow any traffic except for pods in the kube-system namespace by default, implementing a zero-trust network model. As you can see, these two can coexist to enforce network policies on Layer 3/4, which are effective in various cases. For more information, I recommend checking out the relevant resources cited at the bottom of the article.

Service Mesh (Istio)

Although network policies are useful and sufficient for many clusters, they only operate on Layer 3/4, meaning they cannot use Layer 7 features (TLS, headers, path-based routing) for more fine-grained control. Consequently, developers need to implement non-business logic around security, retry mechanisms, metrics collection, and so on by themselves when using network policies. Service mesh, like Istio, aims to alleviate this problem by operating on Layer 7 and enabling us to set up complex traffic rules and secure communications between services efficiently. Istio also supports canary deployments (where traffic is split into multiple versions of a service) and tracks metrics (supports Prometheus) for production-ready clusters.

Istio Mesh

The architecture above illustrates Istio mesh, consisting of Istiod in the control plane and Envoy proxies as sidecars for every service. To configure the service mesh, we can define resources for Istiod using YAML files, just as we define Kubernetes resources. For example, we can use authorization policy resources (explanation here) to define network policies on Layer 7, use virtual service resources (explanation here) to define traffic routing (HTTP vs HTTPS, URL rewriting, canary deployment, retry, etc.), and utilize destination rule resources (explanation here) for load balancing.

We can also substitute the cluster entrance from an Ingress controller to an Istio Ingress Gateway, allowing Istio features to be applied to the cluster entry. Specifically, we can define an Istio Gateway that uses the default implementation of the Istio Ingress Gateway, responsible for domains, TLS certificates, and default policies, and define virtual services to configure complex traffic routing (explained here), unifying the interface for defining traffic rules for both external and internal communication. The examples of how we define all the resources mentioned here and the details are accessible via the links provided above (they are not too complicated but cover too many parameters for this article).

Gateway API

The benefits of an Istio Ingress Gateway over an Ingress controller are not limited to the unification of the interface and include greater and finer control over traffic routing and a separation of concerns. Istio can focus on providing the default implementation of the Ingress Gateway, while cluster operators can focus on setting up the Istio Gateway for all related services, and developers can set up virtual services for their services. Kubernetes Gateway API aims to provide these benefits to users not just with Istio but with many infrastructure providers by setting up a standard Gateway and HTTPRoute (TLSRoute, GRPCRoute, etc.).

Gateway API Model

In this model, infrastructure providers (AWS, Google Cloud, Istio, etc.) focus on implementing the GatewayClass, load balancing implementation, and cluster operators install the custom resources for the GatewayClass and use it (by setting gatewayClassName) to define a Gateway that manages domains, TLS, and default policies. Then, application developers can define HTTPRoutes (or any other for the corresponding protocol) for various traffic routing (header-based, path-based, rewrite, canary deployment, etc.) on traffics via the Gateway (specified in the parentRefs). Instead of an Istio Gateway, we can use a Kubernetes Gateway by using the istio GatewayClass and allow developers to define HTTPRoutes, which significantly reduces the switching cost.

Conclusion

In this article, we covered how to define rules for intra-cluster communication with network policies on Layer 3/4 and a service mesh (Istio) on Layer 7, and how Istio and Gateway API can substitute an Ingress controller for cluster entry with greater control and a separation of concerns. The details of these topics are vastly ommitted from this article, so I recommend checking them out from the resources cited below.

Resources

Back to Top↑