Leveraging Multi-Cluster Service Mesh on GKE Using Multi-Primary Control-Plane Architecture for Improving Scalability

In our first blog, we have explained about Anthos and its capabilities. In this blog, let’s dive deeper into Anthos Service Mesh.

Service Mesh manages communication between individual microservices in a Microservice application. Service Mesh is a paradigm/model, and Istio is one of its implementations. So, then the question arises why do we need a separate tool for managing communication between services.

In the normal Microservice Architecture, communication inside the cluster is not secure. Services talk to each other via HTTP or some insecure protocol. And also, every service inside the cluster can talk to any other service. So, once the request reaches the cluster it is no longer secure. It is an important factor to consider for applications involving sensitive data.

So, now this is where the Service Mesh Architecture comes into the picture.

Anthos Service Mesh Components:

The components of Istio can be described in terms of a data plane and a control plane. The data plane is responsible for carrying the network traffic in the mesh, and the control plane is responsible for configuring and managing the mesh:

Control plane:

As I mentioned earlier, Service Mesh is just a pattern or a paradigm and Istio is one of its implementations. In Istio Architecture, the proxies are Envoy proxies which is an independent open source project that Istio as well as other Service Mesh implementations also use.

The Control Plane component in Istio is Istiod, which manages and injects the Envoy proxies in each of the microservice pods. Note here that, in earlier versions of Istio (upto 1.5), Istio Control Plane was a bundle of multiple components such as Pilot, Galley, Citadel, Mixer, etc. However, in version 1.5 all of these separate components were combined back into one single component called Istiod to make it easier for the users to configure.

Data plane:

Istio Control plane which is Istiod manages the data plane. It assists us in Service to Service communication by injecting a sidecar container in each of the Pods in the mesh. The sidecar runs an intelligent proxy called Envoy to provide routing, security for the Services.

Envoy proxies:

Service Mesh has Envoy proxies to manage communication between services securely. These envoys act as sidecar proxies. They run inside each pod along with our application. We don’t have to configure these envoy proxies in the deployment. YAML, Service Mesh has a control plane that automatically injects these Envoys into each pod of the cluster.

Multi-cluster service mesh


  • Why do we need Multi-Cluster Architecture?
  • How does the traffic flow in a multi-cluster service mesh?
  • Configuring certificates on both clusters
  • Enabling endpoint discovery

Why do we need Multi-Cluster Architecture?

  1. Having Multicluster support, enables users to horizontally scale their applications under a service mesh by deploying the mesh to more than one cluster. This allows users to exceed the scalability limitations of a single Kubernetes cluster.
  2. Beyond scale, in the case of cluster degradation or outage, user workloads may suffer an outage when only deployed to a single Kubernetes cluster. Multicluster enables failover and disaster recovery in these situations by deploying services to more than one cluster, as well as targeting traffic away from unhealthy clusters.

How does the traffic flow in a multi-cluster service mesh?

  1. In a multi-primary control-plane configuration, each cluster has its Istio control plane and each control plane manages its local service mesh. Using Istio gateways, a common root Certificate Authority, and automatic service discovery across multiple GKE clusters, we configure a single logical service mesh.
single logical service mesh
  1. Istio Ingress gateway is an entry point to the Kubernetes cluster. Istio Ingress gateway runs as a pod in your cluster and acts as a load balancer by accepting incoming traffic to the cluster. Gateway will then direct the traffic to one of your microservices using Virtual Service CRD.
Virtual Service CRD

Configuring certificates on both clusters

In Istio, Citadel is used for key and certificate management. It enables strong service-to-service authentication. By default, Citadel generates a self-signed root certificate and key and uses them to sign the workload certificates. If we have two clusters that maintain separate service meshes, then we need two separate Citadel services signing their respective workloads. In order to establish trust between microservices across clusters, both Citadel signing (CA) certificates must be signed by a common root certificate authority. This configuration is done by using a secret in the Kubernetes cluster.

Enabling endpoint discovery

To enable endpoint discovery for a cluster, generate a remote secret and deploy it to each cluster in the mesh. The remote secret contains credentials, granting access to the API server in the cluster. The control planes will then connect and discover the service endpoints for the cluster, enabling cross-cluster load balancing for these services.
Install a remote secret in cluster1 that provides access to cluster2’s API server. Using the following command, we can establish an endpoint discovery.

istioctl x create-remote-secret –name=central1 | kubectl apply -f – –context=cluster2


After setting up the above services, one can truly exercise the multi-cluster Mesh spread across multiple clouds, multiple services while still being on the microservices architecture. Adding to it are the benefits of auto-scaling and auto-load balancing in case of pods or clusters being down or unreachable.

Anthos allows not only to connect these diverse components but also enables users to view and manage them from a single window. If an organization needs to use services and resources across the cloud or multiple clusters within the cloud, then Anthos is the go-to tool for them.

Thought Leadership