layout: post
title: "Aeraki Mesh" subtitle: "Manage Any Layer-7 Protocol in Istio Service Mesh!" description: "Aeraki [Air-rah-ki] is the Greek word for 'breeze'. While Istio connects microservices in a service mesh, Aeraki provides a framework to allow Istio to support more layer 7 protocols other than just HTTP and gRPC. We hope that this breeze can help Istio sail a little further." author: "赵化冰" date: 2021-10-11 image: "https://images.unsplash.com/photo-1561562176-d1d598e49589?ixid=MnwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=1470&q=80" published: true tags:
- Istio
- Service Mesh
- Envoy
- Aeraki
categories: [ Open Source ] showtoc: true metadata:
- text: "Website"
link: "https://www.aeraki.net"
- text: "Github"
link: "https://github.com/aeraki-mesh"
- text: "Online demo"
link: "http://aeraki.zhaohuabing.com:3000/d/pgz7wp-Gz/aeraki-demo?orgId=1&refresh=10s&kiosk"
Aeraki [Air-rah-ki] Mesh is an open-source service mesh I started about a year ago, and have been working on it till recently. Aeraki is the Greek word for ‘breeze’. While Istio connects microservices in a service mesh, Aeraki Mesh provides a non-intrusive, highly extendable way to allow Istio to support none-HTTP open-source and proprietary protocols. I hope this breeze can help Istio and service mesh sail a little further.
We are now facing some challenges with service meshes:
Those obstacles make it very hard, if not impossible, for users to manage the traffic of other widely-used layer-7 protocols in microservices. For example, in a microservices application, we may have the below protocols:
If you have already invested a lot of effort in migrating to a service mesh, of course, you want to get the most out of it — managing the traffic of all the protocols in your microservices.
To address these problems, I created the open-source project, Aeraki Mesh, to provide a non-intrusive, extendable way to manage any layer 7 traffic in an Istio service mesh.
As this diagram shows, Aeraki Framework consists of the following components:
EnvoyFilter API to push the configurations to the sidecar proxies. Aeraki also serves as the RDS server for MetaProtocol proxies in the data plane. Contrary to Envoy RDS, which focuses on HTTP, Aeraki RDS is aimed to provide a general dynamic route capability for all layer-7 protocols.Dubbo and Thrift have already been implemented based on MetaProtocol. More protocols are on the way. If you're using a close-source, proprietary protocol, you can also manage it in your service mesh simply by writing a MetaProtocol codec for it.
Most request/response style, stateless protocols can be built on top of the MetaProtocol Proxy. However, some protocols' routing policies are too "special" to be normalized in MetaProtocol. For example, Redis proxy uses a slot number to map a client query to a specific Redis server node, and the slot number is computed by the key in the request. Aeraki can still manage those protocols as long as there's an available Envoy Filter in the Envoy proxy side. Currently, for protocols in this category, Redis and Kafka are supported in Aeraki.
Let’s look into how MetaProtocol works. Before MetaProtocol is introduced, if we want to proxy traffic for a specific protocol, we need to write an Envoy filter that understands that protocol and add the code to manipulate the traffic, including routing, header modification, fault injection, traffic mirroring, etc.
For most request/response style protocols, the code for traffic manipulation is very similar. Therefore, to avoid duplicating these functionalities in different Envoy filters, Aeraki Framework implements most of the common functions of a layer-7 protocol proxy in a single place — the MetaProtocol Proxy filter.
This approach significantly lowers the barrier to write a new Envoy filter: instead of writing a fully functional filter, now you only need to implement the codec interface. In addition to that, the control plane is already in place — Aeraki works at the control plane to provides MetaProtocol configuration and dynamic routes for all protocols built on top of MetaProtocol.
There are two important data structures in MetaProtocol Proxy: Metadata and Mutation. Metadata is used for routing, and Mutation is used for header manipulation.
At the request path, the decoder(the decode method of the codec implementation) populates the Metadata data structure with key-value pairs parsed from the request, then the Metadata will be passed to the MetaProtocol Router. The Router selects an appropriate upstream cluster after matching the route configuration it receives from Aeraki via RDS and the Metadata.
A custom filter can populate the Mutation data structure with arbitrary key-value pairs if the request needs to be modified: adding a header or changing the value of a header. Then the Mutation data structure will be passed to the encoder(the encode method of the codec implementation). The encoder is responsible for writing the key-value pairs into the wire protocol.
The response path is similar to the request path, only in a different direction.
If you need to implement an application protocol based on MetaProtocol, you can follow the below steps(use Dubbo as an example):
Implement the codec interface to encode and decode the protocol package. You can refer to Dubbo codec and Thrift codec as writing your own implementation.
Define the protocol with Aeraki ApplicationProtocol CRD, as this YAML snippet shows:
apiVersion: metaprotocol.aeraki.io/v1alpha1
kind: ApplicationProtocol
metadata:
name: dubbo
namespace: istio-system
spec:
protocol: dubbo
codec: aeraki.meta_protocol.codec.dubbo
You don’t need to implement the control plane. Aeraki watches services and traffic rules, generates the configurations for the sidecar proxies, and sends the configurations to the data plane via EnvoyFilter and MetaProtocol RDS.
Similar to Istio, protocols are identified by service port prefix. Please name service ports with this pattern: tcp-metaprotocol-{application protocol}-xxx. For example, a Dubbo service port should be named tcp-metaprotocol-dubbo.
You can change the route via MataRouter CRD.
apiVersion: metaprotocol.aeraki.io/v1alpha1
kind: MetaRouter
metadata:
name: test-metaprotocol-route
spec:
hosts:
- org.apache.dubbo.samples.basic.api.demoservice
routes:
- name: v2
- match:
attributes:
method:
exact: sayHello
route:
- destination:
host: org.apache.dubbo.samples.basic.api.demoservice
subset: v2
piVersion: metaprotocol.aeraki.io/v1alpha1
kind: MetaRouter
metadata:
name: test-metaprotocol-route
spec:
hosts:
- org.apache.dubbo.samples.basic.api.demoservice
routes:
- name: traffic-spilt
route:
- destination:
host: org.apache.dubbo.samples.basic.api.demoservice
subset: v1
weight: 20
- destination:
host: org.apache.dubbo.samples.basic.api.demoservice
subset: v2
weight: 80
Live Demo: Service Metrics: Grafana
Live Demo: Service Metrics: Prometheus
Screenshot: Service Metrics:
Recored Demo: Dubbo and Thrift Traffic Management
~/.kube/conf points to the cluster in the first stepgit clone https://github.com/aeraki-mesh/aeraki.git
aeraki/demo/install-demo.sh
Note: Aeraki needs to configure Istio with smart dns. If you already have an Istio installed and don't know how to turn on smart dns, please uninstall it. install-demo.sh will install Istio for you.
http://{istio-ingressgateway_external_ip}:20001http://{istio-ingressgateway_external_ip}:3000http://{istio-ingressgateway_external_ip}:9090You can import Aeraika demo dashboard from file demo/aeraki-demo.json into the Grafana.