layout: post title: "Beyond Gateway API: Introducing Envoy Gateway's Gateway API Extensions" subtitle: description: 'As the official Gateway Controller for the Envoy, Envoy Gateway provides full support for all the features of the Kubernetes Gateway API. In addition, Envoy Gateway extends the Gateway API by introducing a range of enhancements for traffic management, security features, and custom extensions that go beyond the standard API. In this post, we’ll dive into these Envoy Gateway extensions and explore their use cases.' author: "Huabing Zhao（Envoy Gateway Maintainer）" date: 2024-08-31 image: "/img/2024-08-31-introducing-envoy-gateways-gateway-api-extensions/IMG_1624.JPG" published: true tags: [Envoy, Envoy Gateway] categories: [Tech,Open Source]
This article is a summary of my talk,"Gateway API and Beyond: Introducing Envoy Gateway's Gateway API Extensions¹", presented at KubeCon China in Hong Kong, August 2024.
{{< youtube qH2byF7SDO8 >}}
As the official Gateway Controller for the Envoy, Envoy Gateway² provides full support for all the features of the Kubernetes Gateway API³.In addition, Envoy Gateway extends the Gateway API by introducing a range of enhancements for traffic management, security features, and custom extensions that go beyond the standard API. In this post, we’ll dive into these Envoy Gateway extensions and explore their use cases.
Ingress⁴ is a Kubernetes API resource used to define rules for managing inbound traffic to a cluster. While the Ingress API provides users with basic capabilities for defining HTTP routing rules, <font color="red"> **its functionality is quite limited, providing only fundamental features such as Host-based routing, Path-based routing, and TLS termination.** </font>.
In practice, the basic functionality of the Ingress API often falls short of meeting the complex traffic management requirements of modern applications. As a result, various Ingress Controller implementations have extended the Ingress API using non-standard methods like annotations or custom API resources.
For example, a common requirement is to match request paths using regular expressions. However, the Ingress API only supports Prefix and Exact path matching, which is insufficient to meet this need.
To address this relatively simple requirement, some Ingress Controllers have introduced annotations to support regex path matching. For example, the NGINX Ingress Controller provides the nginx.org/path-regex annotation for this purpose.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: cafe-ingress
annotations:
nginx.org/path-regex: "case_sensitive"
spec:
rules:
- http:
paths:
- path: "/tea/[A-Z0-9]+"
backend:
serviceName: tea-svc
servicePort: 80
- path: "/coffee/[A-Z0-9]+"
backend:
serviceName: coffee-svc
servicePort: 80
Other controllers, like Traefik, take a different approach, using custom resources like IngressRoute to achieve the same result.
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: cafe-ingress
namespace: default
spec:
routes:
- match: "PathPrefix(`^/tea/[A-Z0-9]+`)"
kind: Rule
services:
- name: tea-svc
port: 80
- match: "PathRegexp(`^/coffee/[A-Z0-9]+`)"
kind: Rule
services:
- name: coffee-svc
port: 80
Whether it’s through annotations or custom API resources, <font color="red"> **these non-standard extensions hurt the portability of the Ingress API. Users have to relearn and reconfigure different API setups when switching between Ingress Controllers** </font>. This fragmentation makes things more complicated and slows down community progress, making it tougher for the Kubernetes ecosystem to maintain and evolve the API.
To address the limitations of the Ingress API, the Kubernetes community introduced the next generation of Ingress API, known as the Gateway API. This new API specification aims to provide a unified, scalable, and feature-rich way to define rules for managing inbound traffic to a cluster.
Compared to the Ingress API, the Gateway API offers a lot more functionality. It defines multiple resource types, including Gateway, HTTPRoute, GRPCRoute, TLSRoute, TCPRoute, and UDPRoute. It also gives you more configuration options for traffic routing, such as Path matching, Header matching, Host matching, TLS configuration, traffic splitting, request redirection, and more. Many features that previously required annotations or custom API resources can now be handled directly through the Gateway API.
For example, here’s a Gateway API resource that defines an HTTPRoute, implementing the regular expression-based path matching from the earlier example.
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: cafe-httproute
spec:
parentRefs:
- name: eg
rules:
- matches:
- path:
type: RegularExpression
value: "^/tea/[A-Z0-9]+"
backendRefs:
- name: tea-svc
port: 80
- matches:
- path:
type: RegularExpression
value: "^/coffee/[A-Z0-9]+"
backendRefs:
- name: coffee-svc
port: 80
Although the Gateway API offers more functionality than Ingress, it’s important to remember that <font color="red"> **any standard, no matter how well-defined, can only serve as the lowest common denominator across all implementations** </font>. The Gateway API is no exception. Because the Gateway API is designed as a universal API specification to ensure wide compatibility, it cannot directly support features that are closely linked to specific implementation details.
For instance, although features like rate limiting and access control are essential in real-world scenarios, they are implemented differently across data planes like Envoy and NGINX. Because of these differences, the Gateway API cannot offer a universal standard for such functionalities. This is also why the Ingress API saw a proliferation of annotations and custom API resources to fill those gaps.
<font color="red"> **A key innovation of the Gateway API is the [Policy Attachment⁵](https://gateway-api.sigs.k8s.io/reference/policy-attachment/) mechanism, which allows controllers to extend the API’s capabilities through custom policies without modifying the Gateway API itself**</font>. By associating custom policies with resources like Gateway and HTTPRoute, this feature enhances the API’s flexibility and enables advanced traffic management, security, and custom extensions.
In addition to Policy Attachment, the Gateway API aslo supports other extension mechanisms, such as linking custom Backend resources to HTTPRoute and GRPCRoute for routing traffic to non-standard backends, as well as adding custom Filter resources for handling requests and responses.
With these built-in extension mechanisms, the Gateway API strikes a balance between keeping core resources like Gateway and HTTPRoute general enough for broad compatibility, while also providing a standardized way for different controllers to extend functionality. This allows different Ingress Controller implementations to build on the Gateway API’s core resources and offer enhanced features through custom policies, backends, and filters.
Envoy is a powerful cloud-native proxy widely used in service mesh, API gateway, and edge proxy scenarios, offering advanced traffic management capabilities and flexible configuration options. However, configuring Envoy as an Ingress Gateway can be challenging, often requiring users to write hundreds or even thousands of lines of configuration—on top of the complexity of deploying and managing the Envoy instances themselves.
To make configuring and managing Envoy easier, the Envoy community introduced the Envoy Gateway project. <font color=red> **Envoy Gateway is an Ingress Gateway built on Envoy, designed to provide a streamlined, user-friendly experience for managing Envoy as an API Gateway**</font>. It uses the Gateway API as its configuration language, fully compatible with the latest Gateway API version, and supports resources like Gateway, HTTPRoute, GRPCRoute, TLSRoute, TCPRoute, and UDPRoute.
Moreover, Envoy Gateway leverages the Gateway API’s extension mechanisms to offer a rich set of additional features, , including rate limiting, access control, WebAssembly extensions, and more, extending beyond the capabilities of the standard Gateway API.
Envoy Gateway introduces the following custom resources:
The relationship between these custom resources and the standard resources of the Gateway API is illustrated in the diagram below:
Special thanks to Erica Hughberg for drwaing this diagram. She also created many other amazing visuals to explain complex technical concepts in a simple and intuitive way. Follow her on LinkedIn to see more of her work.
This article will focus on the custom policy resources provided by Envoy Gateway, as HTTPRoute Filters and Custom Backends are relatively straightforward and self-explanatory.
Next, let's take a closer look at Envoy Gateway's Gateway API extension features and explore their use cases.
Policy Attachment⁵ is an extension mechanism provided by the Gateway API, allowing a policy to be attached to resources like GatewayClass, Gateway, HTTPRoute, GRPCRoute, and Service to provide additional capabilities. Envoy Gateway leverages the Policy Attachment mechanism to implement various policies, exposing Envoy's powerful traffic management capabilities at the Gateway level.
A policy can be attached to different lelves in the Gateway API resource hierarchy, and multiple policies can be attached to the same resource. The scope and priority of Policy Attachment in Envoy Gateway are defined as follows:
ClientTrafficPolicy is a Policy Attachment resource in Envoy Gateway designed to configure traffic between the client and Envoy. The diagram below illustrates how ClientTrafficPolicy works:
As shown in the diagram, ClientTrafficPolicy is applied before Envoy processes request routing. This means ClientTrafficPolicy can only be applied to Gateway resources and cannot be used with HTTPRoute or GRPCRoute resources.
ClientTrafficPolicy provides the following configuration for the client-Envoy connection:
Below is an example of a ClientTrafficPolicy:
The client-traffic-policy-gateway is a ClientTrafficPolicy resource attached to the eg Gateway resource. It configures traffic between the client and Envoy, setting various parameters including TCP Keepalive, Connection Buffer Size, HTTP Request Timeout, HTTP Idle Timeout, and how to obtain the client’s original IP address. Since the eg Gateway resource has two Listeners—http and https—this ClientTrafficPolicy will apply to both.
Additionally, the client-traffic-policy-https-listener is another ClientTrafficPolicy resource linked directly to the https Listener (by specifying the sectionName field in its targetRef). It overrides the client-traffic-policy-gateway configuration for the https Listener, allowing specific TLS-related parameters to be applied.
BackendTrafficPolicy is similar to ClientTrafficPolicy but focuses on configuring traffic between Envoy and backend services. The diagram below illustrates how BackendTrafficPolicy works:
The BackendTrafficPolicy is applied during the request routing stage, allowing it to be used with both the Gateway and the HTTPRoute and GRPCRoute resources.
Note: When a BackendTrafficPolicy is applied to the Gateway, it will effectively impact all HTTPRoute and GRPCRoute resources associated with that Gateway.
BackendTrafficPolicy provides the following configuration options for managing traffic between Envoy and backend services:
Below is an example of BackendTrafficPolicy:
The backend-traffic-policy-http-route is a BackendTrafficPolicy resource attached to an HTTPRoute named http-route. It is used to configure traffic between Envoy and backend services. This BackendTrafficPolicy configures global rate limiting, load balancing strategies, and circuit breaker policies for the backend connections.
If you have ever configured rate limiting in Envoy or Istio, you know how complex it can be. You need to write a lot of configuration code to define the rate limiting policy, set up the rate limiting service, and configure the rate limiting filter in Envoy. The configuration is often scattered across multiple files, making it difficult to manage and maintain.
This is where BackendTrafficPolicy comes in. As shown in the example above, you can easily configure global rate limiting with just a few lines of YAML code, significantly reducing complexity for users. BackendTrafficPolicy abstracts the rate limiting configuration into a single resource, making it easier to manage and maintain.
SecurityPolicy is used for access control, including CORS policies, Basic Auth, OIDC/OAuth, JWT Authentication, IP-based access control, JWT Claims-based access control, and External Authentication, etc. The diagram below illustrates how SecurityPolicy works:
Please note that the diagram above is a conceptual representation; there isn’t a dedicated “Access Control” component within Envoy Gateway. Instead, Envoy Gateway leverages Envoy’s filter chain to apply SecurityPolicy configurations for controlling access to backend services.
SecurityPolicy provides the following access control configurations:
Below is an example of SecurityPolicy:
The security-policy-http-route is a SecurityPolicy resource attached to an HTTPRoute named http-route, it is configured with OIDC authentication and IP-based access control.
By leveraging SecurityPolicy, you can offload the security policies enforcement—such as user authentication and access control—from your application code to Envoy Gateway, significantly simplifying your application code while greatly enhancing its security posture. Envoy Gateway offers out-of-the-box security policies that support various user authentication and authorization methods. Addtionaly, if you need to integrate with your legacy auth services, you can easily do so through ExtAuth.
While Envoy Gateway offers extensive traffic management features, there may be specific requirements that its built-in capabilities can’t address. In such cases, you can extend Envoy’s functionality using EnvoyExtensionPolicy. This policy allows you to load custom extensions into Envoy to execute user-defined logic for request and response processing.
EnvoyExtensionPolicy supports two types of extensions:
Envoy Gateway enhances Envoy’s native Wasm support by allowing Wasm extensions to be packaged as OCI Images. This means you can bundle your custom Wasm extensions into OCI Images, store them in container registries, and load them through EnvoyExtensionPolicy.
OCI Image support offers version control and enhanced security for Wasm extensions. You can specify the extension’s version via image tags and enforce accee control by storing them in private registries. Plus, you can leverage the broader OCI ecosystem for packaging, distributing, and managing Wasm extensions.
The diagram below illustrates how Wasm OCI Image works in Envoy Gateway:
In addition to OCI Images, Envoy Gateway also supports loading Wasm extensions via HTTP URLs. You can upload your Wasm extensions to an HTTP server and specify the URL in the EnvoyExtensionPolicy.
Below are examples of Wasm extensions, the ExtensionPolicy on the left uses an OCI Image to load the Wasm extension, while the one on the right uses an HTTP URL:
External Process extensions are another extension mechanism offered by Envoy Gateway. This allows users to process requests and responses through an external service. The external process must be deployed separately, and Envoy Gateway communicates with it via remote procedure calls (RPC) to handle the request and response processing. This provides flexibility for running custom logic outside of Envoy’s core, enabling more advanced use cases.
The diagram below illustrates how External Process extensions work:
When low network latency in the data path is required, you can deploy the External Process extension as a sidecar within the same Pod as Envoy Gateway. This deployment mode allows you to replace remote procedure calls (RPC) with local Unix Domain Socket (UDS) calls, significantly reducing latency.
As shown in the diagram, the External Process extension runs inside the same Pod as Envoy and communicates with Envoy using Unix Domain Sockets (UDS). You’ll need to create a Backend resource to define the UDS address for the External Process, and reference this Backend in the EnvoyExtensionPolicy.
Both WebAssembly (Wasm) and External Process extensions offer powerful ways to extend Envoy Gateway. But how do you choose between them? Here are a few factors to consider:
In general, Wasm extensions are a good fit for lightweight, in-path data processing tasks, while External Process extensions are better suited for more complex logic that requires interaction with external systems. Your choice will depend on the specific needs of your application and environment.
In general, Wasm extensions are well-suited for lightweight, data-path processing tasks, while External Process extensions are better for handling more complex logic that involves interaction with external systems. The right choice depends on the specific requirements of your application and environment.
Envoy Gateway simplifies managing Envoy through the Gateway API and its policy extensions. While these configuration resources handle most use cases, there are always some edge cases that aren’t fully covered. In such instances, users can use EnvoyPatchPolicy to apply arbitrary patches to the generated Envoy configuration to meet specific requirements.
The diagram below illustrates how EnvoyPatchPolicy works:
By default, EnvoyPatchPolicy is disabled and must be explicitly enabled in the Envoy Gateway configuration. Once enabled, it allows users to add arbitrary patches to the Envoy configuration generated by Envoy Gateway, including modifications to Listener, Cluster, and Route configurations.
In this example, the EnvoyPatchPolicy resource modifies the Listener default/eg/http in the generated Envoy configuration. It adds the localReplyConfig parameter to the first filter in the Listener’s Default Filter Chain, which is the envoy.http_connection_manager responsible for handling HTTP traffic. This change updates the response code for 404 errors to 406 and sets the response body to “could not find what you are looking for.”
EnvoyPatchPolicy is a powerful tool, but it can be risky, as it relies on the structure and field naming conventions of the generated Envoy configuration, neither of which are guaranteed to be stable and may change with future updates to Envoy Gateway. Therefore, it should be used judiciously and only when necessary.
EnvoyPatchPolicy is generally recommended in the following scenarios:
Before creating an EnvoyPatchPolicy, you can use the egctl tool to view the original Envoy configuration. This allows you to identify the places where you need to make changes.
egctl config envoy-proxy all -oyaml
After writing the EnvoyPatchPolicy, you can also use the egctl tool to validate whether the patched Envoy configuration meets your expectations.
egctl experimental translate -f epp.yaml
Keep in mind that upgrading Envoy Gateway can lead to changes in the Envoy configuration, which might cause your existing EnvoyPatchPolicy to no longer work as expected. <font color=red> **When upgrading, it’s important to review and assess whether the current EnvoyPatchPolicy is still applicable or needs to be updated** </font>.
The Gateway API is the next-generation Ingress API in Kubernetes, offering a rich set of features for managing inbound traffic to clusters. Envoy Gateway is an ingress gateway built on Envoy that fully supports the capabilities of the Gateway API while providing a wide range of enhancements through the API’s extension mechanisms.
Envoy Gateway offers several advanced policies—including ClientTrafficPolicy, BackendTrafficPolicy, SecurityPolicy, EnvoyExtensionPolicy, and EnvoyPatchPolicy—which can be attached to Gateway API resources like Gateway, HTTPRoute, and GRPCRoute to enable fine-grained traffic control. With these policies, users can fine-tune the behavior of client and backend connections, enforce access control, and implement custom extensions, unlocking the full power of Envoy to manage edge traffic.