Crossroads

The main benefit of the Cloud Foundry service mesh is the weighted routing option. Routes can now be assigned to more than one application, a feature many members of the Cloud Foundry community have had on their wish list for several years. The growing popularity of the service mesh idea created a critical mass that led to the idea finally being implemented. (See also the "Microservices and the Service Mesh" box.) The function can be used to define a weighting that distributes the load among the applications involved. The weighting factor of each application statistically determines how many incoming requests it receives on average.

Cloud Foundry

For historical reasons, Cloud Foundry is often equated with the Cloud Foundry Application Runtime (CFAR) platform (and its famous cf push user experience), which includes many projects accumulated under the umbrella of the Cloud Foundry Foundation.

CFAR distinguishes itself from the Cloud Foundry Container Runtime (CFCR), a BOSH-powered variant of Kubernetes. BOSH is an orchestration tool independent of the infrastructure and operating system that allows automation of stateful, distributed applications throughout the application life cycle. It is suitable for coping with large application loads using CFAR or CFCR, as well as for the automated operation of database systems.

Compared with newcomer Kubernetes, Cloud Foundry is almost a prehistoric beast among modern application platforms. The efficient operation of a large number of applications has always been a priority for Cloud Foundry. For example, it was already a service mesh before the term became popular, and Cloud Foundry has always sent incoming requests from an application through a central router and distributed them from there to the application instances assigned to the request. Without such a routing function, self-healing of failed application instances in distributed container clusters (aka Diego cells) would be impossible.

Assigning requests to application instances requires a regular exchange between the router and the container hosts. Applications in Cloud Foundry have always been available from internal and external URLs that abstract the underlying application instances and automatically distribute the load of incoming requests. These application URLs also allow clean communication between the services of a microservice architecture. This setup already resembles the idea of service discovery. Centralized storage of log output is one of Cloud Foundry's native resources. Other tools are only needed to merge the information from application and data services.

Cloud Foundry with Service Mesh

Although CFAR is successful and proven, other technologies are developing progressively. The most prominent representative is certainly Kubernetes, which has adapted the service mesh concept quickly and profoundly. Kubernetes makes using Istio and Envoy child's play: They can be deployed easily in the form of sidecar containers to support applications. This capability increases the pressure on Cloud Foundry to offer a corresponding service mesh functionality, as well. What is surprising about the introduction of Istio and Envoy in Cloud Foundry is that it is only now being implemented; the service mesh subsystem is currently still beta.

The service mesh in Cloud Foundry is designed as an optional extension and can be added to a CFAR as needed. The installation creates an additional routing subsystem in addition to the existing versions for HTTP and TCP traffic. New system domains for Istio (*.istio) and mesh routing (*.mesh) are added. As with all Cloud Foundry routing systems, a load balancer must be installed upstream to distribute incoming requests to the router instances. The load is distributed in two stages by classic, and more static, load balancers and dynamic routers with app awareness (Figure 1).

Integration into the Cloud Foundry command-line interface is still incomplete; cf commands are not available for all functions. Instead, users have to run curl commands against the APIs. Because the Cloud Foundry service mesh is still in beta, not every Cloud Foundry provider supports it yet. However, if you have the provider's own Cloud Foundry environment, you can test the system. The service mesh developers are happy to receive feedback.

Weighted Routing

Weighted routes open up a wide range of new use cases for developers, including blue-green deployments, A/B testing, and canary releases. Blue-green deployments let you deploy an application twice. A separate route is assigned to each deployment, and the assigned weighting factors can be used to distribute the traffic among the applications. One of the two installations can be removed temporarily from load distribution, such as for updates. During the update, the other application installation serves the incoming requests. After the update, the traffic is switched to the new installation. In the case of an unexpected error, however, you can immediately switch back to the unchanged, older application version. If the system behaves as expected, you can then update the second application.

Similarly, distributing traffic to multiple application installations can be used for multivariate testing. Where two variants exist, an A/B test is created that confronts different user groups with different application versions to draw conclusions about acceptance from the observed user behavior. In simple tests (e.g., where two different back-end implementations are competing against each other), weighted routes can help in the testing. However, they only partially cover the requirements of such A/B tests: Many cases require a conscious assignment of users to the individual variants, for which other tools are needed.

In the case of canary releases, a new function is not made available to the entire user community at once. Like A/B testing, the user group is split. One section of the community gathers initial experiences that can be evaluated by the testers before everyone benefits from the new version.

Using the Service Mesh

Once the Cloud Foundry service mesh has been installed successfully, the admin can create weighted routes. As mentioned before, the cf command-line tool does not yet support all commands, so admins have to run the commands directly against the Cloud Controller API with the help of the widely used Curl [3].

To begin, you need to create a route, which names a URL and thus makes it known to the main component in Cloud Foundry routing: the CFAR Gorouter:

# cf create-route development mesh.apps.anynines.com -n testapp

Note that the route is connected to a space at this point in time, but not to an application as yet. Associating a route with an application – route mapping – involves a separate step. First, however, you must take care of some preparatory work because integration is not complete. To begin, you must determine the route's GUID, which can be found in the resources[metadata] field (Listing 1, line 10).

01 $ cf curl /v2/routes?q=host:testapp
02 {
03   "total_results": 2,
04   "total_pages": 1,
05   "prev_url": null,
06   "next_url": null,
07   "resources": [
08     {
09       "metadata": {
10         "guid": "83d60b98-5864-45c6-94ad-4d02f5f4216a",
11         "url": "/v2/routes/83d60b98-5864-45c6-94ad-4d02f5f4216a",
12         "created_at": "2019-11-09T10:40:04Z",
13         "updated_at": "2019-11-09T10:40:04Z"
14       },
15       "entity": {
16         "host": "tired-duck-brash-civet",
17         "path": "",
18         "domain_guid": "fb6bd89f-2ed9-49d4-9ad1-97951a573135",
19         "space_guid": "d6cddaa6-0886-44b3-9590-16717d5cd3c2",
20         "service_instance_guid": null,
21         "port": null,
22         "domain_url": "/v2/shared_domains/fb6bd89f-2ed9-49d4-9ad1-97951a573135",
23         "space_url": "/v2/spaces/d6cddaa6-0886-44b3-9590-16717d5cd3c2",
24         "apps_url": "/v2/routes/83d60b98-5864-45c6-94ad-4d02f5f4216a/apps",
25         "route_mappings_url": "/v2/routes/83d60b98-5864-45c6-94ad-4d02f5f4216a/route_mappings"
26       }
27     }
28   ]
29 }

Creating a weighted route in the form of a route mapping also requires you to specify an application, which you reference through its GUID. The app GUID is easy to determine:

$ cf app testapp --guid a4238936-3f25-463b-4061-d3436e277be3

Now you can create the route mapping, which also includes weighting for the associated application (Listing 2). The weighting factor in this example is 2, which means that this route weighting is twice that of a route with the default value 1. The statistical distribution of the requests to the individual route mappings is calculated by simple averaging. The individual weighting factors of all mappings are divided by their sum, giving the relative distribution ratios. In the case of two routes with weightings of 1 and 2, the result is 1/3 to 2/3; 33 percent of the requests go to the first application and 66 percent to the second (Figure 2).

$ curl /v3/route_mappings -X POST -d '{
  "relationships": {
    "app": {
      "guid": "a4238936-3f25-463b-4061-d3436e277be3"
    },
    "route": {
      "guid": "83d60b98-5864-45c6-94ad-4d02f5f4216a"
    }
  },
  "weight": 2
}'

To complete the above example, you therefore need to create another application and route mapping with the steps presented here. More detailed information on creating and modifying weighted routes can be found in the Cloud Foundry documentation [4].

Conclusions

The introduction of the service mesh can be seen as the Cloud Foundry developers' response to increasing demand. Developers of microservice-based applications need tools such as weighted routes to meet modern product and software development requirements. Comparing the use cases that service mesh technologies such as Istio and Envoy enable in the context of Kubernetes, it is reasonable to assume that these will also follow in Cloud Foundry.

Weighted routing could be followed by support for the Circuit Breaker pattern, rate limiting, and outlier detection. When and how this will happen remains to be seen; CFAR may be adapted step by step. Alternatively, incubation projects from the Cloud Foundry Foundation such as Eirini [5] could facilitate adaptation if the existing CFAR container subsystem is replaced by a Kubernetes back end.