Caching webservice with Varnish

The primary goal is to find a way to reduce the load of a webservice by caching its responses.

The webservice is a RESTfull API serving as a gateway between a private HATEOAS API and a client generating more than 500 000 requests a day.

The first surprise is that if your well educated client, sending you a header Authorization: Bearer, will not be cached by default by Varnish !

Let's force back the standard behaviour with this header for our webservice uri prefix:

sub vcl_recv {
        # Force cache response even with req.http.Authorization set
        if (req.http.Authorization) {
                if (req.url ~ "^/webservice/uri/prefix/") {
                        return (lookup);
                }
        }
}

This has security implication, because anyone allowed to request varnish will be able to retrieve a cached result without authentification.

It is important to validate the Authorization header value before serving the result from cache.

Now, our webservice has three possibles answers :

• 200: the data in JSON
• 404: data was not found
• 410: data is not available anymore

Let's cache our results depending on the reponse code:

sub vcl_fetch {
       if (req.url ~ "^/webservice/uri/prefix/") {
                if (beresp.status == 404) {
                        set beresp.ttl = 7d;
                }
                if (beresp.status == 410) {
                        set beresp.ttl = 7d;
                }
                if (beresp.status == 200) {
                        set beresp.ttl = 24h;
                }
        }
}

With this configuration, we divided by 5 the quantity of request on our gateway from the client who was not able to cache our result himself.