Third Party Developer Blog

ESI ETag Best Practices

o7 Space Friends,

CCP SnowedIn coming at ya with a blog detailing a recent change to ESI response headers. As some of you have already noticed (especially if you follow me on twitter, which you should), ESI now returns an ETag header in every response. This blog is to detail exactly what this means, and how you can use this in your applications to save yourself (and us) some bandwidth.

What are ETags?

Entity Tags are described in RFC7232. ESI strives to follow this RFC, and any violations should be raised as a bug on ESI-Issues.

The tl;dr of the above RFC is there are two types of ETags, strong and weak. ESI can return either. For your application the implementation is the same, you should store and use the entire ETag header value without prejudice whether or not the ETag is strong or weak.

In your application, store the ETag independently of the cache expiry. Content that has expired can be retried with the last known ETag. If the content hasn't changed, the ETag will still be the same, so you'll receive a 304 Not Modified with no response body instead of another 200 OK with data you already have.

Example use

Let's take a look at a simple example of using python's requests module to download the ESI spec. But again, just to reiterate, this concept works on every ESI route.

>>> import requests
>>> spec_url = "https://esi.tech.ccp.is/latest/swagger.json"
>>> esi_spec = requests.get(spec_url)
>>> esi_spec.status_code
200
>>> len(esi_spec.content)
921525
>>> esi_spec.headers["ETag"]
'"09f8b2541e00231360e70eb9d4d6e6504a298f9c8336277c704509a8"'
>>> etag = esi_spec.headers["ETag"]

As you can see, we just downloaded 921525 bytes (after uncompressing) to get a 200 OK response from ESI which came with a strong ETag response header. Assuming you stored this ETag somewhere (we just saved it to etag above), you could use it in the future:

>>> esi_spec_change = requests.get(spec_url, headers={"If-None-Match": etag})
>>> esi_spec_change.status_code
304
>>> len(esi_spec_change.content)
0

We know with the second request the content hasn't changed since the first time, so we can keep using the first response we received. This saves us the transfer bandwidth, uncompressing, and JSON loading this response again.

Caching considerations

Things application developers will want to consider:

1. Store ETag values with their response data independent of the cache timers (ie; do not expire your data just because the Expires time is in the past)
2. Request using the If-None-Match header with the last known ETag as its value
3. When you get a 304 response, continue using the last known data. You can use the updated Expires and Date headers to determine when the next possible update will be

Browser applications

Any modern browser will automatically store and use the ETag properly. When we first implemented this feature we saw an immediate uptick in 304 responses due to this. If you are designing a fully client side web app, you don't need to worry about any of this as it will all be handled gracefully for you.

Summary

Implementing ETag based caching is a great way to prevent excessive amounts of electrons from being needlessly disturbed. Save the planet, request with the If-None-Match header today.

If you have any questions or are looking for help, jump in the #esi channel on tweetfleet slack where plenty of friendly devs are congregating and eagerly awaiting your arrival. If you're not on tweetfleet slack yet, you can get an invite here.

Get a sneak peek at what's coming to ESI by watching our changelog.

New to ESI? Check out our ESI quick reference.

back