Peeking inside OpenStack keystone Fernet token payloads

I've been asked several times now how to go about peeking into the payloads of Fernet tokens, rather than just deconstructing the high-level token structure. They're not quite as straightforward as PKI, where a simple Base64 decoding will get you quite far, nor as opaque and lifeless as UUID tokens.

So given a Fernet token like the one below, how do you know what's inside?

gAAAAABV-fV0dII0Izfc-JUjMvWU2GJG7ov87TiLzgDyX9tzTdtLdYu5Tb1OxPkKwW6hjSfDV9URIdug9KBVOJN77lL_baUpH4ze37Yjz1MFYfCPY0kDNQ43iBXBMQzzEB31kkwxBDY1scYFrMKoKthvKFZzXa89k3r7HWe1Y4ShyXsC0FxCUUw

Using the HTTP API

First of all, if you're human and you only want to see what a Fernet token represents, you just need to validate it against the v3 API and keystone will happily do all the heavy lifting for you:

And you'll get a giant response back like this:

Sort of human readable, right? At least it's a documented API and things are intuitively labeled.

But let's dive deeper!

Validating with the Fernet token formatter

If you really want to see the data stored inside a Fernet token, you must bypass the HTTP API. You have to validate the token directly against a Fernet token formatter instance in the same keystone deployment that created the token, which means digging into implementation details of Fernet's validate_token(). To call that function directly:

And you'll get this:

Unpacking with the Fernet token formatter

Finally, if you want to dig as far as you can go, you can unpack the raw, binary serialized payload using the same formatter object:

Which will reward you with the raw payload data in the token:

At this point, this is data which only keystone can truly understand (as illustrated by the higher level interfaces discussed above). Each element here has been compressed to consume as few bytes as possible, and the structure of this data varies almost entirely based on the use case of the token (unscoped, project-scoped, federated, etc).