Understanding Codable by going behind the scenes

Most of us who have been using Swift to build apps have exchanged data with servers, and this data is usually in some well known format such as JSON, XML etc. The strong type-safety of Swift does not really play well with such formats and it is a mess to decode, for example a JSON, into a native Swift type. To help with this, the sensational open source community, as always, built libraries including the venerable SwiftyJSON, so that tyros such as me can keep our jobs, but given that this is such a common use case, wouldn’t it be nice if the language had some more support?

September 19, 2017: Codable enters the chatroom.

If you took a survey of Swift developers asking for the GOAT (greatest of all time) addition to the Swift language, I am willing to wager that Codable will rank near the very top. Codable, as the name implies (actually it doesn’t), is a typealias for the two protocols Encodable & Decodable. In this article, I will share some of my knowledge of the language and compiler support that makes Codable so useful, and how we can perform some basic customizations of the encoding and decoding processes.

Decoding is the translation of information so that you can understand what it means. In our case, converting JSON into something Swift understands (an instance of a Swift class, for instance). Encoding is, well, the opposite. Throughout this post, I will use JSON as my format of choice, because that is what I am most familiar with, but rest assured that Codable is not specific to JSON. Along with Codable, two built-in encoders/decoders were added (for JSON and PropertyList) to the language, but it should be possible to add your own coders (my typealias for encoder & decoder) for any representation of data.

Also throughout this post, I will use the Decodable protocol for my explicatory material, but similar concepts should apply to the En variants as well.

JSON

JSON is short for Javascript object notation. A Javascript object contains property-value pairs where properties are strings and values are javascript types – integers, strings, floats, booleans, null, arrays (of any JS type) and javascript objects themselves (yeah, yeah, yeah, I am sure there is a mistake in there). As you can imagine, the nesting can be any level deep, but, regardless, this has a nice mapping with the idea of Swift types that have properties of any Swift type.

Let’s use Decodable to convert this JSON:

let jsonData = """
{
  "name": "yolo",
  "rank": 5
}
""".data(using: .utf8)!

to an instance of the Player type:

struct Player: Decodable {
  let name: String
  let rank: Int
}

I carefully chose this example so that both the properties and the types of values line up well. To perform the conversion, all we need to do is add the Decodable protocol conformance to the struct – already done above – and then do: let player = try JSONDecoder().decode(Player.self, from: jsonData)

And voila, that’s all it takes! This works because all the standard Swift types  (Strings, Ints, Arrays of them, etc.) are already Decodable. Now that Player is decodable, you can compose other Swift Types that use this type of property, and the compiler will do the heavy lifting if you just add the protocol conformance. What this means is that nearly all your types can have auto decoding support added to them as long you carefully conform your building blocks to Decodable.

So, what do I mean when I say that the compiler does the heavy lifting? The generated code looks something like this:

struct Player: Decodable {
  
  enum CodingKeys: CodingKey {
    case name
    case rank
  }

  let name: String
  let rank: Int

  init(from decoder: Decoder) throws {
    let container = try decoder.container(keyedBy: CodingKeys.self)
    self.name = try container.decode(String.self, forKey: .name)
    self.rank = try container.decode(Int.self, forKey: .rank)
  }
}

Let’s break this down one piece at a time:

Decodable is a protocol with one requirement – an initializer of the form init(from decoder: Decoder) throws and its only input parameter is a type that conforms to the Decoder protocol. Decodable is the type that will be generated from the payload, whereas the Decoder is the one that decodes (duh!) the payload. This initializer is the one generated for us.

The Decoder protocol requires its conforming type to provide, among a few other things, an instance method that returns a KeyedDecodingContainer. For our purposes, we can think of the keyed container as a view into the underlying JSON payload similar to a dictionary. 

The CodingKeys enum specifies all the keys that will be used to prepare the keyed container. The auto-generated CodingKeys matches the stored properties of the Decodable type. This enumeration itself conforms to the CodingKey protocol. One of its requirements is a stringValue property, which is presumably used when keying into the container. The language has some in-built support that generates the stringValue for each of the cases in the CodingKeys.

Once we have the container ready, we can read the values just like you would read values from a dictionary. To read a value, we use the decode(_:forKey) method on the keyed container. The container already has many overloaded decode methods defined for the primitive Swift types, and these are used in our example.

Let’s take a small step back and think about the JSONDecoder used in let player = try JSONDecoder().decode(Player.self, from: jsonData). This looks suspiciously like something that conforms to Decoder, but a little bit of research (into the documentation and JSONDecoder.swift) reveals that it doesn’t. There is a private class internal to JSONDecoder that does conform to Decoder which is, as it turns out, used to call the Decodable initializer with. 

If the decodable type is defined with optional properties, the decoding succeeds even if the JSON does not contain the corresponding key (or the value is null). This is because the auto-generated initializer uses the decodeIfPresent(_:forKey:) method on the keyed container, which returns an optional.

I reiterate that all this is done behind the scenes, but we can use this knowledge to start customizing the decoding process.

Customized Keys

What if you want to change the mapping between the keys and properties, because your preferred way to name properties differs from the server conventions. Imagine the incoming json to say first_name which you want to be parsed into name

struct Player {
  enum CodingKeys: String, CodingKey {
    case name = "first_name"
    case rank
  }
}

We provide our own CodingKeys enumeration inside the type. We set the raw value type to String, and the expected JSON key as the raw value for the particular case. This helps the compiler to set the stringValue to what we want. So when it peeps into the container, it uses “first_name” as the key instead of “name”.

Note that all the other keys have to be mentioned in the enumeration, even if only one of them is being customized. All the manually provided keys are used for decoding – no more, no less.

Default Values

If we want our properties to be non-optional, and the JSON not containing the key is not considered an error, we can provide a default value by writing our own initializer and saying:

init(from decoder: Decoder) throws {
  let container = try decoder.container(keyedBy: CodingKeys.self)
  self.name = try container.decode(String.self, forKey: .name)
  self.rank = try container.decodeIfPresent(Int.self, forKey: .rank) ?? -1 // default
}

Once again, all the other keys have to be decoded ourselves to match what would have been generated by the compiler.

Conclusion

That brings this blog post to an abrupt end and I hope I was able to reveal some of the tricks behind the compiler auto-magic. Do let me know in the comments if you have any questions or requests. Subscribe to be notified of more Codable stories!

Further reading

Codable Swift Evolution Proposal – This one’s an absolute gem and you are better off reading this instead of my blog post, so I cleverly put this at the bottom.