JSON decoding

Just like JSON encoding, there are two approaches that you can take to decode JSON into a native Go value: using a json.Decoder type or using the json.Unmarshal() function.

Both approaches have their pros and cons, but for the purpose of decoding JSON from an HTTP request body, using json.Decoder is generally the best choice. It’s more efficient than json.Unmarshal(), requires less code, and offers some helpful settings that you can use to tweak its behavior.

It’s easiest to demonstrate how json.Decoder works with code, rather than words, so let’s jump straight in and update our createMovieHandler like so:

package main

import (
    "encoding/json" // New import
    "fmt"
    "net/http"
    "time"

    "greenlight.alexedwards.net/internal/data"
)

...

func (app *application) createMovieHandler(w http.ResponseWriter, r *http.Request) {
    // Declare an anonymous struct to hold the information that we expect to be in the 
    // HTTP request body (note that the field names and types in the struct are a subset
    // of the Movie struct that we created earlier). This struct will be our *target 
    // decode destination*.
    var input struct {
        Title   string   `json:"title"`
        Year    int32    `json:"year"`
        Runtime int32    `json:"runtime"`
        Genres  []string `json:"genres"`
    }

    // Initialize a new json.Decoder instance which reads from the request body, and 
    // then use the Decode() method to decode the body contents into the input struct. 
    // Importantly, notice that when we call Decode() we pass a *pointer* to the input 
    // struct as the target decode destination. If there was an error during decoding,
    // we use our generic errorResponse() helper to send a 400 Bad Request response
    // with the error message to the client.
    err := json.NewDecoder(r.Body).Decode(&input)
    if err != nil {
        app.errorResponse(w, r, http.StatusBadRequest, err.Error())
        return
    }

    // Dump the contents of the input struct in an HTTP response.
    fmt.Fprintf(w, "%+v\n", input)
}

...

There are a few important and interesting things about this code to point out:

• When calling Decode() you must pass a non-nil pointer as the target decode destination. If you don’t use a pointer, it will return a json.InvalidUnmarshalError error at runtime.

• If the target decode destination is a struct — like in our case — the struct fields must be exported (start with a capital letter). Just like with encoding, they need to be exported so that they’re visible to the encoding/json package.

• When decoding a JSON object into a struct, the key-value pairs in the JSON are mapped to the struct fields based on the struct tag names. If there is no matching struct tag, Go will attempt to decode the value into a field that matches the key name (exact matches are preferred, but it will fall back to a case-insensitive match). Any JSON key-value pairs which cannot be successfully mapped to the struct fields will be silently ignored.

• There is no need to close r.Body after it has been read. This will be done automatically by Go’s http.Server, so you don’t have to.

OK, let’s take this for a spin.

Fire up the application, then open a second terminal window and make a request to the POST /v1/movies endpoint with a valid JSON request body containing some movie data. You should see a response similar to this:

# Create a BODY variable containing the JSON data that we want to send.
$ BODY='{"title":"Moana","year":2016,"runtime":107, "genres":["animation","adventure"]}'

# Use the -d flag to send the contents of the BODY variable as the HTTP request body.
# Note that curl will default to sending a POST request when the -d flag is used.
$ curl -i -d "$BODY" localhost:4000/v1/movies
HTTP/1.1 200 OK
Date: Tue, 06 Apr 2021 17:13:46 GMT
Content-Length: 65
Content-Type: text/plain; charset=utf-8

{Title:Moana Year:2016 Runtime:107 Genres:[animation adventure]}

Great! That seems to have worked well. We can see from the data dumped in the response that the values we provided in the request body have been decoded into the appropriate fields of our input struct.

Zero values

Let’s take a quick look at what happens if we omit a particular key-value pair in our JSON request body. For example, let’s make a request with no year in the JSON, like so:

$ BODY='{"title":"Moana","runtime":107, "genres":["animation","adventure"]}'
$ curl -d "$BODY" localhost:4000/v1/movies
{Title:Moana Year:0 Runtime:107 Genres:[animation adventure]}

As you might have guessed, when we do this the Year field in our input struct is left with its zero value (which happens to be 0 because the Year field is an int32 type).

This leads to an interesting question: how can you tell the difference between a client not providing a key-value pair, and providing a key-value pair but deliberately setting it to its zero value? Like this:

$ BODY='{"title":"Moana","year":0,"runtime":107, "genres":["animation","adventure"]}'
$ curl -d "$BODY" localhost:4000/v1/movies
{Title:Moana Year:0 Runtime:107 Genres:[animation adventure]}

The end result is the same, despite the different HTTP requests, and it’s not immediately obvious how to tell the difference between the two scenarios. We’ll circle back to this topic later in the book, but for now, it’s worth just being aware of this behavior.

Additional information

Supported destination types

It’s important to mention that certain JSON types can only be successfully decoded to certain Go types. For example, if you have the JSON string "foo" it can be decoded into a Go string, but trying to decode it into a Go int or bool will result in an error at runtime (as we’ll demonstrate in the next chapter).

The following table shows the supported target decode destinations for the different JSON types:

Using the json.Unmarshal function

As we mentioned at the start of this chapter, it’s also possible to use the json.Unmarshal() function to decode an HTTP request body.

For example, you could use it in a handler like this:

func (app *application) exampleHandler(w http.ResponseWriter, r *http.Request) {
    var input struct {
        Foo string `json:"foo"`
    }

    // Use io.ReadAll() to read the entire request body into a []byte slice.
    body, err := io.ReadAll(r.Body)
    if err != nil {
        app.serverErrorResponse(w, r, err)
        return
    }
    
    // Use the json.Unmarshal() function to decode the JSON in the []byte slice to the
    // input struct. Again, notice that we are using a *pointer* to the input
    // struct as the decode destination.
    err = json.Unmarshal(body, &input)
    if err != nil {
        app.errorResponse(w, r, http.StatusBadRequest, err.Error())
        return
    }

    fmt.Fprintf(w, "%+v\n", input)
}

...

Using this approach is fine — the code works, and it’s clear and simple. But it doesn’t offer any benefits over and above the json.Decoder approach that we’re already taking.

Not only is the code marginally more verbose, but it’s also less efficient. If we benchmark the relative performance for this particular use case, we can see that using json.Unmarshal() requires about 80% more memory (B/op) than json.Decoder, as well as being a tiny bit slower (ns/op).

$ go test -run=^$ -bench=. -benchmem -count=3 -benchtime=5s
goos: linux
goarch: amd64
BenchmarkUnmarshal-8      528088      9543 ns/op     2992 B/op     20 allocs/op
BenchmarkUnmarshal-8      554365     10469 ns/op     2992 B/op     20 allocs/op
BenchmarkUnmarshal-8      537139     10531 ns/op     2992 B/op     20 allocs/op
BenchmarkDecoder-8        811063      8644 ns/op     1664 B/op     21 allocs/op
BenchmarkDecoder-8        672088      8529 ns/op     1664 B/op     21 allocs/op
BenchmarkDecoder-8       1000000      7573 ns/op     1664 B/op     21 allocs/op

Additional JSON decoding nuances

There are a few JSON decoding nuances that are important or interesting to know about, but which don’t fit nicely into the main content of this book. I’ve included this appendix which explains and demonstrates them in detail.