JSON decoding nuances

Chapter 21.5.

Decoding into Go arrays

When you’re decoding a JSON array into a Go array (not a slice) there are a couple of important behaviors to be aware of:

If the Go array is smaller than the JSON array, then the additional JSON array elements are silently discarded.
If the Go array is larger than the JSON array, then the additional Go array elements are set to their zero values.

As an example:

js := `[1, 2, 3]`

var tooShortArray [2]int
err := json.NewDecoder(strings.NewReader(js)).Decode(&tooShortArray)
if err != nil {
    log.Fatal(err)
}

var tooLongArray [4]int
err = json.NewDecoder(strings.NewReader(js)).Decode(&tooLongArray)
if err != nil {
    log.Fatal(err)
}

fmt.Printf("tooShortArray: %v\n", tooShortArray)
fmt.Printf("tooLongArray: %v\n", tooLongArray)

Will print out:

tooShortArray: [1 2]
tooLongArray: [1 2 3 0]

Partial JSON decoding

If you have a lot of JSON input to process and only need a small part of it, it’s often possible to leverage the json.RawMessage type to help deal with this. For example:

// Let's say that the only thing we're interested in is processing the "genres" array in
// the following JSON object
js := `{"title": "Top Gun", "genres": ["action", "romance"], "year": 1986}`

// Decode the JSON object to a map[string]json.RawMessage type. The json.RawMessage 
// values in the map will retain their original, undecoded, JSON values.
var m map[string]json.RawMessage

err := json.NewDecoder(strings.NewReader(js)).Decode(&m)
if err != nil {
    log.Fatal(err)
}

// We can then access the JSON "genres" value from the map and decode it as normal using
// the json.Unmarshal() function.
var genres []string

err = json.Unmarshal(m["genres"], &genres)
if err != nil {
    log.Fatal(err)
}

fmt.Printf("genres: %v\n", genres)

This will print out:

genres: [action romance]

In this toy example, using json.RawMessage doesn’t save us much work. But if you need to process a JSON object with tens or hundreds of key-value pairs and only need a few of them, then taking this approach can save you a lot of typing.

Decoding into the any type

It’s possible to decode JSON values into an any type. When you do this, the underlying value that the any type holds will depend on the type of the JSON value being decoded.

JSON type	⇒	Underlying Go type of any
JSON boolean	⇒	`bool`
JSON string	⇒	`string`
JSON number	⇒	`float64`
JSON array	⇒	`[]any`
JSON object	⇒	`map[string]any`
JSON null	⇒	`nil`

Decoding into an any type can be useful in situations where:

You don’t know in advance exactly what you’re decoding.
You need to decode JSON arrays which contain items with different JSON types.
The key-value pair in a JSON object doesn’t always contain values with the same JSON type.

As an example, consider the following code:

// This JSON array contains both JSON string and JSON boolean types.
js := `["foo", true]`

// Decode the JSON into a []any slice.
var s []any

err := json.NewDecoder(strings.NewReader(js)).Decode(&s)
if err != nil {
    log.Fatal(err)
}

// The first value in the slice will have the underlying Go type string, the second will
// have the underlying Go type bool. We can then type assert them and print them out  
// the values along with their underlying type.
fmt.Printf("item: 0; type: %T; value: %v\n", s[0], s[0].(string))
fmt.Printf("item: 1; type: %T; value: %v\n", s[1], s[1].(bool))

This will print out:

item: 0; type: string; value: foo
item: 1; type: bool; value: true

Decoding a JSON number into an any type

As shown in the table above, when you decode a JSON number into an any type the value will have the underlying type float64 — even if it is an integer in the original JSON. For example:

js := `10` // This JSON number is an integer.

var n any

err := json.NewDecoder(strings.NewReader(js)).Decode(&n)
if err != nil {
    log.Fatal(err)
}

fmt.Printf("type: %T; value: %v\n", n, n)

Will print:

type: float64; value: 10

If you want to get the value as an integer (instead of a float64) you should call the UseNumber() method on your json.Decoder instance before decoding. This will cause all JSON numbers to be decoded to the underlying type json.Number instead of float64.

The json.Number type then provides an Int64() method that you can call to get the number as an int64, or the String() method to get the number as a string. For example:

js := `10`

var n any

dec := json.NewDecoder(strings.NewReader(js))
dec.UseNumber() // Before using the decoder, call the UseNumber() method on it.
err := dec.Decode(&n)
if err != nil {
    log.Fatal(err)
}

// Type assert the any value to a json.Number, and then call the Int64() method
// to get the number as a Go int64.
nInt64, err := n.(json.Number).Int64()
if err != nil {
    log.Fatal(err)
}

// Likewise, you can use the String() method to get the number as a Go string.
nString := n.(json.Number).String()

fmt.Printf("type: %T; value: %v\n", n, n)
fmt.Printf("type: %T; value: %v\n", nInt64, nInt64)
fmt.Printf("type: %T; value: %v\n", nString, nString)

This will print out:

type: json.Number; value: 10
type: int64; value: 10
type: string; value: 10

Struct tag directives

Using the struct tag json:"-" on a struct field will cause it to be ignored when decoding JSON, even if the JSON input contains a corresponding key-value pair. For example:

js := `{"name": "alice", "age": 21}`

var person struct {
    Name string `json:"name"`
    Age  int32 `json:"-"`
}

err := json.NewDecoder(strings.NewReader(js)).Decode(&person)
if err != nil {
    log.Fatal(err)
}

fmt.Printf("%+v", person)

Will print out:

{Name:alice Age:0}

The omitzero and omitempty struct tag directives do not have any effect on JSON decoding behavior.