Managing bad requests

Our createMovieHandler now works well when it receives a valid JSON request body with the appropriate data. But at this point you might be wondering:

• What if the client sends something that isn’t JSON, like XML or some random bytes?
• What happens if the JSON is malformed or contains an error?
• What if the JSON types don’t match the types we are trying to decode into?
• What if the request doesn’t even contain a body?

Well… let’s look and see!

# Send some XML as the request body
$ curl -d '<?xml version="1.0" encoding="UTF-8"?><note><to>Alice</to></note>' localhost:4000/v1/movies
{
    "error": "invalid character '\u003c' looking for beginning of value"
}

# Send some malformed JSON (notice the trailing comma)
$ curl -d '{"title": "Moana", }' localhost:4000/v1/movies
{
    "error": "invalid character '}' looking for beginning of object key string"
}

# Send a JSON array instead of an object
$ curl -d '["foo", "bar"]' localhost:4000/v1/movies
{
    "error": "json: cannot unmarshal array into Go value of type struct { Title string 
    \"json:\\\"title\\\"\"; Year int32 \"json:\\\"year\\\"\"; Runtime int32 \"json:\\
    \"runtime\\\"\"; Genres []string \"json:\\\"genres\\\"\" }"
}

# Send a numeric 'title' value (instead of string)
$ curl -d '{"title": 123}' localhost:4000/v1/movies
{
    "error": "json: cannot unmarshal number into Go struct field .title of type string"
}

# Send an empty request body
$ curl -X POST localhost:4000/v1/movies
{
    "error": "EOF"
}

In all these cases, we can see that our createMovieHandler is doing the right thing. When it receives an invalid request that can’t be decoded into our input struct, no further processing takes place and the client is sent a JSON response containing the error message returned by the Decode() method.

For a private API which won’t be used by members of the public, this behavior is probably fine and you needn’t do anything else.

But for a public-facing API, the error messages themselves aren’t ideal. Some are too detailed and expose information about the underlying API implementation. Others aren’t descriptive enough (like "EOF"), and some are just plain confusing and difficult to understand. There isn’t consistency in the formatting or language used either.

To improve this, we’re going to explain how to triage the errors returned by Decode() and replace them with clearer, easy-to-action, error messages to help the client debug exactly what is wrong with their JSON.

Triaging the Decode error

At this point in our application build, the Decode() method could potentially return the following five kinds of error:

Triaging these potential errors (which we can do using Go’s errors.Is() and errors.As() functions) is going to make the code in our createMovieHandler a lot longer and more complicated. And the logic is something that we’ll need to duplicate in other handlers throughout this project too.

So, to assist with this, let’s create a new readJSON() helper in the cmd/api/helpers.go file. In this helper we’ll decode the JSON from the request body as normal, then triage the errors and replace them with our own custom messages as necessary.

If you’re coding along, go ahead and add the following code to the cmd/api/helpers.go file:

package main

import (
    "encoding/json"
    "errors"
    "fmt" // New import
    "io"  // New import
    "net/http"
    "strconv"

    "github.com/julienschmidt/httprouter"
)

...

func (app *application) readJSON(w http.ResponseWriter, r *http.Request, dst any) error {
    // Decode the request body into the target destination. 
    err := json.NewDecoder(r.Body).Decode(dst)
    if err != nil {
        // If there is an error during decoding, start the triage...
        var syntaxError *json.SyntaxError
        var unmarshalTypeError *json.UnmarshalTypeError
        var invalidUnmarshalError *json.InvalidUnmarshalError

        switch {
        // Use the errors.As() function to check whether the error has the type 
        // *json.SyntaxError. If it does, then return a plain-english error message 
        // which includes the location of the problem.
        case errors.As(err, &syntaxError):
            return fmt.Errorf("body contains badly-formed JSON (at character %d)", syntaxError.Offset)

        // In some circumstances Decode() may also return an io.ErrUnexpectedEOF error
        // for syntax errors in the JSON. So we check for this using errors.Is() and
        // return a generic error message. There is an open issue regarding this at
        // https://github.com/golang/go/issues/25956.
        case errors.Is(err, io.ErrUnexpectedEOF):
            return errors.New("body contains badly-formed JSON")

        // Likewise, catch any json.UnmarshalTypeError errors. These occur when the
        // JSON value is the wrong type for the target destination. If the error relates
        // to a specific field, then we include that in our error message to make it 
        // easier for the client to debug.
        case errors.As(err, &unmarshalTypeError):
            if unmarshalTypeError.Field != "" {
                return fmt.Errorf("body contains incorrect JSON type for field %q", unmarshalTypeError.Field)
            }
            return fmt.Errorf("body contains incorrect JSON type (at character %d)", unmarshalTypeError.Offset)

        // An io.EOF error will be returned by Decode() if the request body is empty. We
        // check for this with errors.Is() and return a plain-english error message 
        // instead.
        case errors.Is(err, io.EOF):
            return errors.New("body must not be empty")

        // A json.InvalidUnmarshalError error will be returned if we pass something 
        // that is not a non-nil pointer as the target destination to Decode(). If this 
        // happens we panic, rather than returning an error to our handler. At the end of 
        // this chapter we'll briefly discuss why panicking is an appropriate thing to do 
        // in this specific situation.
        case errors.As(err, &invalidUnmarshalError):
            panic(err)

        // For any other error, return it as-is.
        default:
            return err
        }
    }

    return nil
}

With this new helper in place, let’s head back to the cmd/api/movies.go file and update our createMovieHandler to use it. Like so:

package main

import (
    "fmt"
    "net/http"
    "time"

    "greenlight.alexedwards.net/internal/data"
)

func (app *application) createMovieHandler(w http.ResponseWriter, r *http.Request) {
    var input struct {
        Title   string   `json:"title"`
        Year    int32    `json:"year"`
        Runtime int32    `json:"runtime"`
        Genres  []string `json:"genres"`
    }

    // Use the new readJSON() helper to decode the request body into the input struct. 
    // If this returns an error we send the client the error message along with a 400
    // Bad Request status code, just like before.
    err := app.readJSON(w, r, &input)
    if err != nil {
        app.errorResponse(w, r, http.StatusBadRequest, err.Error())
        return
    }

    fmt.Fprintf(w, "%+v\n", input)
}

...

Go ahead and restart the API, and then let’s try this out by repeating the same bad requests that we made at the start of the chapter. You should now see our new, customized, error messages similar to this:

# Send some XML as the request body
$ curl -d '<?xml version="1.0" encoding="UTF-8"?><note><to>Alex</to></note>' localhost:4000/v1/movies
{
    "error": "body contains badly-formed JSON (at character 1)"
}

# Send some malformed JSON (notice the trailing comma)
$ curl -d '{"title": "Moana", }' localhost:4000/v1/movies
{
    "error": "body contains badly-formed JSON (at character 20)"
}

# Send a JSON array instead of an object
$ curl -d '["foo", "bar"]' localhost:4000/v1/movies
{
    "error": "body contains incorrect JSON type (at character 1)"
}

# Send a numeric 'title' value (instead of string)
$ curl -d '{"title": 123}' localhost:4000/v1/movies
{
    "error": "body contains incorrect JSON type for \"title\""
}

# Send an empty request body
$ curl -X POST localhost:4000/v1/movies
{
    "error": "body must not be empty"
}

They’re looking really good. The error messages are now simpler, clearer, and consistent in their formatting, plus they don’t expose any unnecessary information about our underlying code.

Feel free to play around with this if you like, and try sending different request bodies to see how the handler reacts.

Making a bad request helper

In the createMovieHandler code above we’re using our generic app.errorResponse() helper to send the client a 400 Bad Request response along with the error message.

Let’s quickly replace this with a specialist app.badRequestResponse() helper function instead:

package main

...

func (app *application) badRequestResponse(w http.ResponseWriter, r *http.Request, err error) {
    app.errorResponse(w, r, http.StatusBadRequest, err.Error())
}

package main

...

func (app *application) createMovieHandler(w http.ResponseWriter, r *http.Request) {
    var input struct {
        Title   string   `json:"title"`
        Year    int32    `json:"year"`
        Runtime int32    `json:"runtime"`
        Genres  []string `json:"genres"`
    }

    err := app.readJSON(w, r, &input)
    if err != nil {
        // Use the new badRequestResponse() helper.
        app.badRequestResponse(w, r, err) 
        return
    }

    fmt.Fprintf(w, "%+v\n", input)
}

...

This is a small change, but a useful one. As our application gradually gets more complex, using specialist helpers like this to manage different kinds of errors will help ensure that our error responses remain consistent across all our endpoints.

Additional information

Panicking vs returning errors

The topic of panicking vs returning errors is something that we talked about in the first Let’s Go book, and I’ve also written a detailed tutorial about it here (which I would recommend checking out if you haven’t read Let’s Go).

So I don’t want to rehash that same information again, other than to say that the decision to panic in the readJSON() helper if we get a json.InvalidUnmarshalError error isn’t taken lightly. As you’re probably aware, it’s generally considered best practice in Go to return your errors and handle them gracefully, rather than panicking.

The only reason that we’re panicking here is because if we get a json.InvalidUnmarshalError at runtime, it’s firmly an unexpected programmer error. It would only happen if we as the developers pass an unsupported value as the target decode destination to Decode(). We shouldn’t see this error under normal operation, and it’s something that should be picked up in development and tests long before deployment.

And if we did return this error, rather than panicking, we would need to introduce additional code to manage it in each of our API handlers — which doesn’t seem like a good trade-off for an error that we’re unlikely to ever see in production.