Parsing query string parameters

Chapter 9.1.

Parsing query string parameters

Over the next few chapters, we’re going to configure the GET /v1/movies endpoint so that a client can control which movie records are returned via query string parameters. For example:

/v1/movies?title=godfather&genres=crime,drama&page=1&page_size=5&sort=-year

If a client sends a query string like this, it is essentially saying to our API: “please return the first 5 records where the movie title includes godfather and the genres include crime and drama, sorted by descending release year”.

So the first thing we’re going to look at is how to parse these query string parameters in our Go code.

As you can hopefully remember from Let’s Go, we can retrieve the query string data from a request by calling the r.URL.Query() method. This returns a url.Values type, which is basically a map holding the query string data.

We can then extract values from this map using the Get() method, which will return the value for a specific key as a string type, or the empty string "" if no matching key exists in the query string.

In our case, we’ll need to carry out extra post-processing on some of these query string values too. Specifically:

The genres parameter will potentially contain multiple comma-separated values — like genres=crime,drama. We will want to split these values apart and store them in a []string slice.
The page and page_size parameters will contain numbers, and we will want to convert these query string values into Go int types.

In addition to that:

There are some validation checks that we’ll want to apply to the query string values, like making sure that page and page_size are not negative numbers.
We want our application to set some sensible default values in case parameters like page, page_size and sort aren’t provided by the client.

Creating helper functions

To assist with this, we’re going to create three new helper functions: readString(), readInt() and readCSV(). We’ll use these helpers to extract and parse values from the query string, or return a default ‘fallback’ value if necessary.

Head to your cmd/api/helpers.go file and add the following code:

File: cmd/api/helpers.go

package main

import (
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "net/url" // New import
    "strconv"
    "strings"

    "greenlight.alexedwards.net/internal/validator" // New import

    "github.com/julienschmidt/httprouter"
)

...

// The readString() helper returns a string value from the query string, or the provided
// default value if no matching key could be found.
func (app *application) readString(qs url.Values, key string, defaultValue string) string {
    // Extract the value for a given key from the query string. If no key exists this
    // will return the empty string "". 
    s := qs.Get(key)

    // If no key exists (or the value is empty) then return the default value.
    if s == "" {
        return defaultValue
    }

    // Otherwise return the string.
    return s
}

// The readCSV() helper reads a string value from the query string and then splits it 
// into a slice on the comma character. If no matching key could be found, it returns
// the provided default value.
func (app *application) readCSV(qs url.Values, key string, defaultValue []string) []string {
    // Extract the value from the query string.
    csv := qs.Get(key)

    // If no key exists (or the value is empty) then return the default value.
    if csv == "" {
        return defaultValue
    }

    // Otherwise parse the value into a []string slice and return it.
    return strings.Split(csv, ",")
}


// The readInt() helper reads a string value from the query string and converts it to an 
// integer before returning. If no matching key could be found it returns the provided 
// default value. If the value couldn't be converted to an integer, then we record an 
// error message in the provided Validator instance. 
func (app *application) readInt(qs url.Values, key string, defaultValue int, v *validator.Validator) int {
    // Extract the value from the query string.
    s := qs.Get(key)

    // If no key exists (or the value is empty) then return the default value.
    if s == "" {
        return defaultValue
    }

    // Try to convert the value to an int. If this fails, add an error message to the 
    // validator instance and return the default value.
    i, err := strconv.Atoi(s)
    if err != nil {
        v.AddError(key, "must be an integer value")
        return defaultValue
    }

    // Otherwise, return the converted integer value.
    return i
}

Adding the API handler and route

Next up, let’s create a new listMoviesHandler for our GET /v1/movies endpoint. For now, this handler will simply parse the request query string using the helpers we just made, and then dump the contents out in an HTTP response.

If you’re following along, go ahead and create the listMoviesHandler like so:

File: cmd/api/movies.go

package main

...

func (app *application) listMoviesHandler(w http.ResponseWriter, r *http.Request) {
    // To keep things consistent with our other handlers, we'll define an input struct
    // to hold the expected values from the request query string.
    var input struct {
        Title    string
        Genres   []string
        Page     int
        PageSize int
        Sort     string
    }

    // Initialize a new Validator instance.
    v := validator.New()

    // Call r.URL.Query() to get the url.Values map containing the query string data.
    qs := r.URL.Query()

    // Use our helpers to extract the title and genres query string values, falling back
    // to defaults of an empty string and an empty slice respectively if they are not 
    // provided by the client.
    input.Title = app.readString(qs, "title", "")
    input.Genres = app.readCSV(qs, "genres", []string{})

    // Get the page and page_size query string values as integers. Notice that we set 
    // the default page value to 1 and default page_size to 20, and that we pass the 
    // validator instance as the final argument here. 
    input.Page = app.readInt(qs, "page", 1, v)
    input.PageSize = app.readInt(qs, "page_size", 20, v)

    // Extract the sort query string value, falling back to "id" if it is not provided
    // by the client (which will imply an ascending sort on movie ID).
    input.Sort = app.readString(qs, "sort", "id")

    // Check the Validator instance for any errors and use the failedValidationResponse()
    // helper to send the client a response if necessary. 
    if !v.Valid() {
        app.failedValidationResponse(w, r, v.Errors)
        return
    }

    // Dump the contents of the input struct in an HTTP response.
    fmt.Fprintf(w, "%+v\n", input)
}

Then we need to create the GET /v1/movies route in our cmd/api/routes.go file, like so:

File: cmd/api/routes.go

package main

...

func (app *application) routes() http.Handler {
    router := httprouter.New()

    router.NotFound = http.HandlerFunc(app.notFoundResponse)
    router.MethodNotAllowed = http.HandlerFunc(app.methodNotAllowedResponse)

    router.HandlerFunc(http.MethodGet, "/v1/healthcheck", app.healthcheckHandler)

    // Add the route for the GET /v1/movies endpoint.
    router.HandlerFunc(http.MethodGet, "/v1/movies", app.listMoviesHandler)
    router.HandlerFunc(http.MethodPost, "/v1/movies", app.createMovieHandler)
    router.HandlerFunc(http.MethodGet, "/v1/movies/:id", app.showMovieHandler)
    router.HandlerFunc(http.MethodPatch, "/v1/movies/:id", app.updateMovieHandler)
    router.HandlerFunc(http.MethodDelete, "/v1/movies/:id", app.deleteMovieHandler)

    return app.recoverPanic(router)
}

And with that, we’re ready to see this in action!

Go ahead and try sending a request to the GET /v1/movies endpoint containing the expected query string parameters, like below.

$ curl "localhost:4000/v1/movies?title=godfather&genres=crime,drama&page=1&page_size=5&sort=year"
{Title:godfather Genres:[crime drama] Page:1 PageSize:5 Sort:year}

That’s looking good — we can see that the values provided in our query string have all been parsed correctly and are included in the input struct.

If you want, try making a request with no query string parameters. In this case, you should see that the values in the input struct take on the defaults we specified in our listMoviesHandler code. Like so:

$ curl localhost:4000/v1/movies
{Title: Genres:[] Page:1 PageSize:20 Sort:id}

Creating a Filters struct

The page, page_size and sort query string parameters are things that you’ll potentially want to use on other endpoints in your API too. So, to help make this easier, let’s quickly split them out into a reusable Filters struct.

If you’re following along, go ahead and create a new internal/data/filters.go file:

$ touch internal/data/filters.go

And then add the following code:

File: internal/data/filters.go

package data

type Filters struct {
    Page     int
    PageSize int
    Sort     string
}

Once that’s done, head back to your listMoviesHandler and update it to use the new Filters struct like so:

File: cmd/api/movies.go

package main

...

func (app *application) listMoviesHandler(w http.ResponseWriter, r *http.Request) {
    // Embed the new Filters struct.
    var input struct {
        Title  string
        Genres []string
        data.Filters
    }

    v := validator.New()

    qs := r.URL.Query()

    input.Title = app.readString(qs, "title", "")
    input.Genres = app.readCSV(qs, "genres", []string{})

    // Read the page and page_size query string values into the embedded struct.
    input.Filters.Page = app.readInt(qs, "page", 1, v)
    input.Filters.PageSize = app.readInt(qs, "page_size", 20, v)
    
    // Read the sort query string value into the embedded struct.
    input.Filters.Sort = app.readString(qs, "sort", "id")

    if !v.Valid() {
        app.failedValidationResponse(w, r, v.Errors)
        return
    }

    fmt.Fprintf(w, "%+v\n", input)
}

At this point, you should be able to run the API again and everything should continue to work like before.