Optimistic concurrency control

The eagle-eyed among you might have noticed a small problem in our updateMovieHandler — there is a race condition if two clients try to update the same movie record at exactly the same time.

To illustrate this, let’s pretend that we have two clients using our API: Alice and Bob. Alice wants to correct the runtime value for The Breakfast Club to 97 mins, and Bob wants to add the genre ‘comedy’ to the same movie.

Now imagine that Alice and Bob send these two update requests at exactly the same time. As we explained in Let’s Go, Go’s http.Server handles each HTTP request in a separate goroutine, so when this happens the code in our updateMovieHandler will be running concurrently in two different goroutines.

Let’s step through what could potentially happen in this scenario:

1. Alice’s goroutine calls app.models.Movies.Get() to retrieve a copy of the movie record (which has version number N).
2. Bob’s goroutine calls app.models.Movies.Get() to retrieve a copy of the movie record (which also has version N).
3. Alice’s goroutine changes the runtime to 97 minutes in its copy of the movie record.
4. Bob’s goroutine updates the genres to include ‘comedy’ in its copy of the movie record.
5. Alice’s goroutine calls app.models.Movies.Update() with its copy of the movie record. The movie record is written to the database and the version number is incremented to N+1.
6. Bob’s goroutine calls app.models.Movies.Update() with its copy of the movie record. The movie record is written to the database and the version number is incremented to N+2.

Despite making two separate updates, only Bob’s update will be reflected in the database at the end because the two goroutines were racing each other to make the change. Alice’s update to the movie runtime will be lost when Bob’s update overwrites it with the old runtime value retrieved during step 2 in the list above. And this happens silently — there’s nothing to inform either Alice or Bob of the problem.

Preventing the data race

Now we understand that the data race exists and why it’s happening, how can we prevent it?

There are a couple of options, but the simplest and cleanest approach in this case is to use a form of optimistic locking based on the version number in our movie record.

The fix works like this:

1. Alice and Bob’s goroutines both call app.models.Movies.Get() to retrieve a copy of the movie record. Both of these copies have the version number N.
2. Alice and Bob’s goroutines make their respective changes to the movie record.
3. Alice and Bob’s goroutines call app.models.Movies.Update() with their copies of the movie record. But the update is only executed if the version number in the database is still N. If it has changed, then we don’t execute the update and send the client an error message instead.

This means that the first update request that reaches our database will succeed, and whoever is making the second update will receive an error message instead of having their change applied.

To make this work, we’ll need to change the SQL statement for updating a movie so that it looks like this:

UPDATE movies 
SET title = $1, year = $2, runtime = $3, genres = $4, version = version + 1
WHERE id = $5 AND version = $6
RETURNING version

Notice that in the WHERE clause we’re now looking for a record with a specific ID and a specific version number?

If no matching record can be found, this query will result in a sql.ErrNoRows error and we know that the version number has been changed (or the record has been deleted completely). Either way, it’s a form of edit conflict and we can use this as a trigger to send the client an appropriate error response.

Implementing optimistic locking

OK, that’s enough theory… let’s put this into practice!

We’ll start by creating a custom ErrEditConflict error that we can return from our database models in the event of a conflict. We’ll use this later in the book when working with user records too, so it makes sense to define it in the internal/data/models.go file like so:

package data

import (
    "database/sql"
    "errors"
)

var (
    ErrRecordNotFound = errors.New("record not found")
    ErrEditConflict   = errors.New("edit conflict")
)

...

Next, let’s update our database model’s Update() method to execute the new SQL query and manage the situation where a matching record couldn’t be found.

package data

...

func (m MovieModel) Update(movie *Movie) error {
    // Add the 'AND version = $6' clause to the SQL query.
    query := `
        UPDATE movies 
        SET title = $1, year = $2, runtime = $3, genres = $4, version = version + 1
        WHERE id = $5 AND version = $6
        RETURNING version`

    args := []any{
        movie.Title,
        movie.Year,
        movie.Runtime,
        pq.Array(movie.Genres),
        movie.ID,
        movie.Version, // Add the expected movie version.
    }

    // Execute the SQL query. If no matching row could be found, we know the movie 
    // version has changed (or the record has been deleted) and we return our custom
    // ErrEditConflict error.
    err := m.DB.QueryRow(query, args...).Scan(&movie.Version)
    if err != nil {
        switch {
        case errors.Is(err, sql.ErrNoRows):
            return ErrEditConflict
        default:
            return err
        }
    }

    return nil
}

...

Next let’s head to our cmd/api/errors.go file and create a new editConflictResponse() helper. We want this to send a 409 Conflict response, along with a plain-English error message that explains the problem to the client.

package main

...

func (app *application) editConflictResponse(w http.ResponseWriter, r *http.Request) {
    message := "unable to update the record due to an edit conflict, please try again"
    app.errorResponse(w, r, http.StatusConflict, message)
}

And then as the final step, we need to change our updateMovieHandler so that it checks for an ErrEditConflict error and calls the editConflictResponse() helper if necessary. Like so:

package main

...

func (app *application) updateMovieHandler(w http.ResponseWriter, r *http.Request) {
    
    ...

    // Intercept any ErrEditConflict error and call the new editConflictResponse()
    // helper.
    err = app.models.Movies.Update(movie)
    if err != nil {
        switch {
        case errors.Is(err, data.ErrEditConflict):
            app.editConflictResponse(w, r)
        default:
            app.serverErrorResponse(w, r, err)
        }
        return
    }

    err = app.writeJSON(w, http.StatusOK, envelope{"movie": movie}, nil)
    if err != nil {
        app.serverErrorResponse(w, r, err)
    }
}

At this point, our updateMovieHandler should now be safe from the race condition that we’ve been talking about. If two goroutines are executing the code at the same time, the first update will succeed, and the second will fail because the version number in the database no longer matches the expected value.

Let’s try this out by using the xargs command to send a bunch of concurrent requests to our endpoint. Assuming that your computer executes the requests closely enough together, you should find that some requests succeed but the others now fail with a 409 Conflict status code. Like so:

$ xargs -I % -P8 curl -X PATCH -d '{"runtime": "97 mins"}' "localhost:4000/v1/movies/4" < <(printf '%s\n' {1..8})
 {
    "movie": {
        "id": 4,
        "title": "Breakfast Club",
        "year": 1985,
        "runtime": "97 mins",
        "genres": [
            "drama"
        ],
        "version": 4
    }
}
{
    "error": "unable to update the record due to an edit conflict, please try again"
}
{
    "error": "unable to update the record due to an edit conflict, please try again"
}
{
    "error": "unable to update the record due to an edit conflict, please try again"
}
{
    "error": "unable to update the record due to an edit conflict, please try again"
}
{
    "error": "unable to update the record due to an edit conflict, please try again"
}
{
    "error": "unable to update the record due to an edit conflict, please try again"
}
{
    "movie": {
        "id": 4,
        "title": "Breakfast Club",
        "year": 1985,
        "runtime": "97 mins",
        "genres": [
            "drama"
        ],
        "version": 5
    }
}

Bringing this to a close, the race condition that we’ve been demonstrating in this chapter is fairly innocuous. But in other applications this exact class of race condition can have much more serious consequences — such as when updating the stock level for a product in an online store, or updating the balance of an account.

As I mentioned briefly in Let’s Go, it’s good to get into the habit of thinking about race conditions whenever you write code, and structure your applications to either manage them or avoid them completely — no matter how innocuous they might seem.

Additional information

Round-trip locking

One of the nice things about the optimistic locking pattern that we’ve used here is that you can extend it so the client passes the version number that they expect in an If-Match or X-Expected-Version header.

In certain applications, this can be useful to help the client ensure they are not sending their update request based on outdated information.

Very roughly, you could implement this by adding a check to your updateMovieHandler like so:

func (app *application) updateMovieHandler(w http.ResponseWriter, r *http.Request) {
    id, err := app.readIDParam(r)
    if err != nil {
        app.notFoundResponse(w, r)
        return
    }

    movie, err := app.models.Movies.Get(id)
    if err != nil {
        switch {
        case errors.Is(err, data.ErrRecordNotFound):
            app.notFoundResponse(w, r)
        default:
            app.serverErrorResponse(w, r, err)
        }
        return
    }

    // If the request contains a X-Expected-Version header, verify that the movie 
    // version in the database matches the expected version specified in the header.
    if r.Header.Get("X-Expected-Version") != "" {
        if strconv.Itoa(int(movie.Version)) != r.Header.Get("X-Expected-Version") {
            app.editConflictResponse(w, r)
            return
        }
    }

    ...
}

Locking on other fields or types

Using an incrementing integer version number as the basis for an optimistic lock is safe and computationally cheap. I’d recommend using this approach unless you have a specific reason not to.

As an alternative, you could use a last_updated timestamp as the basis for the lock. But this is less safe — there’s the theoretical possibility that two clients could update a record at exactly the same time, and using a timestamp also introduces the risk of further problems if your server’s clock is wrong or becomes wrong over time.

If it’s important to you that the version identifier isn’t guessable, then a good option is to use a high-entropy random string such as a UUID in the version field. PostgreSQL has a UUID type and the uuid-ossp extension which you could use for this purpose like so:

UPDATE movies 
SET title = $1, year = $2, runtime = $3, genres = $4, version = uuid_generate_v4()
WHERE id = $5 AND version = $6
RETURNING version