Sending background emails

As we mentioned briefly in the last chapter, sending the welcome email from the registerUserHandler method adds quite a lot of latency to the total request/response round-trip for the client.

One way we could reduce this latency is by sending the email in a background goroutine. This would effectively ‘decouple’ the task of sending an email from the rest of the code in our registerUserHandler, and means that we could return an HTTP response to the client without waiting for the email sending to complete.

At its very simplest, we could adapt our handler to execute the email send in a background goroutine like this:

package main
...

func (app *application) registerUserHandler(w http.ResponseWriter, r *http.Request) {

    ...

    // Launch a goroutine which runs an anonymous function that sends the welcome email.
    go func() {
        err := app.mailer.Send(user.Email, "user_welcome.tmpl", user)
        if err != nil {
            // Importantly, if there is an error sending the email then we use the 
            // app.logger.Error() helper to manage it, instead of the 
            // app.serverErrorResponse() helper like before.
            app.logger.Error(err.Error())
        }
    }()

    // Note that we also change this to send the client a 202 Accepted status code.
    // This status code indicates that the request has been accepted for processing, but 
    // the processing has not yet been completed.
    err = app.writeJSON(w, http.StatusAccepted, envelope{"user": user}, nil)
    if err != nil {
        app.serverErrorResponse(w, r, err)
    }
}

When this code is executed now, a new ‘background’ goroutine will be launched for sending the welcome email. The code in this background goroutine will be executed concurrently with the subsequent code in our registerUserHandler, which means we are no longer waiting for the email to be sent before we return a JSON response to the client. Most likely, the background goroutine will still be executing its code long after the registerUserHandler has returned.

There are a couple of things I’d like to emphasize here:

• We use the app.logger.Error() method to manage any errors in our background goroutine. This is because by the time we encounter the errors, the client will probably have already been sent a 202 Accepted response by our writeJSON() helper.

  Note that we don’t want to use the app.serverErrorResponse() helper to handle any errors in our background goroutine, as that would result in us trying to write a second HTTP response and getting a "http: superfluous response.WriteHeader call" error from our http.Server at runtime.

• The code running in the background goroutine forms a closure over the user and app variables. It’s important to be aware that these ‘closed over’ variables are not scoped to the background goroutine, which means that any changes you make to them will be reflected in the rest of your codebase. For a simple example of this, see the following playground code.

  In our case we aren’t changing the value of these variables in any way, so this behavior won’t cause us any issues. But it is important to keep in mind.

OK, let’s try this out!

Restart the API, then go ahead and register another new user with the email address carol@example.com. Like so:

$ BODY='{"name": "Carol Smith", "email": "carol@example.com", "password": "pa55word"}'
$ curl -w '\nTime: %{time_total}\n' -d "$BODY" localhost:4000/v1/users
{
    "user": {
        "id": 4,
        "created_at": "2021-04-11T21:21:12+02:00",
        "name": "Carol Smith",
        "email": "carol@example.com",
        "activated": false
    }
}

Time: 0.268639

This time, you should see that the time taken to return the response is much faster — in my case 0.27 seconds compared to the previous 2.33 seconds.

And if you take a look at your Mailtrap inbox, you should see that the email for carol@example.com has been delivered correctly. Like so:

Recovering panics

It’s important to bear in mind that any panic which happens in this background goroutine will not be automatically recovered by our recoverPanic() middleware or Go’s http.Server, and will cause our whole application to terminate.

In very simple background goroutines this is less of a worry. But the code involved in sending an email is quite complex (including calls to a third-party package) and the risk of a runtime panic is non-negligible. So we need to make sure that any panic in this background goroutine is manually recovered, using a similar pattern to the one in our recoverPanic() middleware.

I’ll demonstrate.

Reopen your cmd/api/users.go file and update the registerUserHandler like so:

package main

import (
    "errors"
    "fmt" // New import
    "net/http"

    "greenlight.alexedwards.net/internal/data"
    "greenlight.alexedwards.net/internal/validator"
)

func (app *application) registerUserHandler(w http.ResponseWriter, r *http.Request) {

    ...

    // Launch a background goroutine to send the welcome email.
    go func() {
        // Run a deferred function which uses recover() to catch any panic, and log an
        // error message instead of terminating the application.
        defer func() {
            pv := recover()
            if pv != nil {
                app.logger.Error(fmt.Sprintf("%v", pv))
            }
        }()

        // Send the welcome email.
        err := app.mailer.Send(user.Email, "user_welcome.tmpl", user)
        if err != nil {
            app.logger.Error(err.Error())
        }
    }()

    err = app.writeJSON(w, http.StatusAccepted, envelope{"user": user}, nil)
    if err != nil {
        app.serverErrorResponse(w, r, err)
    }
}

Using a helper function

If you need to execute a lot of background tasks in your application, it can get tedious to keep repeating the same panic recovery code — and there’s a risk that you might forget to include it altogether.

To help take care of this, it’s possible to create a simple helper function which wraps the panic recovery logic. If you’re following along, open your cmd/api/helpers.go file and create a new background() helper method as follows:

package main

...

// The background() helper accepts an arbitrary function as a parameter.
func (app *application) background(fn func()) {
    // Launch a background goroutine.
    go func() {
        // Recover any panic.
        defer func() {
            pv := recover()
            if pv != nil {
                app.logger.Error(fmt.Sprintf("%v", pv))
            }
        }()

        // Execute the arbitrary function that we passed as the parameter.
        fn()
    }()
}

This background() helper leverages the fact that Go has first-class functions, which means that functions can be assigned to variables and passed as parameters to other functions.

In this case, we’ve set up the background() helper so that it accepts any function with the signature func() as a parameter and stores it in the variable fn. It then spins up a background goroutine, uses a deferred function to recover any panics and log the error, and then executes the function itself by calling fn().

Now that this is in place, let’s update our registerUserHandler to use it like so:

package main

import (
    "errors"
    "net/http"

    "greenlight.alexedwards.net/internal/data"
    "greenlight.alexedwards.net/internal/validator"
)

func (app *application) registerUserHandler(w http.ResponseWriter, r *http.Request) {

    ...
  
    // Use the background helper to execute an anonymous function that sends the welcome
    // email.
    app.background(func() {
        err := app.mailer.Send(user.Email, "user_welcome.tmpl", user)
        if err != nil {
            app.logger.Error(err.Error())
        }
    })

    err = app.writeJSON(w, http.StatusAccepted, envelope{"user": user}, nil)
    if err != nil {
        app.serverErrorResponse(w, r, err)
    }
}

Let’s double-check that this is still working. Restart the API, then create another new user with the email address dave@example.com:

$ BODY='{"name": "Dave Smith", "email": "dave@example.com", "password": "pa55word"}'
$ curl -w '\nTime: %{time_total}\n' -d "$BODY" localhost:4000/v1/users
{
    "user": {
       "id": 5,
       "created_at": "2021-04-11T21:33:07+02:00",
       "name": "Dave Smith",
       "email": "dave@example.com",
       "activated": false
    }
}

Time: 0.267692

If everything is set up correctly, you’ll now see the corresponding email appear in your Mailtrap inbox again.