Intercepting shutdown signals

Chapter 11.2.

Intercepting shutdown signals

Before we get into the nuts and bolts of how to intercept signals, let’s move the code related to our http.Server out of the main() function and into a separate file. This will give us a clean and clear starting point from which we can build up the graceful shutdown functionality.

If you’re following along, create a new cmd/api/server.go file:

$ touch cmd/api/server.go

And then add a new app.serve() method which initializes and starts our http.Server, like so:

File: cmd/api/server.go

package main

import (
    "fmt"
    "log/slog"
    "net/http"
    "time"
)

func (app *application) serve() error {
    // Declare an HTTP server using the same settings as in our main() function.
    srv := &http.Server{
        Addr:         fmt.Sprintf(":%d", app.config.port),
        Handler:      app.routes(),
        IdleTimeout:  time.Minute,
        ReadTimeout:  5 * time.Second,
        WriteTimeout: 10 * time.Second,
        ErrorLog:     slog.NewLogLogger(app.logger.Handler(), slog.LevelError),
    }

    // Likewise log a "starting server" message.
    app.logger.Info("starting server", "addr", srv.Addr, "env", app.config.env)

    // Start the server as normal, returning any error.
    return srv.ListenAndServe()
}

With that in place, we can simplify our main() function to use this new app.serve() method like so:

File: cmd/api/main.go

package main

import (
    "context"
    "database/sql"
    "flag"
    "log/slog"
    "os"
    "time"

    "greenlight.alexedwards.net/internal/data"

    _ "github.com/lib/pq"
)

...

func main() {
    var cfg config

    flag.IntVar(&cfg.port, "port", 4000, "API server port")
    flag.StringVar(&cfg.env, "env", "development", "Environment (development|staging|production)")

    flag.StringVar(&cfg.db.dsn, "db-dsn", os.Getenv("GREENLIGHT_DB_DSN"), "PostgreSQL DSN")

    flag.IntVar(&cfg.db.maxOpenConns, "db-max-open-conns", 25, "PostgreSQL max open connections")
    flag.IntVar(&cfg.db.maxIdleConns, "db-max-idle-conns", 25, "PostgreSQL max idle connections")
    flag.DurationVar(&cfg.db.maxIdleTime, "db-max-idle-time", 15*time.Minute, "PostgreSQL max connection idle time")

    flag.Float64Var(&cfg.limiter.rps, "limiter-rps", 2, "Rate limiter maximum requests per second")
    flag.IntVar(&cfg.limiter.burst, "limiter-burst", 4, "Rate limiter maximum burst")
    flag.BoolVar(&cfg.limiter.enabled, "limiter-enabled", true, "Enable rate limiter")

    flag.Parse()

    logger := slog.New(slog.NewTextHandler(os.Stdout, nil))

    db, err := openDB(cfg)
    if err != nil {
        logger.Error(err.Error())
        os.Exit(1)
    }
    defer db.Close()

    logger.Info("database connection pool established")

    app := &application{
        config: cfg,
        logger: logger,
        models: data.NewModels(db),
    }

    // Call app.serve() to start the server.
    err = app.serve()
    if err != nil {
        logger.Error(err.Error())
        os.Exit(1)
    }
}

...

Catching SIGINT and SIGTERM signals

Next, we want to update our application so that it ‘catches’ any SIGINT and SIGTERM signals. As we mentioned above, SIGKILL signals are not catchable (and will always cause the application to terminate immediately), and we’ll leave SIGQUIT with its default behavior (as it’s handy if you want to execute a non-graceful shutdown via a keyboard shortcut).

To catch the signals, we’ll need to spin up a background goroutine which runs for the lifetime of our application. In this background goroutine, we can use the signal.Notify() function to listen for specific signals and relay them to a channel for further processing.

I’ll demonstrate.

Open up the cmd/api/server.go file and update it like so:

File: cmd/api/server.go

package main

import (
    "fmt"
    "log/slog"
    "net/http"
    "os"        // New import
    "os/signal" // New import
    "syscall"   // New import
    "time"
)

func (app *application) serve() error {
    srv := &http.Server{
        Addr:         fmt.Sprintf(":%d", app.config.port),
        Handler:      app.routes(),
        IdleTimeout:  time.Minute,
        ReadTimeout:  5 * time.Second,
        WriteTimeout: 10 * time.Second,
        ErrorLog:     slog.NewLogLogger(app.logger.Handler(), slog.LevelError),
    }

    // Start a background goroutine.
    go func() {
        // Create a quit channel which carries os.Signal values.
        quit := make(chan os.Signal, 1)

        // Use signal.Notify() to listen for incoming SIGINT and SIGTERM signals and 
        // relay them to the quit channel. Any other signals will not be caught by
        // signal.Notify() and will retain their default behavior.
        signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)

        // Read the signal from the quit channel. This code will block until a signal is
        // received.
        s := <-quit

        // Log a message to say that the signal has been caught. Notice that we also
        // call the String() method on the signal to get the signal name and include it
        // in the log entry attributes.
        app.logger.Info("caught signal", "signal", s.String())

        // Exit the application with a 0 (success) status code.
        os.Exit(0)
    }()

    // Start the server as normal.
    app.logger.Info("starting server", "addr", srv.Addr, "env", app.config.env)

    return srv.ListenAndServe()
}

At the moment this new code isn’t doing much — after intercepting the signal, all we do is log a message and then exit our application. But the important thing is that it demonstrates the pattern of how to catch specific signals and handle them in your code.

One thing I’d like to quickly emphasize about this: our quit channel is a buffered channel with size 1.

We need to use a buffered channel here because signal.Notify() does not wait for a receiver to be available when sending a signal to the quit channel. If we had used a regular (non-buffered) channel here instead, a signal could be ‘missed’ if our quit channel is not ready to receive at the exact moment that the signal is sent. By using a buffered channel, we avoid this problem and ensure that we never miss a signal.

OK, let’s try this out.

First, run the application and then press Ctrl+C on your keyboard to send a SIGINT signal. You should see a "caught signal" log entry with "signal":"interrupt" in the attributes, similar to this:

$ go run ./cmd/api
time=2023-09-10T10:59:13.722+02:00 level=INFO msg="database connection pool established"
time=2023-09-10T10:59:13.722+02:00 level=INFO msg="starting server" addr=:4000 env=development
time=2023-09-10T10:59:14.345+02:00 level=INFO msg="caught signal" signal=interrupt

You can also restart the application and try sending a SIGTERM signal. This time, the log entry attributes should contain signal=terminated, like so:

$ pkill -SIGTERM api

$ go run ./cmd/api
time=2023-09-10T10:59:13.722+02:00 level=INFO msg="database connection pool established"
time=2023-09-10T10:59:13.722+02:00 level=INFO msg="starting server" addr=:4000 env=development
time=2023-09-10T10:59:14.345+02:00 level=INFO msg="caught signal" signal=terminated

In contrast, sending a SIGKILL or SIGQUIT signal will continue to cause the application to exit immediately without the signal being caught, so you shouldn’t see a "caught signal" message in the logs. For example, if you restart the application and issue a SIGKILL…

$ pkill -SIGKILL api

The application should be terminated immediately, and the logs will look like this:

$ go run ./cmd/api
time=2023-09-10T10:59:13.722+02:00 level=INFO msg="database connection pool established"
time=2023-09-10T10:59:13.722+02:00 level=INFO msg="starting server" addr=:4000 env=development
signal: killed