Linux path · Processes · Topic 3 of 12

Signals and graceful shutdown

Not started

A signal is a small asynchronous message the kernel delivers to a process: a number with a meaning, such as SIGINT (Ctrl+C), SIGTERM (please exit), SIGKILL (exit now), SIGHUP (terminal closed, often reused as "reload config"), SIGCHLD (a child exited) or SIGPIPE (the reader of your pipe is gone). For most signals a process can install a handler, ignore it, or accept the default action, which is often to terminate. SIGKILL and SIGSTOP are the exceptions: they cannot be caught, blocked or ignored.

Why it matters: Every deploy, autoscale and Ctrl+C stops processes with signals. Handling SIGTERM is the difference between finishing in-flight requests and dropping them.

  1. 1A signal is a small notification

    The kernel interrupts a process to tell it something: stop, hang up, a child exited, you wrote to a closed pipe.

  2. 2SIGTERM can be handled

    It is the polite request to exit. A program can catch it, finish its work, close connections and exit 0.

  3. 3SIGKILL and SIGSTOP cannot

    They can't be caught, blocked or ignored. SIGKILL ends the process at once, with no cleanup.

  4. 4Managers escalate

    systemd, Docker and Kubernetes send SIGTERM, wait a grace period, then send SIGKILL.

1

How it works

The explanation, with examples you can run.

TERM versus KILL

sleep 100 & kill -TERM $!; wait $!; echo "exit=$?"sleep 100 & kill -KILL $!; wait $!; echo "exit=$?"
exit=143bash: line 3:   197 Killed                  sleep 100exit=137

(Both blocks were fed to bash as one script, which is where the bash: line N messages come from; an interactive shell prints job notices such as [1]+ Killed instead, and PIDs will differ.) sleep has no handler, so both signals kill it, and the shell reports 128 plus the signal number: 143 for SIGTERM (15), 137 for SIGKILL (9). kill -l lists all signals and their numbers. kill with no option sends SIGTERM.

Handle SIGTERM

A script that cleans up when asked:

cat > graceful.sh <<'EOF'#!/bin/bashtrap 'echo "got SIGTERM, finishing up"; exit 0' TERMecho "working (pid $$)"while true; do sleep 1; doneEOFchmod +x graceful.sh./graceful.sh & sleep 1.5; kill $!; wait $!; echo "exit=$?"./graceful.sh & sleep 1.5; kill -9 $!; wait $!; echo "exit=$?"
working (pid 200)got SIGTERM, finishing upexit=0working (pid 204)bash: line 14:   204 Killed                  ./graceful.shexit=137

With SIGTERM the handler runs and the script exits 0. With SIGKILL it never gets the chance. (Bash runs a trap only after the current foreground command returns, so this one reacts within a second, when sleep 1 ends.)

What graceful shutdown means for a service

On SIGTERM a server should:

  1. Stop accepting new work (close the listening socket, fail readiness checks).
  2. Finish or hand back in-flight requests and jobs.
  3. Flush buffers, commit offsets, close database connections.
  4. Exit with status 0 before the grace period runs out.

Process managers all escalate the same way. systemd sends KillSignal= (SIGTERM by default) to every process in the unit, waits TimeoutStopSec= (90 seconds by default) and then sends SIGKILL. docker stop waits 10 seconds by default; Kubernetes gives a pod 30 seconds (terminationGracePeriodSeconds). If shutdown regularly takes longer, raise the grace period rather than letting work be killed halfway.

Common traps

  • PID 1 ignores signals it has no handler for. The kernel does not apply default actions to a namespace's init process, so an app running as PID 1 in a container that doesn't handle SIGTERM ignores it until SIGKILL arrives. Handle the signal, or run under a small init.
  • Shell wrappers swallow signals. If an entrypoint script starts your server as a child, the signal goes to the shell, not the server. End the script with exec your-server so the server replaces the shell.
  • SIGPIPE. Writing to a closed pipe or socket raises SIGPIPE, which kills the process by default. Network servers usually ignore it and handle the EPIPE error instead.

On macOS

Signals are POSIX and behave the same; kill -l prints names without numbers in zsh. macOS has no systemd; launchd stops jobs with SIGTERM and sends SIGKILL after the job's ExitTimeOut (the default is system-defined).

2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.Why should a service handle SIGTERM rather than rely on being killed?
  2. 2.A process exits with status 137. What happened?
  3. 3.Which signals can a program never catch?

0 of 3 right so far. Get all three to finish this topic.