How the OS runs programs: processes, threads, virtual memory

Processes

A process is a running program plus everything the kernel tracks for it: an address space holding its code, data, heap and stacks; a table of open files and sockets; its user and permissions; and one or more threads. Processes are isolated: a crash or a wild pointer in one can't corrupt another. They talk only through channels the kernel provides, such as pipes, sockets, files, signals and explicitly shared memory.

On Unix, a new process is created by fork (copy the current process) followed by exec (replace its program). Copy-on-write makes fork cheap: parent and child share pages until one of them writes.

Threads and scheduling

A thread is the unit the kernel schedules. Threads in the same process share its memory and files, and each has its own stack, registers and program counter. A machine with 8 cores can run 8 threads at the same instant; everything else waits in a run queue.

The scheduler decides who runs. When a thread blocks (waiting for disk, a socket or a lock), or uses up its time slice (a few milliseconds), the kernel performs a context switch: it saves the thread's registers, picks another thread, and restores its registers. Switching itself takes on the order of microseconds, but the incoming thread often finds the caches and the TLB filled with someone else's data. Thousands of busy threads therefore spend real time just being switched, which is one reason servers use thread pools and event loops instead of a thread per connection.

Virtual memory

Every address a program uses is virtual. The CPU's memory management unit (MMU) translates it through page tables, maintained by the kernel per process, into a physical address, one page (usually 4 KB) at a time. A small cache of recent translations, the TLB, keeps this fast.

Consequences you meet in practice:

• Isolation: the same virtual address in two processes maps to different physical pages.
• Lazy allocation: malloc of 1 GB only reserves address space. The first touch of each page causes a page fault, and the kernel then assigns a physical page. Resident set size (RSS) is the memory actually backed by RAM.
• Memory-mapped files: a file can be mapped into the address space, and pages are read from disk only when touched.
• Swapping and the OOM killer: when RAM runs out, the kernel can write rarely used pages to disk (much slower) or, on Linux, kill a process to free memory. In containers the limit is the cgroup's, not the machine's.

User mode and kernel mode

Programs run in user mode and can't touch hardware or other processes' memory. To read a file, send a packet or create a thread, they make a system call, a controlled switch into kernel mode. Each costs roughly hundreds of nanoseconds or more, which is why buffered I/O, batching and APIs such as io_uring exist.