Determining and configuring hardware settings

Three trees, three different jobs

/proc is the kernel talking about itself. It is a virtual filesystem — nothing in it is on a disk — holding one numbered directory per running process plus files describing the machine: /proc/cpuinfo, /proc/meminfo, /proc/interrupts, /proc/ioports, /proc/dma. Reading it is how tools like ps and free get their numbers.

/sys is newer, and it is the structured version of the same idea: one directory per device, arranged by the bus it hangs off, with the driver bound to it and the attributes it exposes. Where /proc grew organically and mixes everything together, sysfs is the tree udev and systemd actually walk.

/dev holds the device nodes — the special files a program opens to talk to a device. They are created at boot and on hotplug by udev, which watches kernel events, applies the rules in /etc/udev/rules.d and /lib/udev/rules.d, and can name a device predictably so that a USB disk gets the same path every time it is plugged in.

Modules are how the kernel gets a driver

Most drivers are not compiled into the kernel; they are loadable modules under /lib/modules/$(uname -r)/. lsmod lists what is loaded — it is a formatter for /proc/modules and takes no module argument — modprobe loads a module by name and pulls in whatever it depends on, and modprobe -r or rmmod unloads it. insmod loads a single file and resolves no dependencies at all, which is why it is almost never the right command.

Two more commands complete the set, and they are the two candidates most often meet for the first time in the exam room. modinfo reads the metadata compiled into the .ko file itself — description, author, licence, the modules it depends on, and one parm: line per parameter the driver accepts — so it works on a module that is not loaded and is how you find out which options a driver takes before setting them. depmod scans everything under /lib/modules/$(uname -r) and rebuilds modules.dep, the dependency map modprobe consults; you run it after installing a module by hand. The four divide cleanly: modinfo describes, lsmod lists, modprobe loads, depmod indexes.

Configuration for modules lives in /etc/modprobe.d/ — options passed to a driver, aliases, and blacklist lines that stop a module being loaded automatically. That last one is the usual fix when two drivers both claim a device, and usb_storage is the module most often blacklisted that way.

$ modinfo usb_storagefilename:     /lib/modules/6.1.0-18/kernel/.../usb-storage.kolicense:      GPLdescription:  USB Mass Storage driver for Linuxdepends:      usb-common,usbcoreparm:         delay_use:seconds to delay (uint)# Nothing was loaded — that came out of the file$ modprobe usb_storage$ lsmod | grep usb_storageusb_storage     81920  0usbcore        348160  2 usb_storage,xhci_hcd

Reading and setting kernel tunables

A running kernel has several hundred settings that can be changed without rebooting, and they are exposed as files under /proc/sys. Reading one is cat; changing one is writing to it. sysctl is the front end for the same tree with dots in place of slashes, so /proc/sys/net/ipv4/ip_forward is net.ipv4.ip_forward to sysctl, and sysctl -a lists everything.

A write to /proc/sys lasts until the next reboot, because nothing in /proc is on a disk. To make a setting survive, put it in /etc/sysctl.conf or a file under /etc/sysctl.d/, which are read at boot; sysctl -p applies them again immediately. Turning a machine into a router — net.ipv4.ip_forward = 1 — is the example the exam reaches for every time.

$ cat /proc/sys/net/ipv4/ip_forward0$ sysctl net.ipv4.ip_forwardnet.ipv4.ip_forward = 0$ sysctl -w net.ipv4.ip_forward=1net.ipv4.ip_forward = 1# Gone at the next reboot. To keep it:$ echo 'net.ipv4.ip_forward = 1' > /etc/sysctl.d/99-router.conf$ sysctl -p /etc/sysctl.d/99-router.conf

Firmware, and what is below the operating system

Before Linux runs at all, the machine's own firmware has already decided which devices exist. Classic BIOS and modern UEFI both offer a setup screen where integrated peripherals — the onboard serial port, the audio chip, virtualization extensions — can be enabled or disabled, and a device switched off there is invisible to Linux no matter what you do afterwards. UEFI adds an EFI system partition, a FAT-formatted partition holding boot loaders, which is why /boot/efi appears in the mount table on modern installs.

One small thing worth knowing about where these tools live: the ls* inspection commands are ordinary user programs in /usr/bin — /usr/bin/lspci, /usr/bin/lsusb, /usr/bin/lsblk — while the ones that change kernel state live in /sbin or /usr/sbin, which is why /sbin/lsmod and /sbin/modprobe are the paths that turn up in a question about a command a normal user cannot run. lsblk, the block-device version of the same idea, lists disks and partitions as a tree.