Troubleshooting a network connection

Working out why a connection fails, in the order that isolates the fault fastest: the interface, the gateway, the route, name resolution, then the service itself.

Lesson 3 of 4 in objective 109. Networking fundamentals, part of LPIC-1 Exam 102-500.

Work outwards, and the first test that fails is the fault. A search over 5 places, tried in this order: the first hit ends it, and everything under the one that hits is never looked at. Going down a step means that test answered. First, ip addr (is there an address at all?); a hit there means the interface — no address, or 169.254 because DHCP never answered. Then, ping the gateway (ip route names it); a hit there means the local network, not anything beyond it. Then, ping an address (a public resolver, so DNS is kept out of it); a hit there means routing, since no name was involved. Then, ping the same NAME; a hit there means resolution and nothing else — dig, host, /etc/resolv.conf. Last, ss -tulpn (listening on which address, owned by which process?); a hit there means the service, not the network — 127.0.0.1 is invisible from elsewhere. A miss at every one of them: Every test answers: network fine, service listening — the logs say why. ip addr is there an address at all? the interface — no address, or 169.254 because DHCP never answered that test answered ping the gateway ip route names it the local network, not anything beyond it ping an address a public resolver, so DNS is kept out of it routing, since no name was involved ping the same NAME resolution and nothing else — dig, host, /etc/resolv.conf ss -tulpn listening on which address, owned by which process? the service, not the network — 127.0.0.1 is invisible from elsewhere Every test answers: network fine, service listening — the logs say why
Work outwards, and the first test that fails is the fault.

Work outwards

Start at the interface: ip addr shows whether it has an address at all, and one in 169.254 means DHCP never answered. Then the local network: ping the default gateway, which ip route names. Then beyond it: ping something by address, such as a public resolver, which tests routing without involving DNS. Then ping the same thing by NAME — if the address works and the name does not, the fault is resolution and nothing else, and dig, host and /etc/resolv.conf are where to look.

traceroute (or tracepath, or mtr) shows each hop and where the path stops, which distinguishes "the packets never leave" from "they leave and nothing comes back". Note that many hosts and routers deliberately do not answer ICMP, so a silent hop in the middle of an otherwise working trace is normal rather than a fault.

Read a traceroute by how far it gets, not by whether every hop answers. A range from This host to The destination. Nothing out: nothing leaves. Stops part way: no replies back. Runs all the way: gaps are normal. This host The destination Nothing out nothing leaves Stops part way no replies back Runs all the way gaps are normal
Read a traceroute by how far it gets, not by whether every hop answers.

When the network is fine and the service is not

ss -tulpn lists what is actually listening, on which addresses, and which process owns it — a service bound only to 127.0.0.1 is invisible from anywhere else, and that is the answer to a surprising number of "the firewall must be blocking it" reports. netcat (nc -zv host port) tests reachability of one port from the client side, which separates a filtered port from a refused one: refused means something answered and nothing is listening, timed out means a firewall silently dropped it.

Once the fault is isolated, the logs say why. journalctl -u SERVICE for the service's own account of itself, and /var/log for anything not in the journal.

One port probe, and what each of its three answers proves. One test, and exactly one way out of it. The test is: What does nc -zv report about the port? (the distinction is something answering versus nothing at all). Refused, at once leads to Nothing is listening (something answered, so the path works — the service is the fault); A silent timeout leads to A firewall dropped it (refused and filtered are different faults); It connects leads to Port open (whatever is wrong sits above the network). What does nc -zv report about the port? the distinction is something answering versus nothing at all Refused, at once A silent timeout It connects Nothing is listening something answered, so the path works — the service is the fault A firewall dropped it refused and filtered are different faults Port open whatever is wrong sits above the network
One port probe, and what each of its three answers proves.

The two toolkits, and the names a question will hand you

Half of this objective is recognising which packet a command came out of. net-tools is the legacy set — ifconfig, route, netstat, arp and hostname — and it works by reading /proc. iproute2 is the current set — ip, ss and bridge — and it talks to the kernel over netlink, which is why it can express things the older tools cannot: several addresses on one interface without alias devices, policy routing, multiple routing tables. The pairings are explicitly examinable in both directions: ifconfig to ip addr and ip link, route to ip route, arp to ip neigh, and netstat to ss. Note that tracepath belongs to neither set, and that hostname is the one net-tools command nothing has replaced.

hostname with no options and no arguments prints the name the kernel is currently holding, read through the gethostname call. Given an argument and run as root it SETS that name, but only until the next reboot, since the persistent value is /etc/hostname and, on a systemd machine, hostnamectl. uname -n reports the same kernel value through a different tool. Its address options are the pair worth separating: hostname -i asks the resolver what the host's own name maps to, so it goes through /etc/hosts and very often answers with a loopback address the installer put there, while hostname -I reads the addresses actually assigned to the interfaces and needs no resolution at all. A provisioning script that must not depend on name resolution wants -I, or ip -o addr show when it also needs the interface and the prefix length. -f gives the fully qualified name and -s the short one.

Several of the classic diagnostics ship as an IPv4 and an IPv6 pair, with the IPv6 member carrying a 6 suffix: traceroute6, tracepath6 and ping6. Current distributions usually make these links to a dual-stack binary that picks the family from the address or from -4 and -6, but the separate names are still valid and are named in the objectives — so on an older host whose traceroute rejects -6, traceroute6 is the program to run. The IPv6 variants otherwise take the same options as their IPv4 counterparts.

Worth carrying in

ip addr / ip route
Does the interface have an address, and where do packets go by default.
ping -c 4 address
Reachability without DNS. Ping the gateway first, then beyond it.
traceroute / mtr
Each hop, and where the path stops.
dig name / host name
Ask DNS directly. +short for just the answer.
ss -tulpn
What is listening, on which address, owned by which process.
nc -zv host port
Refused means nothing is listening; timeout means something dropped it.
hostname / hostname -I
The running name; -I the addresses on the interfaces. -i goes through the resolver and often says 127.0.0.1.
traceroute6 / ping6
The separate IPv6 programs, for hosts whose IPv4 binary has no -6.
ifconfig, route, netstat, arp
net-tools. Replaced by ip addr, ip route, ss and ip neigh.

What the exam does with this

Objective
109. Networking fundamentals
Share of the exam
23.33% (the whole objective)
Questions in this lesson
20
Signed for by a person
0

Partly checked. None of the 20 questions here has been read against the cited source by a person. 20 questions have been checked against their cited clause by an automated pass — which is not the same thing, and is not a signature.

Only questions a person has signed for are used in mock exams here. That is the whole difference between the two kinds of checking above.

How these questions are written — where each question comes from, what the verification ledger records, and what happens when one is found wrong.

Drill this lesson

A lesson is one sitting: the trainer draws a short run from these questions alone and spaces the ones you get wrong.

Practise Troubleshooting a network connection

Questions in this lesson

Practise Troubleshooting a network connection

The rest of objective 109