Searching text with regular expressions

Regular expressions, and the two dialects Linux tools use: what basic and extended expressions differ on, the metacharacters worth memorising, and how grep and sed apply them.

Lesson 7 of 8 in objective 103. GNU and Unix commands, part of LPIC-1 Exam 101-500.

The whole practical difference between the two dialects. Basic — One or more: \+; Zero or one: \?; Alternation: \|; Grouping: \( \); Spoken by: grep, sed. Extended — One or more: +; Zero or one: ?; Alternation: |; Grouping: ( ); Spoken by: grep -E, sed -E Basic Extended One or more \+ + Zero or one \? ? Alternation \| | Grouping \( \) ( ) Spoken by grep, sed grep -E, sed -E
The whole practical difference between the two dialects.

The metacharacters

. matches any single character. * means "zero or more of the thing before it" — so a* matches an empty string, which is the source of half the confusion about it. ^ anchors to the start of a line and $ to the end, so ^$ matches an empty line. Square brackets are a set: [aeiou] one vowel, [^0-9] one character that is not a digit, [a-z] a range. A backslash removes the special meaning of whatever follows, which is how you match a literal dot.

The extended metacharacters are +, ?, |, () and {}. + is one or more, ? is zero or one, | is alternation, parentheses group, and {2,4} is a repeat count. In BASIC regular expressions these need backslashes — \+, \?, \| — and in EXTENDED ones they do not. That is the entire practical difference between the dialects, and it is why the same pattern can work in egrep and fail in grep.

One pattern, cut into the metacharacters both dialects share. ^[aeiou].*\.$ — part 1, ^: anchors to the start of the line; part 2, [aeiou]: one vowel from the set. [^0-9] is one non-digit; part 3, .: any single character; part 4, *: zero or more of the dot to its left — zero still matches; part 5, \.: the backslash cancels it, leaving a literal dot; part 6, $: anchors to the end. ^$ alone is an empty line. 1 ^ 2 [aeiou] 3 . 4 * 5 \. 6 $ 1 anchors to the start of the line 2 one vowel from the set. [^0-9] is one non-digit 3 any single character 4 zero or more of the dot to its left — zero still matches 5 the backslash cancels it, leaving a literal dot 6 anchors to the end. ^$ alone is an empty line
One pattern, cut into the metacharacters both dialects share.

Which tool speaks which dialect

grep uses basic expressions; grep -E (the modern spelling of egrep) uses extended ones; grep -F (fgrep) turns the pattern off entirely and matches fixed strings, which is both faster and the right answer when the search text contains dots and asterisks you mean literally. sed uses basic expressions too, with sed -E for extended.

The grep flags that carry marks: -i ignores case, -v inverts the match, -c counts matching lines rather than printing them, -l prints only the names of matching files, -o prints only the matched part, -r searches recursively, and -n prefixes the line number. grep -c counts LINES, not matches, which is a distinction a question will lean on.

Which command reads a pattern which way, including not at all. Left column, What you type; right column, How the pattern is read. grep and sed both point at Basic expressions (Write + ? | ( ) with a backslash in front). grep -E, egrep and sed -E all point at Extended expressions (Same metacharacters, no backslash. egrep is just the old name). grep -F and fgrep both point at Fixed strings (Not read at all. Faster, and right for literal dots). What you type How the pattern is read grep sed Basic expressions Write + ? | ( ) with a backslash in front grep -E egrep sed -E Extended expressions Same metacharacters, no backslash. egrep is just the old name grep -F fgrep Fixed strings Not read at all. Faster, and right for literal dots
Which command reads a pattern which way, including not at all.

Worth carrying in

^ $
Anchors: start of line, end of line. ^$ is an empty line.
.
Any single character.
*
Zero or more of the preceding element.
[^0-9]
One character not in the set.
grep -E
Extended expressions: + ? | ( ) { } work without backslashes.
grep -F
Fixed strings, no pattern interpretation.
grep -v
Invert: print lines that do NOT match.
grep -c
Count matching lines — lines, not matches.
grep -o
Print only the matched portion.

What the exam does with this

Objective
103. GNU and Unix commands
Share of the exam
43.33% (the whole objective)
Questions in this lesson
15
Signed for by a person
0

Partly checked. None of the 15 questions here has been read against the cited source by a person. 15 questions have been checked against their cited clause by an automated pass — which is not the same thing, and is not a signature.

Only questions a person has signed for are used in mock exams here. That is the whole difference between the two kinds of checking above.

How these questions are written — where each question comes from, what the verification ledger records, and what happens when one is found wrong.

Drill this lesson

A lesson is one sitting: the trainer draws a short run from these questions alone and spaces the ones you get wrong.

Practise Searching text with regular expressions

Questions in this lesson

Practise Searching text with regular expressions

The rest of objective 103