103.7 Lesson 1

There is another type of atom named bracket expression. Although not a single character, brackets [] (including their content) are considered a single atom. A bracket expression usually is just a list of literal characters enclosed by [], making the atom match any single character from the list. For example, the expression [1b] can be found in both strings abcd and a1cd. To specify characters the atom should not correspond to, the list must begin with ^, as in [^1b]. It is also possible to specify ranges of characters in bracket expressions. For example, [0−9] matches digits 0 to 9 and [a−z] matches any lowercase letter. Ranges must be used with caution, as they might not be consistent across distinct locales.

Bracket expression lists also accept classes instead of just single characters and ranges. Traditional character classes are:

Character classes can be combined with single characters and ranges, but may not be used as an endpoint of a range. Also, character classes may be used only in bracket expressions, not as an independent atom outside the brackets.

The reach of an atom, either a single character atom or a bracket atom, can be adjusted using an atom quantifier. Atom quantifiers define atom sequences, that is, matches occur when a contiguous repetition for the atom is found in the string. The substring corresponding to the match is called a piece. Notwithstanding, quantifiers and other features of regular expressions are treated differently depending on which standard is being used.

As defined by POSIX, there are two forms of regular expressions: “basic” regular expressions and “extended” regular expressions. Most text related programs in any conventional Linux distribution support both forms, so it is important to know their differences in order to avoid compatibility issues and to pick the most suitable implementation for the intended task.

The * quantifier has the same function in both basic and extended REs (atom occurs zero or more times) and it’s a literal character if it appears at the beginning of the regular expression or if it’s preceded by a backslash \. The plus sign quantifier ` will select pieces containing one or more atom matches in sequence. With the question mark quantifier `?`, a match will occur if the corresponding atom appears once or if it doesn't appear at all. If preceded by a backslash `\`, their special meaning is not considered. Basic regular expressions also support ` and ? quantifiers, but they need to be preceded by a backslash. Unlike extended regular expressions, + and ? by themselves are literal characters in basic regular expressions.

A bound is an atom quantifier that, as the name implies, allows a user to specify precise quantity boundaries for an atom. In extended regular expressions, a bound may appear in three forms:

In any case, if a substring matches with a regular expression and a longer substring starting at the same point also matches, the longer substring will be considered.

Basic regular expressions also support bounds, but the delimiters must be preceded by \: \{ and \}. By themselves, { and } are interpreted as literal characters. A \{ followed by a character other than a digit is a literal character, not the beginning of a bound.

Basic regular expressions also differ from extended regular expressions in another important aspect: an extended regular expression can be divided into branches, each one an independent regular expression. Branches are separated by | and the combined regular expression will match anything that corresponds to any of the branches. For example, he|him will match if either substring he or him are found in the string being examined. Basic regular expressions interpret | as a literal character. However, most programs supporting basic regular expressions will allow branches with \|.

An extended regular expression enclosed in () can be used in a back reference. For example, ([[:digit:]])\1 will match any regular expression that repeats itself at least once, because the \1 in the expression is the back reference to the piece matched by the first parenthesized subexpression. If more than one parenthesized subexpression exist in the regular expression, they can be referenced with \2, \3 and so on.

For basic REs, subexpressions must be enclosed by \( and \), with ( and ) by themselves ordinary characters. The back reference indicator is used like in extended regular expressions.

The immediate benefit offered by regular expressions is to improve searches on filesystems and in text documents. The -regex option of command find allows to test every path in a directory hierarchy against a regular expression. For example,

searches for files greater than 100 megabytes (100 units of 1048576 bytes), but only in paths inside the user’s home directory that do contain a match with .*/\..*, that is, a /. surrounded by any other number of characters. In other words, only hidden files or files inside hidden directories will be listed, regardless of the position of /. in the corresponding path. For case insensitive regular expressions, the -iregex option should be used instead:

In this example, the regular expression contains branches (written in extended style) to list only specific font files under the /usr/share/fonts directory hierarchy. Extended regular expressions are not supported by default, but find allows for them to be enabled with -regextype posix-extended or -regextype egrep. The default RE standard for find is findutils-default, which is virtually a basic regular expression clone.

It is often necessary to pass the output of a program to command less when it doesn’t fit on the screen. Command less splits its input in pages, one screenful at a time, allowing the user to easily navigate the text up and down. In addition, less also allows a user to perform regular expression based searches. This feature is notably important because less is the default paginator used for many everyday tasks, like inspecting journal entries or consulting manual pages. When reading a manual page, for instance, pressing the / key will open a search prompt. This is a typical scenario in which regular expressions are useful, as command options are listed just after a page margin in the general manual page layout. However, the same option might appear many times through the text, making literal searches unfeasible. Regardless of that, typing ^[[:blank:]]*-o — or more simply: ^ *-o —  in the search prompt will jump immediately to option the -o section (if it exists) after pressing Enter, thus allowing one to consult an option description more rapidly.