Chapter 13 of 17
Transform structured text
Choose a tool that understands the input structure and validate the output.
The mental model.
cut selects delimited fields, tr translates or deletes characters, sed applies stream edits, and awk processes records and fields. Use the simplest tool that matches the format rather than combining opaque expressions.
In awk, $0 is the whole record, $1 is the first field, and NF is the number of fields. -F chooses a field separator. Quote the awk program so that the shell does not expand its dollar signs.
Regular expressions and shell globs are different languages. In a regex, . matches a character and * repeats the preceding atom. In a shell glob, * matches a string of characters. Anchor regexes with ^ and $ when you mean a whole line.
CSV can contain quoted delimiters and JSON can contain nested values. A naive cut or grep expression is not a general parser. Use a CSV-aware tool for real CSV and jq for JSON, and inspect sample output before replacing source data.
Read it. Predict it. Try it.
Before running these commands, predict what each line will change or print. Launch a quest below to open the simulated terminal.
cut -d: -f1 /etc/passwd
awk -F: '{print $1}' /etc/passwd
echo "Linux" | tr a-z A-ZThe first two commands select usernames from a colon-delimited file; the third changes character case.
Check your understanding
Explain the why.
Put it into practice.
Complete the exercises, then try the Boss Challenge without hints. Each quest checks its own objectives. You can revisit this chapter at any time.
Make the learning stick.
Close the explanation and describe the main idea in your own words. Change a filename or input and predict the outcome. Tomorrow, repeat a quest without hints. Then apply it in a disposable real Linux environment.
Open the real-system labs →