This is a Sed (Stream Editor) example. A Stream Editor is used to perform basic transformations on text read from a file or a pipe. The result is sent to standard output. The syntax for the sed command has no output file specification, but results can be saved to a file using output redirection. The editor does not modify the original input.
What distinguishes sed from other editors, such as vi and ed, is its ability to filter text that it gets from a pipeline feed. You do not need to interact with the editor while it is running; that is why sed is sometimes called a batch editor. This feature allows use of editing commands in scripts, greatly easing repetitive editing tasks. When facing replacement of text in a large number of files, sed is a great help.
The following table shows an overview of the whole article:
1. SED Commands
The sed program can perform text pattern substitutions and deletions using regular expressions, like the ones used with the grep command. The editing commands are similar to the ones used in the vi editor.
1.1 SED editing commands
| Command | Result |
|---|---|
| a\ | Append text below current line. |
| c\ | Change text in the current line with new text. |
| d | Delete text. |
| i\ | Insert text above current line. |
| p | Print text. |
| r | Read a file. |
| s | Search and replace text. |
| w | Write to a file. |
1.2 Sed options
| Option | Effect |
|---|---|
| -e SCRIPT | Add the commands in SCRIPT to the set of commands to be run while processing the input. |
| -f | Add the commands contained in the file SCRIPT-FILE to the set of commands to be run while processing the input. |
| -n | Silent mode. |
| -V | Print version information and exit. |
2. Interactive editing
2.1 Printing lines containing a pattern
This is something you can do with grep, of course, but you can’t do a “find and replace” using that command. This is just to get you started.
This is our example text file:
Test.txt
We shall not spend a large expense of time Before we reckon with your several loves, And make us even with you. My thanes and kinsmen, Henceforth be earls, the first that ever Scotland In such an honour named. What's more to do, Which would be planted newly with the time, As calling home our exiled friends abroad That fled the snares of watchful tyranny; Producing forth the cruel ministers Of this dead butcher and his fiend-like queen, Who, as 'tis thought, by self and violent hands Took off her life; this, and what needful else That calls upon us, by the grace of Grace, We will perform in measure, time and place: So, thanks to all at once and to each one, Whom we invite to see us crown'd at Scone. Macbeth, William Shakespeare
We want sed to find all the lines containing our search pattern, in this case “erors”. We use the p to obtain the result:
As you notice, sed prints the entire file, but the lines containing the search string are printed twice. This is not what we want. In order to only print those lines matching our pattern, use the -n option:
2.2 Deleting lines of input containing a pattern
We use the same example text file. Now we only want to see the lines not containing the search string:
The d command results in excluding lines from being displayed. Matching lines starting with a given pattern and ending in a second pattern are showed like this:
Note that the last dot needs to be escaped in order to actually match. In our example the expression just matches any character, including the last dot.
2.3 Ranges of lines
This time we want to take out the lines containing the errors. In the example these are lines 2 to 4. Specify this range to address, together with the d command:
To print the file starting from a certain line until the end of the file, use a command similar to this:
This only prints the first two lines of the example file. The following command prints the first line containing the pattern “a text”, up to and including the next line
containing the pattern “a line”:
2.4 Find and replace with sed
In the example file, we will now search and replace the errors instead of only (de)selecting the lines containing the search string.
To insert a string at the beginning of each line of a file, for instance for quoting:
Insert some string at the end of each line:
Multiple find and replace commands are separated with individual -e options:
Keep in mind that by default sed prints its results to the standard output, most likely your terminal window. If
you want to save the output to a file, redirect it:
sed option 'some/expression' file_to_process > sed_output_in_a_file
3. Non-interactive editing
3.1 Reading sed commands from a file
Multiple sed commands can be put in a file and executed using the -f option. When creating such a file,
make sure that:
- No trailing white spaces exist at the end of lines.
- No quotes are used.
- When entering text to add or replace, all except the last line end in a backslash.
3.2 Writing output files
Writing output is done using the output redirection operator >. This is an example script used to create very simple HTML files from plain text files.
script.sed
1i\ <html>\ <head><title>generated html by sed</title></head>\ <body bgcolor="#ffffff">\ <pre> $a\ </pre>\ </body>\ </html>
txt2html.sh
#!/bin/bash # This is a simple script that you can use for converting text into HTML. # First we take out all newline characters, so that the appending only happens # once, then we replace the newlines. echo "converting $1..." SCRIPT="script.sed" NAME="$1" RESULT="$2" TEMPFILE="sed.$PID.tmp" sed "s/\n/^M/" $NAME | sed -f $SCRIPT | sed "s/^M/\n/" > $TEMPFILE mv $TEMPFILE $RESULT echo "done."
$1 holds the first argument to a given command, in this case the name of the file to convert:
input.txt
This is a sed generated html file
After transforming we get the following output:
output.txt
<html> <head><title>generated html by sed</title></head> <body bgcolor="#ffffff"> <pre> This is a sed generated html file </pre> </body> </html>
This is not really how it is done; this example just demonstrates sed capabilities.
4. Summary
The sed stream editor is a powerful command line tool, which can handle streams of data: it can take input lines from a pipe. This makes it fit for non-interactive use. The sed editor uses vi-like commands and accepts regular expressions.
The sed tool can read commands from the command line or from a script. It is often used to perform find-and-replace actions on lines containing a pattern.
