Getting Started with sed

Overview

sed (stream editor) is a powerful command-line tool for text processing. It allows you to manipulate and transform text in streams or files efficiently. This guide covers basic to advanced sed operations, with practical examples for system administrators and developers.

Basic Syntax

sed [options] 'script' inputfile

script: The operation to perform (e.g., search, replace).
inputfile: The file to be edited.

Key Concepts

s: Substitute (search and replace).
Addressing: Specify lines to act on (e.g., line numbers, patterns).
Flags: Modify behavior (e.g., global replacement with g).

Beginner Lesson: Basic Operations

Search and Replace

Replace the first occurrence of oldtext with newtext:

sed 's/oldtext/newtext/' file.txt

Replace all occurrences in a line with the g flag:

sed 's/oldtext/newtext/g' file.txt

Preview Changes

By default, sed outputs changes to the console. To save changes, use -i:

sed -i 's/oldtext/newtext/' file.txt

Delete Lines

Delete specific lines (e.g., line 3):

sed '3d' file.txt

Delete lines matching a pattern:

sed '/pattern/d' file.txt

Extract Specific Lines

Print only lines 5 to 10:

sed -n '5,10p' file.txt

Intermediate Lesson: Advanced Techniques

Using Regular Expressions

Replace all digits with #:

sed 's/[0-9]/#/g' file.txt

Inserting or Appending Text

Insert before line 2:

sed '2i\This is inserted text' file.txt

Append after line 2:

sed '2a\This is appended text' file.txt

Replacing Multiple Patterns

Replace multiple patterns in one command:

sed -e 's/foo/bar/' -e 's/baz/qux/' file.txt

Transform Case

Convert lowercase to uppercase:

sed 's/[a-z]/\U&/g' file.txt

Use Cases for System Admins

Batch Configuration Changes

Replace settings in multiple .conf files:

sed -i 's/old_setting/new_setting/' *.conf

Log File Cleanup

Remove sensitive data (e.g., IP addresses):

sed -r 's/[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/REDACTED/g' log.txt

Mass User Management

Update usernames in /etc/passwd:

sed -i 's/old_user/new_user/' /etc/passwd

Automation in Scripts

Automate renaming files in a directory:

for file in *.txt; do
  sed -i 's/foo/bar/' "$file"
done

Conclusion

sed is an indispensable tool for text processing and automation in Linux. Whether you're performing simple search-and-replace operations or managing complex configurations, sed offers a powerful and efficient solution. By mastering these techniques, you can streamline your workflow and enhance your productivity as a system administrator or developer.