# Sed Cheat Sheet: 30 One-Liners from Real Production Logs - URL: https://karandeepsingh.ca/posts/sed-command-cheat-sheet-30-essential-one-liners/ - Author: Karandeep Singh - Published: 2025-01-28 - Updated: 2026-07-16 - Series: Mastering sed - Tags: bash, sed - Summary: Thirty sed one-liners for everyday log and config work. Each pattern solves a common problem (extract errors, clean configs, parse timestamps, transform JSON). Organized by use case with input/output examples and performance notes. I manage Jenkins build fleets on AWS, and most of my sed use comes from that work: digging through build logs, cleaning up configuration files, and normalizing text before it goes anywhere else. The patterns in this cheat sheet cover jobs like those: extract error messages, clean configuration files, parse timestamps, transform structured text. This isn't a complete sed reference. It's the 30 patterns I actually reach for, organized by the problems they solve. Each includes the input, the sed command, the output, and when to reach for it. ![Terminal showing sed command](https://karandeepsingh.ca/images/posts/sed-command-cheat-sheet-30-essential-one-liners-card.jpg) ## Why These 30 Patterns Every pattern in this list earns its place through day-to-day needs: - **Patterns 1-5:** Basic substitutions for config file updates during deployments - **Patterns 6-10:** Line selection for extracting specific log entries - **Patterns 11-18:** Log cleanup and normalization - **Patterns 19-25:** Text transformation for data migration - **Patterns 26-30:** Advanced patterns for JSON/XML parsing The most-used patterns: #2 (global substitution), #11 (delete matching lines), #14 (delete empty lines), #22 (trim whitespace). These four handle 80% of daily tasks. ## Pattern 1: Basic Substitution (First Match Per Line) **Problem:** API logs show `ERROR` but monitoring expects `WARN` for non-critical issues. **Input (application.log):** ``` 2024-02-20 10:30:05 ERROR connection timeout to auth-api 2024-02-20 10:30:06 ERROR connection timeout to auth-api 2024-02-20 10:30:07 INFO request completed ``` **Command:** ```text $ sed 's/ERROR/WARN/' application.log 2024-02-20 10:30:05 WARN connection timeout to auth-api 2024-02-20 10:30:06 WARN connection timeout to auth-api 2024-02-20 10:30:07 INFO request completed ``` **Notes:** Only replaces first `ERROR` on each line. If a line has `ERROR...ERROR`, only first changes. Use pattern #2 for all occurrences. **Performance:** Processes large log files quickly on a single CPU core. ### 2. Global Substitution (All Occurrences) ```bash sed 's/old/new/g' file.txt ``` **Example:** Replace all occurrences of "color" with "colour" throughout the file ```bash sed 's/color/colour/g' document.txt ``` ### 3. Case-Insensitive Substitution ```bash sed 's/old/new/gi' file.txt ``` **Example:** Replace all occurrences of "user" (in any case) with "customer" ```bash sed 's/user/customer/gi' users.txt ``` ### 4. Substituting with Backup ```bash sed -i.bak 's/old/new/g' file.txt ``` **Example:** Replace all "http://" with "https://" and create a backup ```bash sed -i.bak 's/http:\/\//https:\/\//g' urls.txt ``` ### 5. Using Alternative Delimiters ```bash sed 's|old|new|g' file.txt ``` **Example:** Replace file paths with cleaner syntax ```bash sed 's|/var/www/old/|/var/www/new/|g' config.txt ``` ## Line Selection and Filtering These commands help you target specific lines for operations based on line numbers, patterns, or ranges. ### 6. Apply Command to Specific Line by Number ```bash sed '5s/old/new/' file.txt ``` **Example:** Replace "username" with "user_id" only on line 5 ```bash sed '5s/username/user_id/' config.txt ``` ### 7. Apply Command to a Range of Lines ```bash sed '5,10s/old/new/g' file.txt ``` **Example:** Add prefix to lines 5-10 ```bash sed '5,10s/^/PREFIX: /' log.txt ``` ### 8. Apply Command to Lines Matching a Pattern ```bash sed '/pattern/s/old/new/g' file.txt ``` **Example:** Replace "disabled" with "enabled" only in lines containing "feature" ```bash sed '/feature/s/disabled/enabled/g' settings.txt ``` ### 9. Apply Command from Pattern to End of File ```bash sed '/pattern/,$s/old/new/g' file.txt ``` **Example:** Update version in all configuration blocks after the main section ```bash sed '/\[main\]/,$s/version=1\.0/version=2.0/g' config.ini ``` ### 10. Apply Command to Range Between Two Patterns ```bash sed '/start_pattern/,/end_pattern/s/old/new/g' file.txt ``` **Example:** Update database settings only within the database configuration block ```bash sed '/\[database\]/,/\[/s/host=localhost/host=db.example.com/g' config.ini ``` ## Line Deletion and Insertion These commands add or remove entire lines, essential for cleaning up files or adding new content. ### 11. Delete Lines Matching a Pattern ```bash sed '/pattern/d' file.txt ``` **Example:** Remove all comment lines ```bash sed '/^#/d' config.txt ``` ### 12. Delete a Specific Line by Number ```bash sed '5d' file.txt ``` **Example:** Remove the header line (line 1) ```bash sed '1d' data.csv ``` ### 13. Delete a Range of Lines ```bash sed '5,10d' file.txt ``` **Example:** Remove lines 1-5 (like skipping a header) ```bash sed '1,5d' report.txt ``` ### 14. Delete Empty Lines ```bash sed '/^$/d' file.txt ``` **Example:** Clean up a file by removing all blank lines ```bash sed '/^$/d' script.sh ``` ### 15. Delete Lines Matching Multiple Patterns ```bash sed '/pattern1/d; /pattern2/d' file.txt ``` **Example:** Remove both commented lines and empty lines ```bash sed '/^#/d; /^$/d' config.txt ``` ### 16. Append Text After a Match ```bash sed '/pattern/a\new text' file.txt ``` **Example:** Add a warning after any configuration line containing "deprecated" ```bash sed '/deprecated/a\# WARNING: This setting will be removed in the next version' config.txt ``` ### 17. Insert Text Before a Match ```bash sed '/pattern/i\new text' file.txt ``` **Example:** Add a header before the main configuration section ```bash sed '/\[main\]/i\# Main Configuration - Updated 2023-08-01' config.ini ``` ### 18. Replace an Entire Line ```bash sed '/pattern/c\new line' file.txt ``` **Example:** Replace version line with new version ```bash sed '/^version=/c\version=2.0.0' app.properties ``` ## Advanced Text Manipulation These powerful commands handle more complex text manipulation requirements. ### 19. Using Capture Groups in Substitutions ```bash sed 's/\(pattern\).*$/\1 new text/' file.txt ``` **Example:** Keep usernames but standardize the email domain ```bash sed 's/\(user[^@]*\)@.*/\1@company.com/' emails.txt ``` ### 20. Multiple Substitutions in One Command ```bash sed -e 's/old1/new1/g' -e 's/old2/new2/g' file.txt ``` **Example:** Replace multiple HTML tags in one pass ```bash sed -e 's/

/# /g' -e 's/

/## /g' -e 's/<\/h[12]>//g' html.txt ``` ### 21. Removing Leading Whitespace ```bash sed 's/^[ \t]*//' file.txt ``` **Example:** Fix indentation in a text file ```bash sed 's/^[ \t]*//' poorly_formatted.txt ``` ### 22. Removing Trailing Whitespace ```bash sed 's/[ \t]*$//' file.txt ``` **Example:** Clean up trailing spaces from code ```bash sed 's/[ \t]*$//' code.js ``` ### 23. Adding Line Numbers ```bash sed = file.txt | sed 'N;s/\n/\t/' ``` **Example:** Create a numbered list of items ```bash sed = todo.txt | sed 'N;s/\n/. /' ``` ### 24. Converting DOS/Windows Line Endings to Unix ```bash sed 's/\r$//' file.txt ``` **Example:** Fix line endings in a script transferred from Windows ```bash sed 's/\r$//' windows_script.sh ``` ### 25. Swapping the Order of Two Words ```bash sed 's/\(word1\) \(word2\)/\2 \1/g' file.txt ``` **Example:** Reverse "first name, last name" to "last name, first name" ```bash sed 's/\([^,]*\), \(.*\)/\2 \1/' names.txt ``` ## Advanced sed Techniques These techniques showcase some of sed's more powerful capabilities for complex text processing. ### 26. Printing Specific Lines Only ```bash sed -n '/pattern/p' file.txt ``` **Example:** Extract all error messages from a log file ```bash sed -n '/ERROR:/p' application.log ``` ### 27. Applying Multiple Commands to the Same Pattern ```bash sed '/pattern/{s/old/new/g; s/foo/bar/g}' file.txt ``` **Example:** Update multiple fields in a JSON configuration ```bash sed '/"database":/,/}/{s/"host": "localhost"/"host": "db.example.com"/; s/"port": 3306/"port": 5432/}' config.json ``` ### 28. Conditional Command Execution ```bash sed '/pattern/{/another/d;}' file.txt ``` **Example:** Remove debug messages from production logs only ```bash sed '/\[PRODUCTION\]/{/DEBUG:/d;}' application.log ``` ### 29. Working with CSV Using Hold Space ```bash sed 'h; s/[^,]*,//; G; s/\n/,/; s/,\([^,]*\),.*/,\1/' file.csv ``` **Example:** Swap first and last columns in a two-column CSV ```bash sed -E 's/^([^,]+),([^,]+)$/\2,\1/' ids_names.csv ``` ### 30. Processing Multi-line Blocks ```bash sed -n '/start/,/end/p' file.txt ``` **Example:** Extract complete XML tags with their content ```bash sed -n '//,/<\/user>/p' users.xml ``` ## Real-World Examples: Solving Common Tasks Now let's see how these commands can be applied to real-world tasks that developers and system administrators face daily. ### Example 1: Updating Configuration Files ```bash # Change server settings in Apache config sed -i.bak \ -e 's/^Listen 80$/Listen 8080/' \ -e '/^#.*SSL/s/^#//' \ -e '/DocumentRoot/s|/var/www/html|/var/www/app|' \ httpd.conf ``` This complex example: 1. Changes the listen port from 80 to 8080 2. Uncomments all SSL-related settings 3. Updates the document root path 4. Creates a backup of the original file ### Example 2: Log File Analysis ```bash # Extract error timestamps and messages sed -n '/ERROR/s/\(.*\) ERROR: \(.*\)/\1 | \2/p' application.log ``` This command: 1. Finds lines containing "ERROR" 2. Reformats them to show just the timestamp and message 3. Uses capture groups to preserve and rearrange the important information ### Example 3: Data Cleaning for CSV Files ```bash # Clean and format a CSV file for import sed -e 's/^[ \t]*//' -e 's/[ \t]*$//' -e 's/"//g' -e '/^$/d' -e 's/,\{2,\}/,NA,/g' data.csv ``` This command chain: 1. Removes leading whitespace 2. Removes trailing whitespace 3. Removes all quotation marks 4. Deletes empty lines 5. Replaces consecutive commas with ",NA," to properly handle missing values ### Example 4: HTML Conversion to Markdown ```bash # Convert simple HTML to Markdown sed -e 's/

\(.*\)<\/h1>/# \1/g' \ -e 's/

\(.*\)<\/h2>/## \1/g' \ -e 's/\(.*\)<\/b>/**\1**/g' \ -e 's/\(.*\)<\/i>/*\1*/g' \ -e 's/\(.*\)<\/a>/[\2](\1)/g' \ simple.html ``` This complex transformation: 1. Converts HTML headers to Markdown headers 2. Converts bold and italic tags to Markdown syntax 3. Transforms HTML links to Markdown link format ## Understanding Sed's Syntax Deeply To truly master `sed`, it's important to understand its core syntax components. The sed command structure breaks down as follows: ``` sed [options] 'address1[,address2][!]{command}' file ``` Where: - **options** modify sed's behavior (like `-i` for in-place editing) - **address** specifies which lines to operate on (line numbers or patterns) - **!** inverts the line selection (operate on all lines EXCEPT those selected) - **command** is the operation to perform (like 's' for substitute) For example: ```bash sed -i '5,10!s/old/new/g' file.txt ``` This applies the substitution to all lines EXCEPT lines 5 through 10. ## Sed Options You Should Know These options dramatically extend sed's capabilities: **-i[suffix]** - Edit files in place (with optional backup) ```bash sed -i.bak 's/old/new/g' file.txt ``` **-e** - Specify multiple commands ```bash sed -e 's/1/one/g' -e 's/2/two/g' file.txt ``` **-f** - Read commands from a file ```bash sed -f commands.sed file.txt ``` **-n** - Suppress automatic printing ```bash sed -n '/pattern/p' file.txt ``` **-r or -E** - Use extended regular expressions ```bash sed -E 's/([0-9]+)/Number: \1/g' file.txt ``` ## Best Practices from the Experts These practices will make your sed commands more robust: 1. **Test commands without `-i` first** to preview changes before modifying files. 2. **Use alternative delimiters** like `|` or `@` when your pattern contains slashes. 3. **Create backups** with `-i.bak` for important files. 4. **Comment complex sed scripts** for better maintainability. 5. **Prefer multiple simple commands** over a single complex one for readability. While these one-liners are powerful, complex text processing might be better handled by purpose-built tools for specific formats. ## When to Use Sed vs. Alternative Tools This quick reference helps decide when to use sed versus other text processing tools: **Sed** is best for line-based text transformations and search & replace operations. ```bash # Example use case sed 's/error/warning/g' log.txt ``` **Awk** excels at column-based data processing and complex logic. ```bash # Example use case awk '$3 > 100 {print $1}' data.txt ``` **Grep** is ideal for pattern searching and line filtering. ```bash # Example use case grep -A 3 "ERROR" log.txt ``` **Perl** handles complex regex and multi-line patterns extremely well. ```bash # Example use case perl -0pe 's/start(.*)end/new/s' file.txt ``` **Jq** is specialized for JSON processing. ```bash # Example use case jq '.users[] | .name' data.json ``` ## Portable Sed Commands for Cross-Platform Use When working across different Unix systems, be aware of GNU vs. BSD sed differences. These commands work on both: ```bash # Portable in-place edit (works on Linux and macOS) sed -i.bak 's/old/new/g' file.txt && rm file.txt.bak # Portable line selection (without GNU extensions) sed -n '10,20p' file.txt # Portable extended regex (without -E or -r) sed 's/[0-9]\{3\}-[0-9]\{2\}-[0-9]\{4\}/XXX-XX-XXXX/g' file.txt ``` ## Creating a Sed Command Library Many experienced Unix admins keep a file of useful sed commands. Here's how to start yours: ```bash # Create a sed commands file mkdir -p ~/bin/sed-scripts # Example: Save a useful command cat > ~/bin/sed-scripts/xml-to-json.sed << 'EOF' # Convert simple XML to JSON s/<\([a-zA-Z]*\)>/{\"\1\":/g s/<\/[a-zA-Z]*>/},/g s/>\([^{]*\)\"\1\"