Master Jenkins UserRemoteConfig for dynamic Git repository management. Includes Groovy examples, …
Sed for JSON Manipulation: Parsing Without jq in 5 Simple Patterns
Summary
“I need to update this JSON config file, but I can’t install jq on this server.”
If you’ve ever faced this scenario, you know the immediate sense of dread it brings. JSON’s nested structure and strict syntax make it notoriously difficult to manipulate with text-processing tools. Yet in locked-down environments, minimal containers, or legacy systems, installing specialized JSON parsers isn’t always an option.
That’s where sed
comes in—the venerable stream editor available on virtually every Unix-like system. While parsing JSON with regex is generally discouraged (and rightly so), there are specific patterns and techniques that make it viable for common JSON manipulation tasks.
After years of DevOps work in environments where specialized tools weren’t available, I’ve refined a set of reliable sed
patterns for JSON manipulation. In this guide, I’ll share the five most useful patterns I’ve developed, along with validation techniques to ensure your JSON remains valid after modification.
Why Use sed for JSON (And When to Avoid It)
Before diving into specific patterns, let’s address the elephant in the room: sed
is not a JSON parser. Robert Love’s “Linux System Programming” emphasizes that using the right tool for each job is crucial for robust system programming. So why consider using sed
for JSON at all?
Valid reasons include:
- Minimal environments - Containers, embedded systems, or locked-down servers where installing additional tools isn’t possible
- CI/CD pipelines - When adding dependencies complicates your workflow
- Simple, predictable changes - For quick updates to well-known JSON structures
- One-off scripts - When writing a dedicated parser is overkill
- Shell script integration - When your script already uses
sed
for other operations
However, you should avoid sed
for JSON when:
- The JSON structure is highly variable or unpredictable
- You’re dealing with deeply nested structures
- The JSON includes complex array manipulations
- You need to query or transform the JSON structure
- Absolute reliability is critical (e.g., financial systems)
As Brian Ward notes in “How Linux Works”, text processing tools like sed
operate on line-based patterns, while JSON is hierarchical. This fundamental mismatch means you need carefully crafted patterns for reliable results.
Expand your knowledge with A Developer's Playful Tale of JSON Validation & AWS Lambda | App Config
Pattern 1: Updating Simple Key-Value Pairs
The most straightforward JSON modification is updating a simple key-value pair at the root level. Let’s start with this basic JSON file (config.json
):
{
"apiVersion": "v1",
"port": 8080,
"debug": true,
"logLevel": "info",
"database": {
"host": "localhost",
"port": 5432,
"credentials": {
"user": "admin",
"password": "secret123"
}
},
"features": ["authentication", "logging", "metrics"]
}
Replacing String Values
To update a simple string value:
# Update API version
sed -i 's/"apiVersion": "v1"/"apiVersion": "v2"/' config.json
This works by matching the exact key-value string and replacing it. However, this approach can be brittle because it doesn’t account for whitespace variations or the position in the file.
A More Robust Approach
Here’s a more reliable pattern using context and proper regex anchoring:
# Update API version with more robust pattern
sed -i 's/\("apiVersion": \)"v1"/\1"v2"/' config.json
This pattern:
- Uses a capture group
\("apiVersion": \)
to preserve the key part - Only replaces the value portion
"v1"
- Rebuilds the string with the captured key and new value
Updating Numeric Values
For numeric values, the pattern is similar:
# Update port number
sed -i 's/\("port": \)8080/\12000/' config.json
Toggling Boolean Values
For boolean values:
# Toggle debug mode from true to false
sed -i 's/\("debug": \)true/\1false/' config.json
Key Points for Pattern 1
According to “Unix Power Tools”, the most robust sed
substitutions preserve context using capture groups. For JSON manipulation, I’ve found these practices essential:
Deepen your understanding in Revolutionizing DevOps: The Impact of AI and Machine Learning
- Use capture groups to preserve keys - Only replace the value portion
- Be specific with your patterns - Include enough context to prevent false matches
- Use minimal anchoring - Too strict patterns break with whitespace variations
- Test extensively - Verify your changes across formatting variations
Pattern 2: Navigating Nested Objects
JSON’s power comes from nested structures, which present a greater challenge for line-based tools like sed
. Let’s tackle our nested database configuration:
"database": {
"host": "localhost",
"port": 5432,
"credentials": {
"user": "admin",
"password": "secret123"
}
}
Approach 1: Single-Line Replacement for Shallow Nesting
For relatively shallow nesting (1-2 levels), you can use:
# Update database host
sed -i 's/\("database":[^{]*{[^}]*"host": \)"localhost"/\1"db.example.com"/' config.json
This pattern:
- Matches
"database":
followed by any characters up to the opening brace - Continues matching until it finds
"host":
- Captures all of this in a group
- Replaces only the host value
Approach 2: Line-Based Targeting for Deeper Nesting
For deeper nesting, a multi-line approach is more reliable:
# Update database user credentials
sed -i '/"credentials": {/,/}/s/\("user": \)"admin"/\1"dbadmin"/' config.json
This pattern:
- Restricts operations to lines between
"credentials": {
and the next}
- Within that range, performs a substitution to update the user
Approach 3: Recursive Descent for Deep Nesting
For the deepest nested structures, recursive descent with multiple conditions provides the most reliability:
# Update database password with deep nesting targeting
sed -i '/"database": {/,/}/{/"credentials": {/,/}/{s/\("password": \)"secret123"/\1"new-secure-pwd"/}}' config.json
This pattern:
- First restricts to the database object section
- Within that scope, further restricts to the credentials object
- Within that narrowed context, performs the password substitution
Key Points for Pattern 2
When working with nested JSON objects, the “Shell Scripting” guide recommends breaking complex patterns into logical units. For nested JSON, I’ve found:
Explore this further in Navigating the Future: DevOps Trends to Watch in 2023
- Use address ranges - Limit operations to specific sections with
/pattern1/,/pattern2/
- Narrow scope progressively - Apply multiple filters to target deep structures
- Be wary of similar keys - Choose unique context patterns to avoid false matches
- Consider field position - Sometimes targeting by position is more reliable than by name
Pattern 3: Handling JSON Arrays
JSON arrays present unique challenges because their elements lack keys. Let’s work with this features array:
"features": ["authentication", "logging", "metrics"]
Replacing a Specific Array Element
To replace a specific array element, you need to target its exact position:
# Replace "logging" with "advanced-logging"
sed -i 's/\("features": \[\("[^"]*", \)*\)"logging"/\1"advanced-logging"/' config.json
This is quite complex, so let’s break it down:
"features": \[
matches the array opening\("[^"]*", \)*
matches any number of preceding elements with their commas"logging"
targets the specific element\1
preserves everything before the target element
Adding an Element to an Array
Adding an element to the end of an array:
# Add "notifications" to the features array
sed -i 's/\("features": \[.*\)\]/\1, "notifications"]/' config.json
This pattern:
- Captures everything from
"features": [
to just before the closing bracket - Replaces the closing bracket with a comma, the new element, and a new closing bracket
Removing an Element from an Array
Removing an element is perhaps the trickiest operation:
# Remove "metrics" from features array
sed -i 's/\("features": \[.*\), "metrics"\]/\1]/' config.json
# Alternative for middle elements with comma handling
sed -i 's/\("features": \[.*\)"metrics", /\1/; s/\("features": \[.*\), "metrics"/\1/' config.json
The second example shows how to handle both cases (element with trailing comma and element with preceding comma).
Key Points for Pattern 3
Array manipulation requires careful consideration of edge cases. According to Brian Ward’s “How Linux Works”, text processing should account for all possible input variations:
Discover related concepts in A Developer's Playful Tale of JSON Validation & AWS Lambda | App Config
- Consider element position - First, middle, and last elements need different handling
- Watch for commas - Ensure proper comma placement when adding/removing elements
- Use multiple passes if needed - Complex array operations may require multiple sed commands
- Validate afterwards - Array operations are particularly prone to breaking JSON validity
Pattern 4: Deleting Properties and Blocks
Sometimes you need to remove entire properties or blocks from JSON. This pattern handles both simple properties and complex nested structures.
Removing a Simple Property
To delete a simple key-value pair:
# Remove the logLevel property
sed -i '/"logLevel": "[^"]*"/d' config.json
This directly deletes any line containing the logLevel
property.
Removing a Property and Fixing Commas
The previous approach doesn’t handle commas, which can break JSON syntax. Here’s a more robust approach:
# Remove logLevel property and handle trailing comma
sed -i '/"logLevel": "[^"]*",/d; s/,\s*\n\s*}/\n}/' config.json
# Remove logLevel property and handle preceding comma
sed -i 's/,\s*"logLevel": "[^"]*"//; s/\n\s*,/\n/' config.json
Removing a Nested Block
For removing entire nested structures, you need to identify the block boundaries:
# Remove the entire credentials object
sed -i '/"credentials": {/,/}/d' config.json
This removes all lines from the credentials object opening to its closing brace.
Safely Removing Blocks with Comma Handling
To handle commas properly when removing blocks:
# Remove credentials block and handle commas
sed -i -e '/"credentials": {/,/}/{ /,\s*$/{ s/,\s*$//; b; }; /^\s*}/{n;s/^\s*,\s*//;b}; d;}' config.json
This complex pattern:
- Identifies the credentials block boundaries
- If the line before the block ends with a comma, removes it
- If the line after the block starts with a comma, removes it
- Deletes all lines in the block
Key Points for Pattern 4
Removing elements requires particular attention to maintaining valid JSON syntax:
Uncover more details in Mastering sed for Configuration Files: The Definitive Guide to Updating YAML, JSON, TOML, and Properties Files
- Handle commas carefully - Both trailing and leading commas need cleaning up
- Consider whitespace - Account for various indentation patterns
- Multi-step approach - Complex removals may require multiple operations
- Validate structure - Removal operations are high-risk for breaking JSON syntax
Pattern 5: Adding New Properties and Objects
Finally, let’s look at adding entirely new properties or objects to JSON.
Adding a Simple Property
To add a property to the root object:
# Add a new timeout property at the root level
sed -i 's/\([^{]*{[^}]*\)}/\1,\n "timeout": 30\n}/' config.json
This pattern:
- Captures everything from the opening
{
to the last property before the closing}
- Adds a comma, newline, and the new property before the closing brace
Adding a Nested Property
To add a property to a nested object:
# Add sslEnabled property to database object
sed -i '/"database": {/,/}/{s/\([^{]*{[^}]*\)}/\1,\n "sslEnabled": true\n }/}' config.json
This restricts the operation to the database section and then adds the new property.
Adding a Complete Nested Object
To add an entirely new nested object:
# Add a new monitoring object
sed -i 's/\([^{]*{[^}]*\)}/\1,\n "monitoring": {\n "enabled": true,\n "interval": 60\n }\n}/' config.json
This follows the same pattern but adds a multi-line JSON object with proper indentation.
Targeting Array Positions
For more precise positioning, you can target known properties:
# Add property after apiVersion
sed -i '/"apiVersion"/a\ "environment": "production",' config.json
This adds the new property on a new line after the line containing “apiVersion”.
Key Points for Pattern 5
Adding new properties requires careful attention to structure:
Journey deeper into this topic with Mastering sed for Configuration Files: The Definitive Guide to Updating YAML, JSON, TOML, and Properties Files
- Maintain proper indentation - Keep consistent formatting for readability
- Handle commas properly - Ensure commas separate properties correctly
- Consider pretty-printing afterward - Complex additions may benefit from reformatting
- Incremental approach - Add complex structures in multiple steps
Validation Techniques for sed-Modified JSON
After modifying JSON with sed
, validation is crucial. Here are techniques I’ve developed to ensure JSON remains valid:
Basic Syntax Validation
The simplest approach uses Python (available on most systems):
# Validate JSON after modification
function validate_json() {
python3 -c "import json; json.load(open('$1'))" 2>/dev/null
return $?
}
# Example usage
sed -i 's/"debug": true/"debug": false/' config.json
if ! validate_json config.json; then
echo "JSON validation failed!"
# Restore from backup if using sed -i.bak
[ -f config.json.bak ] && mv config.json.bak config.json
exit 1
fi
In-Place Validation with Automatic Rollback
This pattern combines modification and validation in one step:
#!/bin/bash
# Safe JSON editing with automatic rollback
JSON_FILE="config.json"
BACKUP="${JSON_FILE}.bak"
# Create backup
cp "$JSON_FILE" "$BACKUP"
# Perform modification
sed -i 's/"apiVersion": "v1"/"apiVersion": "v2"/' "$JSON_FILE"
# Validate
if ! python3 -c "import json; json.load(open('$JSON_FILE'))" 2>/dev/null; then
echo "JSON validation failed, rolling back changes"
mv "$BACKUP" "$JSON_FILE"
exit 1
else
echo "JSON successfully modified and validated"
# Remove backup if no longer needed
rm "$BACKUP"
fi
Pretty-Printing After Modification
For better readability after complex modifications:
#!/bin/bash
# Modify and pretty-print JSON
JSON_FILE="config.json"
# Create backup
cp "$JSON_FILE" "${JSON_FILE}.bak"
# Perform modification
sed -i 's/"port": 8080/"port": 9090/' "$JSON_FILE"
# Validate and pretty-print
if python3 -c "
import json, sys
with open('$JSON_FILE') as f:
data = json.load(f)
with open('$JSON_FILE', 'w') as f:
json.dump(data, f, indent=2)
sys.exit(0)
" 2>/dev/null; then
echo "JSON successfully modified, validated, and formatted"
else
echo "JSON validation failed, rolling back changes"
mv "${JSON_FILE}.bak" "$JSON_FILE"
exit 1
fi
Structured Validation with Specific Checks
For more advanced validation:
Enrich your learning with A Developer's Playful Tale of JSON Validation & AWS Lambda | App Config
#!/bin/bash
# Validate specific JSON properties after modification
JSON_FILE="config.json"
# Create backup
cp "$JSON_FILE" "${JSON_FILE}.bak"
# Perform modification
sed -i 's/"apiVersion": "v1"/"apiVersion": "v2"/' "$JSON_FILE"
# Validate structure and specific values
if python3 -c "
import json, sys
try:
with open('$JSON_FILE') as f:
data = json.load(f)
# Verify specific properties
assert data['apiVersion'] == 'v2', 'API version not updated correctly'
assert 'port' in data, 'Required property \"port\" is missing'
print('JSON validation successful')
sys.exit(0)
except Exception as e:
print(f'Validation error: {e}')
sys.exit(1)
" 2>/dev/null; then
echo "JSON successfully modified and validated"
else
echo "JSON validation failed, rolling back changes"
mv "${JSON_FILE}.bak" "$JSON_FILE"
exit 1
fi
Real-World Examples: sed in CI/CD Pipelines
Let’s put these patterns into practice with real-world examples from CI/CD environments where specialized JSON tools might not be available.
Gain comprehensive insights from Alternatives to envsubst: Finding the Right Templating Solution for Your CI/CD Pipelines
Example 1: Dynamic Environment Configuration
#!/bin/bash
# Update application.json for different environments
ENV="${1:-dev}"
CONFIG_FILE="application.json"
# Create backup
cp "$CONFIG_FILE" "${CONFIG_FILE}.bak"
case "$ENV" in
dev)
# Development environment settings
sed -i 's/\("apiUrl": \)"[^"]*"/\1"https:\/\/dev-api.example.com"/' "$CONFIG_FILE"
sed -i 's/\("logLevel": \)"[^"]*"/\1"debug"/' "$CONFIG_FILE"
sed -i 's/\("debug": \)[^,]*/\1true/' "$CONFIG_FILE"
sed -i '/"features": \[/,/]/s/\("features": \[\)[^]]*\]/\1"login", "profile", "dev-console"]/' "$CONFIG_FILE"
;;
staging)
# Staging environment settings
sed -i 's/\("apiUrl": \)"[^"]*"/\1"https:\/\/staging-api.example.com"/' "$CONFIG_FILE"
sed -i 's/\("logLevel": \)"[^"]*"/\1"info"/' "$CONFIG_FILE"
sed -i 's/\("debug": \)[^,]*/\1false/' "$CONFIG_FILE"
sed -i '/"features": \[/,/]/s/\("features": \[\)[^]]*\]/\1"login", "profile"]/' "$CONFIG_FILE"
;;
prod)
# Production environment settings
sed -i 's/\("apiUrl": \)"[^"]*"/\1"https:\/\/api.example.com"/' "$CONFIG_FILE"
sed -i 's/\("logLevel": \)"[^"]*"/\1"warn"/' "$CONFIG_FILE"
sed -i 's/\("debug": \)[^,]*/\1false/' "$CONFIG_FILE"
sed -i '/"features": \[/,/]/s/\("features": \[\)[^]]*\]/\1"login", "profile"]/' "$CONFIG_FILE"
# Remove development-only settings
sed -i '/"devTools": {/,/}/d' "$CONFIG_FILE"
sed -i 's/,\s*\n\s*}/\n}/' "$CONFIG_FILE" # Fix trailing commas
;;
*)
echo "Unknown environment: $ENV"
exit 1
;;
esac
# Validate the JSON
if ! python3 -c "import json; json.load(open('$CONFIG_FILE'))" 2>/dev/null; then
echo "JSON validation failed, reverting changes"
mv "${CONFIG_FILE}.bak" "$CONFIG_FILE"
exit 1
fi
echo "Successfully configured application for $ENV environment"
Example 2: Dynamic Kubernetes Resource Configuration
#!/bin/bash
# Update Kubernetes resource limits in deployment.json
RESOURCE_PRESET="${1:-medium}"
DEPLOYMENT_FILE="deployment.json"
# Create backup
cp "$DEPLOYMENT_FILE" "${DEPLOYMENT_FILE}.bak"
# Extract the current resource section for easier targeting
RESOURCE_SECTION=$(grep -A 10 '"resources":' "$DEPLOYMENT_FILE")
case "$RESOURCE_PRESET" in
small)
# Small resource allocation
sed -i '/"resources": {/,/}/{
s/\("cpu": \)"[^"]*"/\1"100m"/
s/\("memory": \)"[^"]*"/\1"128Mi"/
}' "$DEPLOYMENT_FILE"
;;
medium)
# Medium resource allocation
sed -i '/"resources": {/,/}/{
s/\("cpu": \)"[^"]*"/\1"500m"/
s/\("memory": \)"[^"]*"/\1"512Mi"/
}' "$DEPLOYMENT_FILE"
;;
large)
# Large resource allocation
sed -i '/"resources": {/,/}/{
s/\("cpu": \)"[^"]*"/\1"1000m"/
s/\("memory": \)"[^"]*"/\1"1Gi"/
}' "$DEPLOYMENT_FILE"
;;
*)
echo "Unknown resource preset: $RESOURCE_PRESET"
exit 1
;;
esac
# Also update replica count based on preset
case "$RESOURCE_PRESET" in
small)
sed -i 's/\("replicas": \)[0-9]*/\11/' "$DEPLOYMENT_FILE"
;;
medium)
sed -i 's/\("replicas": \)[0-9]*/\13/' "$DEPLOYMENT_FILE"
;;
large)
sed -i 's/\("replicas": \)[0-9]*/\15/' "$DEPLOYMENT_FILE"
;;
esac
# Validate the JSON
if ! python3 -c "import json; json.load(open('$DEPLOYMENT_FILE'))" 2>/dev/null; then
echo "JSON validation failed, reverting changes"
mv "${DEPLOYMENT_FILE}.bak" "$DEPLOYMENT_FILE"
exit 1
fi
echo "Successfully updated deployment resources to $RESOURCE_PRESET preset"
Example 3: Feature Flag Management
#!/bin/bash
# Manage feature flags in config.json
ACTION="$1"
FEATURE="$2"
CONFIG_FILE="config.json"
# Validate input
if [ -z "$ACTION" ] || [ -z "$FEATURE" ]; then
echo "Usage: $0 [enable|disable|add|remove] feature_name"
exit 1
fi
# Create backup
cp "$CONFIG_FILE" "${CONFIG_FILE}.bak"
# Check if features section exists
if ! grep -q '"features":' "$CONFIG_FILE"; then
echo "Error: Features section not found in $CONFIG_FILE"
exit 1
fi
case "$ACTION" in
enable)
# Enable an existing feature
sed -i '/"features": {/,/}/{s/\("'"$FEATURE"'": \)false/\1true/}' "$CONFIG_FILE"
;;
disable)
# Disable an existing feature
sed -i '/"features": {/,/}/{s/\("'"$FEATURE"'": \)true/\1false/}' "$CONFIG_FILE"
;;
add)
# Add a new feature (enabled by default)
sed -i '/"features": {/,/}/{/}/i\ "'"$FEATURE"'": true,}' "$CONFIG_FILE"
;;
remove)
# Remove a feature entirely
sed -i '/"features": {/,/}/{/\s*"'"$FEATURE"'": [^,]*,\?/d}' "$CONFIG_FILE"
# Fix trailing commas if needed
sed -i '/"features": {/,/}/s/,\s*}/\n }/' "$CONFIG_FILE"
;;
*)
echo "Unknown action: $ACTION (use enable, disable, add, or remove)"
exit 1
;;
esac
# Validate the JSON
if ! python3 -c "import json; json.load(open('$CONFIG_FILE'))" 2>/dev/null; then
echo "JSON validation failed, reverting changes"
mv "${CONFIG_FILE}.bak" "$CONFIG_FILE"
exit 1
fi
echo "Successfully $ACTION""d feature '$FEATURE'"
Best Practices for JSON Manipulation with sed
After years of working with sed
for JSON manipulation, I’ve established these best practices:
1. Always Create Backups
# Always use -i with a suffix for in-place editing
sed -i.bak 's/"debug": true/"debug": false/' config.json
As Robert Love emphasizes in “Linux System Programming”, data integrity requires defense-in-depth strategies.
2. Validate After Every Change
# Simple validation function
validate_json() {
python3 -c "import json; json.load(open('$1'))" 2>/dev/null
return $?
}
# Use with error handling
sed -i.bak 's/"debug": true/"debug": false/' config.json
if ! validate_json config.json; then
echo "Invalid JSON after modification"
mv config.json.bak config.json
exit 1
fi
3. Use Capture Groups for Context
# Bad: Fragile pattern without context
sed -i 's/"v1"/"v2"/' config.json # Might replace wrong values!
# Good: Captures key context for precision
sed -i 's/\("apiVersion": \)"v1"/\1"v2"/' config.json
4. Break Complex Operations into Steps
# Update multiple values in stages with validation between steps
sed -i.bak1 's/"port": 8080/"port": 9090/' config.json
validate_json config.json || { mv config.json.bak1 config.json; exit 1; }
sed -i.bak2 's/"debug": true/"debug": false/' config.json
validate_json config.json || { mv config.json.bak2 config.json; exit 1; }
5. Consider Formatting Variations
# More flexible pattern that handles whitespace variations
sed -i 's/\("port"[[:space:]]*:[[:space:]]*\)8080/\19090/' config.json
According to Brian Ward in “How Linux Works”, robust pattern matching accounts for all valid input variations.
6. Document Complex Patterns
# Well-documented complex sed command
sed -i '
# Target the database credentials block
/"credentials": {/,/}/{
# Replace the password with a new value
s/\("password": \)"[^"]*"/\1"new-secure-password"/
}' config.json
7. Test on Representative Samples
Always test your patterns on sample data that represents all formatting variations you might encounter in production.
8. Consider Function Libraries
Build reusable functions for common JSON operations:
Master this concept through A Developer's Playful Tale of JSON Validation & AWS Lambda | App Config
# Function library for JSON manipulation with sed
update_json_string_property() {
local file="$1"
local property="$2"
local value="$3"
sed -i.bak "s/\\(\"$property\": \\)\"[^\"]*\"/\\1\"$value\"/" "$file"
if ! validate_json "$file"; then
mv "$file.bak" "$file"
return 1
fi
return 0
}
# Usage
update_json_string_property "config.json" "apiVersion" "v2"
Limitations and Alternatives
While these patterns are effective for many scenarios, there are limitations to using sed
for JSON manipulation:
When to Use Other Tools
- Complex queries - When you need to extract data based on complex criteria
- Deeply nested structures - When targeting elements requires traversing many layers
- Large-scale transformations - When performing many operations that might compound errors
- Array manipulations - When operations go beyond simple replacements
Lightweight Alternatives to jq
If you can’t use jq
but need more robust JSON handling, consider:
Python one-liners - For systems with Python but no dedicated JSON tools
python3 -c " import json, sys; data = json.load(open('config.json')); data['apiVersion'] = 'v2'; json.dump(data, open('config.json', 'w'), indent=2) "
Node.js one-liners - For systems with Node.js installed
node -e " const fs = require('fs'); const data = JSON.parse(fs.readFileSync('config.json')); data.apiVersion = 'v2'; fs.writeFileSync('config.json', JSON.stringify(data, null, 2)); "
grep/cut/awk combinations - For extremely minimal environments
Delve into specifics at Alternatives to envsubst: Finding the Right Templating Solution for Your CI/CD Pipelines
# Extract a value using grep/cut (very primitive approach) grep '"apiVersion"' config.json | cut -d'"' -f4
Conclusion: Mastering sed for JSON Manipulation
While specialized JSON tools like jq
are preferred when available, sed
can be a powerful alternative in constrained environments. The five patterns we’ve explored provide a framework for reliably manipulating JSON using standard Unix tools:
- Updating simple key-value pairs - Direct value replacements with context preservation
- Navigating nested objects - Targeting specific sections with address ranges
- Handling JSON arrays - Strategies for list manipulation with comma handling
- Deleting properties and blocks - Removing elements while maintaining syntax
- Adding new properties - Injecting configuration with proper formatting
As with any advanced technique, practice and careful testing are essential. The validation strategies outlined here should be incorporated into your workflows to ensure your JSON manipulations maintain data integrity.
Remember Robert Love’s advice from “Linux System Programming”: use the right tool for the job. When you can install proper JSON parsers, do so. But when constraints require using sed
, these patterns will help you navigate JSON manipulation with confidence.
Have you developed your own techniques for JSON manipulation with sed
? I’d love to hear about your approaches and challenges in the comments below!
Similar Articles
Related Content
More from devops
Explore how OpenAI transforms development workflows, empowering developers and DevOps teams with …
Discover the best Linux automation tools like Ansible, Terraform, and Docker. Learn how to automate …
You Might Also Like
Master text processing with this comprehensive sed command cheat sheet. From basic substitutions to …
Master the art of replacing text across multiple files with sed. This step-by-step guide covers …
Discover how to implement Google's Site Reliability Engineering (SRE) principles using Bash scripts. …
Knowledge Quiz
Test your general knowledge with this quick quiz!
The quiz consists of 5 multiple-choice questions.
Take as much time as you need.
Your score will be shown at the end.
Question 1 of 5
Quiz Complete!
Your score: 0 out of 5
Loading next question...
Contents
- Why Use sed for JSON (And When to Avoid It)
- Pattern 1: Updating Simple Key-Value Pairs
- Pattern 2: Navigating Nested Objects
- Pattern 3: Handling JSON Arrays
- Pattern 4: Deleting Properties and Blocks
- Pattern 5: Adding New Properties and Objects
- Validation Techniques for sed-Modified JSON
- Real-World Examples: sed in CI/CD Pipelines
- Best Practices for JSON Manipulation with sed
- Limitations and Alternatives
- Conclusion: Mastering sed for JSON Manipulation