Sed in CI/CD: Safe Patterns for GitHub Actions and Jenkins Sed in CI/CD: Safe Patterns for GitHub Actions and Jenkins

The Pipeline Failure Pattern That Should Make You Distrust sed

Consider a Jenkins pipeline that promotes release candidates from staging to production, where one step runs a single line of bash:

sed -i 's/replicas: 2/replicas: 6/g' k8s/deployment.yaml

It works perfectly for a long time. Then a flaky network call between Jenkins and the artifact registry makes an earlier step fail, someone retries the job from the failed step, and Jenkins re-runs the sed step too. The first run flips replicas: 2 to replicas: 6. The second run searches for replicas: 2, finds nothing, and exits 0. A third retry does the same.

That part is fine. The real danger appears when the same pattern is copied into a different stage that updates image: app:1.0 to image: app:1.1. The retry semantics are the same, but suppose a hotfix has already bumped the image to app:1.2. A retry tries to replace app:1.0 with app:1.1 — except app:1.0 is no longer in the file. Silent no-op. The pipeline goes green. Production rolls forward to the wrong image. Customer transactions start 502’ing.

The bug is simple, but it can take most of a working day to track down, mostly because nobody believes sed could “fail successfully.”

CI is not the terminal. In the terminal you run a sed command, look at the output, and move on. In CI you have re-runs, retry-on-flaky plugins, and webhooks that re-trigger workflows. A sed command that “works” on your laptop will run multiple times across a pipeline’s lifecycle. Every single one of those runs has to do the right thing or do nothing — never something different.

The patterns below come out of exactly that failure mode, and they apply across Jenkins pipelines, GitHub Actions services, and CodeBuild projects. Six rules, in order of how often they get violated.

1. Idempotency: The First Rule

Running the same sed command N times against the same file should produce the same result every time. If it doesn’t, you don’t have a deploy script — you have a coin flip.

The naive pattern that breaks in this scenario:

sed -i 's/version: 1/version: 2/' app.yaml

This works once. The first run finds version: 1 and replaces it with version: 2. The second run finds nothing and exits 0. If the second run was expected to perform the substitution (because of a retry, or because the file was reset by a previous step), you’ve silently shipped the wrong config.

The fix is to anchor on the field name, not the value:

sed -i -E 's/^version: [0-9]+$/version: 2/' app.yaml

Now any value of version: becomes version: 2. Run it once, twice, or many times — the file always lands in the same state. An app.yaml containing version: 7 becomes version: 2 on the first run and stays at version: 2 on every retry.

The same logic applies to every “set this field to that value” pattern. Anchor the regex to the key, the line start, or both. Match anything for the value. Replace with the desired value:

# Idempotent: replicas always becomes 6
sed -i -E 's/^([[:space:]]*replicas:)[[:space:]]+[0-9]+/\1 6/' deployment.yaml

# Idempotent: image tag always becomes the value of $TAG
sed -i -E 's|^([[:space:]]*image:[[:space:]]+app:)[^[:space:]]+|\1'"$TAG"'|' deployment.yaml

If you can’t write an idempotent sed, you probably want a templating tool (Helm, Kustomize, envsubst) instead. Sed is for surgical edits, not for managing state.

2. Exit Codes and set -e Don’t Save You

Most CI shells run with set -e so the job fails on the first non-zero exit code. People assume this protects them from sed failures. It does not.

Sed returns 0 even when the substitution matched zero lines. There is no --fail-if-no-match flag. From sed’s point of view, “I read the file, applied your script, found nothing to change” is a successful run.

This is the silent failure mode from the scenario above: the retry runs sed against a file where the source pattern no longer exists, sed shrugs and exits 0, and the pipeline carries on.

A good pattern to put at the top of every CI step that uses sed:

set -euo pipefail

CONFIG=k8s/deployment.yaml
VERSION="${GITHUB_REF_NAME#v}"

grep -qE '^[[:space:]]*version:' "$CONFIG" || {
    echo "ERROR: no 'version:' field found in $CONFIG"
    exit 1
}

sed -i -E "s/^([[:space:]]*version:)[[:space:]]+.*/\1 ${VERSION}/" "$CONFIG"

The grep -q runs first. It exits 0 if the field exists, 1 if it doesn’t. With set -e (or the explicit || clause) the job fails loudly with a useful message before sed ever runs. If the schema of your config file changed and the field you were targeting no longer exists, you find out at the start of the deploy step instead of three steps later when health checks start flapping.

A more defensive variant verifies the substitution actually happened:

before=$(sha256sum "$CONFIG" | cut -d' ' -f1)
sed -i -E "s/^([[:space:]]*version:)[[:space:]]+.*/\1 ${VERSION}/" "$CONFIG"
after=$(sha256sum "$CONFIG" | cut -d' ' -f1)

if [ "$before" = "$after" ]; then
    echo "ERROR: sed did not modify $CONFIG (already at version ${VERSION}?)"
    # Decide: is this a hard failure or expected on retry?
    exit 1
fi

Whether “no change” is an error depends on context. For a one-shot deploy step, it’s a bug. For an idempotent retry, it’s expected. Either way, log it, don’t swallow it.

3. Dry-Run Before Mutation

In the terminal, you can run sed, eyeball the output, and decide whether to add -i. In CI, the diff between “what I expected” and “what sed produced” lives in a job log nobody reads until the post-mortem.

Run the substitution without -i first, diff against the original, and emit that diff to the build log:

diff -u "$CONFIG" <(sed -E "s/^([[:space:]]*version:)[[:space:]]+.*/\1 ${VERSION}/" "$CONFIG") || true

The || true is important — diff returns 1 when files differ, which is the case you actually want. Without it, set -e will kill the job.

Real example from a GitHub Actions step:

- name: Show planned config change
  run: |
    diff -u helm/values.yaml \
      <(sed -E "s|^(  tag: ).*|\1${{ github.ref_name }}|" helm/values.yaml) \
      || true

- name: Apply config change
  run: |
    sed -i -E "s|^(  tag: ).*|\1${{ github.ref_name }}|" helm/values.yaml

A trivial amount of CI time, every deploy. When something goes wrong much later, the diff is right there in the log next to the failing step. That record is worth far more than the seconds it costs.

For bigger scripts (multiple sed commands, ranges, deletions), write the planned changes to a temp file and diff that. The build log then shows the final state:

sed -E '
    s/^(name:).*/\1 my-app/
    s/^(replicas:).*/\1 6/
    /^debug:/d
' "$CONFIG" > /tmp/planned.yaml

diff -u "$CONFIG" /tmp/planned.yaml || true
mv /tmp/planned.yaml "$CONFIG"

That mv is worth a moment. Writing to a temp file and renaming is atomic on most filesystems; sed -i is not — it deletes and recreates. If the runner gets killed mid-write (OOM, spot reclaim, network drop on a remote filesystem), sed -i can leave you with a truncated file. Temp-file-and-rename doesn’t.

4. In-Place with Backup: The GNU vs BSD Trap

This one bites every team eventually. GNU sed and BSD sed disagree on what -i means.

GNU sed (Linux runners, most Docker images):

sed -i 's/foo/bar/' file.txt        # works
sed -i.bak 's/foo/bar/' file.txt    # works, creates file.txt.bak

BSD sed (macOS runners, FreeBSD agents):

sed -i 's/foo/bar/' file.txt        # ERROR: requires an argument after -i
sed -i '' 's/foo/bar/' file.txt     # works on BSD, FAILS on GNU
sed -i.bak 's/foo/bar/' file.txt    # works on BOTH

Picture a pipeline running on Linux Jenkins agents. A developer on a MacBook hits the BSD error locally and “fixes” it by adding -i ''. The next CI run blows up because GNU sed parses '' as the script argument.

The portable form is sed -i.bak with a real backup suffix. It works on GNU sed, BSD sed, and most other implementations. If you don’t want the backup, delete it explicitly:

sed -i.bak -E "s/^(version:).*/\1 2/" config.yaml
rm -f config.yaml.bak

Two lines instead of one, but portable. The backup also makes rollback trivial:

sed -i.bak -E "s/^(version:).*/\1 ${VERSION}/" config.yaml
if ! validate_config config.yaml; then
    mv config.yaml.bak config.yaml
    exit 1
fi
rm -f config.yaml.bak

If you want to avoid the in-place mode entirely — a good default in any pipeline that touches a file the build artifact will consume — the portable wrapper writes through a temp file:

safe_sed() {
    local script="$1"
    local file="$2"
    local tmp
    tmp=$(mktemp)
    sed -E "$script" "$file" > "$tmp" && mv "$tmp" "$file"
}

safe_sed 's/^(version:).*/\1 2/' config.yaml

This works on every sed in existence, gives you atomic replacement, and never leaves a .bak file lying around.

5. Variable Interpolation Without Injection

The moment you put a CI variable inside a sed command, you’ve created two problems: a small attack surface, and a much bigger reliability one.

The naive pattern:

sed -i "s/version: .*/version: $VERSION/" config.yaml

This works when $VERSION is 1.2.3. It breaks when $VERSION is 1.2.3/main (a slash kills the substitution). It does worse things when $VERSION contains & (sed expands & to the entire matched string) or \1 (sed treats it as a backreference). And if $VERSION comes from a PR title, a Git tag from a fork, or a JSON payload — congratulations, you have a sed-injection bug.

A common version of this bug: a pipeline pulls a “release notes” string from a webhook and tries to put it into a YAML file:

sed -i "s|notes: .*|notes: $NOTES|" release.yaml

Suppose the file already has notes: 1.0.0 and $NOTES is Bug fix & improvements. In the replacement, sed expands & to the entire matched (old) line — notes: 1.0.0 — not to your new value. The file ends up with notes: Bug fix notes: 1.0.0 improvements. The deploy still goes out, but the changelog is unreadable. The sed_escape_replacement helper shown below fixes exactly this by escaping & (along with / and \) so it’s inserted literally instead of being expanded.

Two layers of escaping matter here: the shell layer (your double-quoted string) and the sed layer (the replacement string). For values from a known-safe source like GITHUB_REF_NAME for tags matching vX.Y.Z, single-quoting and string concatenation is enough:

VERSION="${GITHUB_REF_NAME#v}"
sed -i -E 's/^(version:).*/\1 '"${VERSION}"'/' config.yaml

The single quotes around s/^(version:).*/ keep the shell from interpreting the regex specials. The double-quoted "${VERSION}" lets the shell substitute. The trailing / is back inside single quotes. This works as long as $VERSION doesn’t contain /, &, or \.

For untrusted input — anything from a PR, an external API, a user-controlled file — escape the replacement before it touches sed:

sed_escape_replacement() {
    # Escape /, &, and \ for the right side of sed s///
    printf '%s' "$1" | sed -e 's/[\/&\\]/\\&/g'
}

NOTES_SAFE=$(sed_escape_replacement "$NOTES")
sed -i "s|^notes: .*|notes: ${NOTES_SAFE}|" release.yaml

When the value can be wild, switch to a tool designed for the format: yq for YAML, jq for JSON. Sed is for known-safe substitutions on known-shape files.

The other half of this is the pattern delimiter. The default s/... uses / as the separator, which breaks on URLs and paths. Switch to |:

# Bad: slashes in URL break the regex
sed -i 's/url: .*/url: https:\/\/api.example.com/' config.yaml

# Good
sed -i 's|^url: .*|url: https://api.example.com|' config.yaml

Default to | for any sed command that handles paths, URLs, or arbitrary strings.

6. Pipeline-Specific Recipes

The patterns above are platform-agnostic. Here are three of the most common recipes.

GitHub Actions: Helm chart version from a tag

When a Git tag like v2.4.1 triggers a release workflow, push that version into the Helm chart before packaging.

jobs:
  package:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Update Helm chart
        env:
          V: ${{ github.ref_name }}
        run: |
          set -euo pipefail
          V="${V#v}"

          grep -qE '^version:' charts/myapp/Chart.yaml
          grep -qE '^  tag:' charts/myapp/values.yaml

          diff -u charts/myapp/Chart.yaml \
            <(sed -E "s/^(version:).*/\1 ${V}/" charts/myapp/Chart.yaml) || true

          sed -i.bak -E "s/^(version:).*/\1 ${V}/" charts/myapp/Chart.yaml
          sed -i.bak -E "s|^(  tag:).*|\1 \"${V}\"|" charts/myapp/values.yaml
          rm -f charts/myapp/Chart.yaml.bak charts/myapp/values.yaml.bak

      - name: Package
        run: helm package charts/myapp

The grep -q precheck catches schema drift. The diff line writes a “before/after” record into the workflow log. sed -i.bak with explicit cleanup keeps the step portable across ubuntu-latest and macos-latest runners.

Jenkins: per-environment Jenkinsfile parameters

A single Jenkinsfile that needs different parameters for dev, staging, and prod. Don’t fork the file — rewrite the parameter block in a setup stage.

stage('Configure') {
    steps {
        sh '''
            set -euo pipefail
            CONFIG=deploy/config.yaml

            grep -qE '^environment:' "$CONFIG"

            case "${ENV}" in
              dev)     REPLICAS=1; DOMAIN=dev.example.com ;;
              staging) REPLICAS=2; DOMAIN=staging.example.com ;;
              prod)    REPLICAS=6; DOMAIN=app.example.com ;;
              *)       echo "unknown env: ${ENV}"; exit 1 ;;
            esac

            sed -i.bak -E \
              -e "s/^(environment:).*/\\1 ${ENV}/" \
              -e "s/^(replicas:).*/\\1 ${REPLICAS}/" \
              -e "s|^(domain:).*|\\1 ${DOMAIN}|" \
              "$CONFIG"
            rm -f "${CONFIG}.bak"
        '''
    }
}

Note the doubled backslashes — Groovy strings eat one layer before the shell sees them. The first time you use sed inside a Jenkinsfile this looks like a sed bug. It isn’t.

Retry-safety matters more in Jenkins than anywhere else, because Jenkins re-runs failed stages by default and the “Replay” button re-runs the whole thing. Every sed in a Jenkinsfile must be idempotent. If it isn’t, your second-to-third retry will eat the file.

CodeBuild: rewriting appspec.yml before CodeDeploy

In a CodeBuild project that produces a CodeDeploy-compatible artifact, appspec.yml often needs the build’s image URI baked in. The buildspec uses sed in a pre_build phase.

version: 0.2
env:
  shell: bash
phases:
  pre_build:
    commands:
      - set -euo pipefail
      - SHORT_SHA="${CODEBUILD_RESOLVED_SOURCE_VERSION:0:7}"
      - IMAGE_URI="${ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/myapp:${SHORT_SHA}"
      - grep -qE '^[[:space:]]*image:' deploy/appspec.yml
      - |
        diff -u deploy/appspec.yml \
          <(sed -E "s|^([[:space:]]*image:[[:space:]]+).*|\1${IMAGE_URI}|" deploy/appspec.yml) || true
      - |
        sed -i.bak -E "s|^([[:space:]]*image:[[:space:]]+).*|\1${IMAGE_URI}|" deploy/appspec.yml
        rm -f deploy/appspec.yml.bak
artifacts:
  files:
    - deploy/appspec.yml
    - deploy/scripts/**/*

The image URI uses : characters, so the sed delimiter is |. The short-SHA tag from CODEBUILD_RESOLVED_SOURCE_VERSION is idempotent on retry — the same commit produces the same URI, so a re-run of the build phase rewrites the file to the same content.

For more on the buildspec format around this sed call, see Advanced buildspec.yml: Python, Go, ECR Push, EKS Deploy.

What the Fix Looks Like

The way to harden a pipeline against this failure mode is four specific changes to every step that uses sed:

1. Precede every sed call with a grep -q precheck. If the field isn’t there, the pipeline fails loudly instead of silently no-op’ing.
2. Anchor every regex to the field name. No more s/version: 1/version: 2/ patterns. Always s/^(version:).*/\1 2/.
3. Emit a diff -u before any step that mutates a file. The build logs get slightly bigger. That’s fine.
4. Make every sed -i a sed -i.bak with an explicit rm of the backup. Portability and a free rollback hook in one change.

Under these rules, a change that would previously fail silently fails loudly instead. If a developer renames version to appVersion in a YAML file, the next pipeline run fails at the grep -q step with a clear error message rather than shipping the wrong config.

One more rule worth baking into every CI review: if your sed command is more than two substitutions long, or if the file nests more than two levels deep, you should probably be using yq, jq, or a real templating tool. Sed is a scalpel. The patterns above keep it from slipping. They don’t make sed the right tool for every job.

If you take one thing from this article: never run sed -i in a pipeline without a grep -q precheck and a diff log line. That alone catches the failure described at the start.

Related

• Sed Cheat Sheet: 30 One-Liners from Real Production Logs
• Sed Gotchas: GNU vs BSD, In-Place Backup, and Safety Patterns
• Advanced buildspec.yml: Python, Go, ECR Push, EKS Deploy
• Sed for Log Analysis: Extract Errors, Filter by Time, Find Patterns