# Sed in CI/CD: Safe Patterns for GitHub Actions and Jenkins - URL: https://karandeepsingh.ca/posts/sed-in-ci-cd-pipelines/ - Author: Karandeep Singh - Published: 2026-05-05 - Series: Mastering sed - Tags: bash, sed, cicd - Summary: A safety-first guide to running sed in automated pipelines — idempotency, exit codes, dry-run flags, in-place backup, and patterns that survive multiple runs without corrupting config files. ## The Pipeline Failure Pattern That Should Make You Distrust sed Consider a Jenkins pipeline that promotes release candidates from staging to production, where one step runs a single line of bash: ```bash sed -i 's/replicas: 2/replicas: 6/g' k8s/deployment.yaml ``` It works perfectly for a long time. Then a flaky network call between Jenkins and the artifact registry makes an earlier step fail, someone retries the job from the failed step, and Jenkins re-runs the sed step too. The first run flips `replicas: 2` to `replicas: 6`. The second run searches for `replicas: 2`, finds nothing, and exits 0. A third retry does the same. That part is fine. The real danger appears when the same pattern is copied into a different stage that updates `image: app:1.0` to `image: app:1.1`. The retry semantics are the same, but suppose a hotfix has already bumped the image to `app:1.2`. A retry tries to replace `app:1.0` with `app:1.1` — except `app:1.0` is no longer in the file. Silent no-op. The pipeline goes green. Production rolls forward to the wrong image. Customer transactions start 502'ing. The bug is simple, but it can take most of a working day to track down, mostly because nobody believes sed could "fail successfully." CI is not the terminal. In the terminal you run a sed command, look at the output, and move on. In CI you have re-runs, retry-on-flaky plugins, and webhooks that re-trigger workflows. A sed command that "works" on your laptop will run multiple times across a pipeline's lifecycle. Every single one of those runs has to do the right thing or do nothing — never something different. The patterns below come out of exactly that failure mode, and they apply across Jenkins pipelines, GitHub Actions services, and CodeBuild projects. Six rules, in order of how often they get violated. ## 1. Idempotency: The First Rule Running the same sed command N times against the same file should produce the same result every time. If it doesn't, you don't have a deploy script — you have a coin flip. The naive pattern that breaks in this scenario: ```bash sed -i 's/version: 1/version: 2/' app.yaml ``` This works once. The first run finds `version: 1` and replaces it with `version: 2`. The second run finds nothing and exits 0. If the second run was expected to perform the substitution (because of a retry, or because the file was reset by a previous step), you've silently shipped the wrong config. The fix is to anchor on the field name, not the value: ```bash sed -i -E 's/^version: [0-9]+$/version: 2/' app.yaml ``` Now any value of `version:` becomes `version: 2`. Run it once, twice, or many times — the file always lands in the same state. An `app.yaml` containing `version: 7` becomes `version: 2` on the first run and stays at `version: 2` on every retry. The same logic applies to every "set this field to that value" pattern. Anchor the regex to the key, the line start, or both. Match anything for the value. Replace with the desired value: ```bash # Idempotent: replicas always becomes 6 sed -i -E 's/^([[:space:]]*replicas:)[[:space:]]+[0-9]+/\1 6/' deployment.yaml # Idempotent: image tag always becomes the value of $TAG sed -i -E 's|^([[:space:]]*image:[[:space:]]+app:)[^[:space:]]+|\1'"$TAG"'|' deployment.yaml ``` If you can't write an idempotent sed, you probably want a templating tool (Helm, Kustomize, envsubst) instead. Sed is for surgical edits, not for managing state. ## 2. Exit Codes and `set -e` Don't Save You Most CI shells run with `set -e` so the job fails on the first non-zero exit code. People assume this protects them from sed failures. It does not. Sed returns 0 even when the substitution matched zero lines. There is no `--fail-if-no-match` flag. From sed's point of view, "I read the file, applied your script, found nothing to change" is a successful run. This is the silent failure mode from the scenario above: the retry runs sed against a file where the source pattern no longer exists, sed shrugs and exits 0, and the pipeline carries on. A good pattern to put at the top of every CI step that uses sed: ```bash set -euo pipefail CONFIG=k8s/deployment.yaml VERSION="${GITHUB_REF_NAME#v}" grep -qE '^[[:space:]]*version:' "$CONFIG" || { echo "ERROR: no 'version:' field found in $CONFIG" exit 1 } sed -i -E "s/^([[:space:]]*version:)[[:space:]]+.*/\1 ${VERSION}/" "$CONFIG" ``` The `grep -q` runs first. It exits 0 if the field exists, 1 if it doesn't. With `set -e` (or the explicit `||` clause) the job fails loudly with a useful message before sed ever runs. If the schema of your config file changed and the field you were targeting no longer exists, you find out at the start of the deploy step instead of three steps later when health checks start flapping. A more defensive variant verifies the substitution actually happened: ```bash before=$(sha256sum "$CONFIG" | cut -d' ' -f1) sed -i -E "s/^([[:space:]]*version:)[[:space:]]+.*/\1 ${VERSION}/" "$CONFIG" after=$(sha256sum "$CONFIG" | cut -d' ' -f1) if [ "$before" = "$after" ]; then echo "ERROR: sed did not modify $CONFIG (already at version ${VERSION}?)" # Decide: is this a hard failure or expected on retry? exit 1 fi ``` Whether "no change" is an error depends on context. For a one-shot deploy step, it's a bug. For an idempotent retry, it's expected. Either way, log it, don't swallow it. ## 3. Dry-Run Before Mutation In the terminal, you can run sed, eyeball the output, and decide whether to add `-i`. In CI, the diff between "what I expected" and "what sed produced" lives in a job log nobody reads until the post-mortem. Run the substitution without `-i` first, diff against the original, and emit that diff to the build log: ```bash diff -u "$CONFIG" <(sed -E "s/^([[:space:]]*version:)[[:space:]]+.*/\1 ${VERSION}/" "$CONFIG") || true ``` The `|| true` is important — `diff` returns 1 when files differ, which is the case you actually want. Without it, `set -e` will kill the job. Real example from a GitHub Actions step: ```yaml - name: Show planned config change run: | diff -u helm/values.yaml \ <(sed -E "s|^( tag: ).*|\1${{ github.ref_name }}|" helm/values.yaml) \ || true - name: Apply config change run: | sed -i -E "s|^( tag: ).*|\1${{ github.ref_name }}|" helm/values.yaml ``` A trivial amount of CI time, every deploy. When something goes wrong much later, the diff is right there in the log next to the failing step. That record is worth far more than the seconds it costs. For bigger scripts (multiple sed commands, ranges, deletions), write the planned changes to a temp file and diff that. The build log then shows the final state: ```bash sed -E ' s/^(name:).*/\1 my-app/ s/^(replicas:).*/\1 6/ /^debug:/d ' "$CONFIG" > /tmp/planned.yaml diff -u "$CONFIG" /tmp/planned.yaml || true mv /tmp/planned.yaml "$CONFIG" ``` That `mv` is worth a moment. Writing to a temp file and renaming is atomic on most filesystems; `sed -i` is not — it deletes and recreates. If the runner gets killed mid-write (OOM, spot reclaim, network drop on a remote filesystem), `sed -i` can leave you with a truncated file. Temp-file-and-rename doesn't. ## 4. In-Place with Backup: The GNU vs BSD Trap This one bites every team eventually. GNU sed and BSD sed disagree on what `-i` means. GNU sed (Linux runners, most Docker images): ```bash sed -i 's/foo/bar/' file.txt # works sed -i.bak 's/foo/bar/' file.txt # works, creates file.txt.bak ``` BSD sed (macOS runners, FreeBSD agents): ```bash sed -i 's/foo/bar/' file.txt # ERROR: requires an argument after -i sed -i '' 's/foo/bar/' file.txt # works on BSD, FAILS on GNU sed -i.bak 's/foo/bar/' file.txt # works on BOTH ``` Picture a pipeline running on Linux Jenkins agents. A developer on a MacBook hits the BSD error locally and "fixes" it by adding `-i ''`. The next CI run blows up because GNU sed parses `''` as the script argument. The portable form is `sed -i.bak` with a real backup suffix. It works on GNU sed, BSD sed, and most other implementations. If you don't want the backup, delete it explicitly: ```bash sed -i.bak -E "s/^(version:).*/\1 2/" config.yaml rm -f config.yaml.bak ``` Two lines instead of one, but portable. The backup also makes rollback trivial: ```bash sed -i.bak -E "s/^(version:).*/\1 ${VERSION}/" config.yaml if ! validate_config config.yaml; then mv config.yaml.bak config.yaml exit 1 fi rm -f config.yaml.bak ``` If you want to avoid the in-place mode entirely — a good default in any pipeline that touches a file the build artifact will consume — the portable wrapper writes through a temp file: ```bash safe_sed() { local script="$1" local file="$2" local tmp tmp=$(mktemp) sed -E "$script" "$file" > "$tmp" && mv "$tmp" "$file" } safe_sed 's/^(version:).*/\1 2/' config.yaml ``` This works on every sed in existence, gives you atomic replacement, and never leaves a `.bak` file lying around. ## 5. Variable Interpolation Without Injection The moment you put a CI variable inside a sed command, you've created two problems: a small attack surface, and a much bigger reliability one. The naive pattern: ```bash sed -i "s/version: .*/version: $VERSION/" config.yaml ``` This works when `$VERSION` is `1.2.3`. It breaks when `$VERSION` is `1.2.3/main` (a slash kills the substitution). It does worse things when `$VERSION` contains `&` (sed expands `&` to the entire matched string) or `\1` (sed treats it as a backreference). And if `$VERSION` comes from a PR title, a Git tag from a fork, or a JSON payload — congratulations, you have a sed-injection bug. A common version of this bug: a pipeline pulls a "release notes" string from a webhook and tries to put it into a YAML file: ```bash sed -i "s|notes: .*|notes: $NOTES|" release.yaml ``` Suppose the file already has `notes: 1.0.0` and `$NOTES` is `Bug fix & improvements`. In the replacement, sed expands `&` to the **entire matched (old) line** — `notes: 1.0.0` — not to your new value. The file ends up with `notes: Bug fix notes: 1.0.0 improvements`. The deploy still goes out, but the changelog is unreadable. The `sed_escape_replacement` helper shown below fixes exactly this by escaping `&` (along with `/` and `\`) so it's inserted literally instead of being expanded. Two layers of escaping matter here: the shell layer (your double-quoted string) and the sed layer (the replacement string). For values from a known-safe source like `GITHUB_REF_NAME` for tags matching `vX.Y.Z`, single-quoting and string concatenation is enough: ```bash VERSION="${GITHUB_REF_NAME#v}" sed -i -E 's/^(version:).*/\1 '"${VERSION}"'/' config.yaml ``` The single quotes around `s/^(version:).*/` keep the shell from interpreting the regex specials. The double-quoted `"${VERSION}"` lets the shell substitute. The trailing `/` is back inside single quotes. This works as long as `$VERSION` doesn't contain `/`, `&`, or `\`. For untrusted input — anything from a PR, an external API, a user-controlled file — escape the replacement before it touches sed: ```bash sed_escape_replacement() { # Escape /, &, and \ for the right side of sed s/// printf '%s' "$1" | sed -e 's/[\/&\\]/\\&/g' } NOTES_SAFE=$(sed_escape_replacement "$NOTES") sed -i "s|^notes: .*|notes: ${NOTES_SAFE}|" release.yaml ``` When the value can be wild, switch to a tool designed for the format: `yq` for YAML, `jq` for JSON. Sed is for known-safe substitutions on known-shape files. The other half of this is the pattern delimiter. The default `s/...` uses `/` as the separator, which breaks on URLs and paths. Switch to `|`: ```bash # Bad: slashes in URL break the regex sed -i 's/url: .*/url: https:\/\/api.example.com/' config.yaml # Good sed -i 's|^url: .*|url: https://api.example.com|' config.yaml ``` Default to `|` for any sed command that handles paths, URLs, or arbitrary strings. ## 6. Pipeline-Specific Recipes The patterns above are platform-agnostic. Here are three of the most common recipes. ### GitHub Actions: Helm chart version from a tag When a Git tag like `v2.4.1` triggers a release workflow, push that version into the Helm chart before packaging. ```yaml jobs: package: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Update Helm chart env: V: ${{ github.ref_name }} run: | set -euo pipefail V="${V#v}" grep -qE '^version:' charts/myapp/Chart.yaml grep -qE '^ tag:' charts/myapp/values.yaml diff -u charts/myapp/Chart.yaml \ <(sed -E "s/^(version:).*/\1 ${V}/" charts/myapp/Chart.yaml) || true sed -i.bak -E "s/^(version:).*/\1 ${V}/" charts/myapp/Chart.yaml sed -i.bak -E "s|^( tag:).*|\1 \"${V}\"|" charts/myapp/values.yaml rm -f charts/myapp/Chart.yaml.bak charts/myapp/values.yaml.bak - name: Package run: helm package charts/myapp ``` The `grep -q` precheck catches schema drift. The diff line writes a "before/after" record into the workflow log. `sed -i.bak` with explicit cleanup keeps the step portable across `ubuntu-latest` and `macos-latest` runners. ### Jenkins: per-environment Jenkinsfile parameters A single Jenkinsfile that needs different parameters for `dev`, `staging`, and `prod`. Don't fork the file — rewrite the parameter block in a setup stage. ```groovy stage('Configure') { steps { sh ''' set -euo pipefail CONFIG=deploy/config.yaml grep -qE '^environment:' "$CONFIG" case "${ENV}" in dev) REPLICAS=1; DOMAIN=dev.example.com ;; staging) REPLICAS=2; DOMAIN=staging.example.com ;; prod) REPLICAS=6; DOMAIN=app.example.com ;; *) echo "unknown env: ${ENV}"; exit 1 ;; esac sed -i.bak -E \ -e "s/^(environment:).*/\\1 ${ENV}/" \ -e "s/^(replicas:).*/\\1 ${REPLICAS}/" \ -e "s|^(domain:).*|\\1 ${DOMAIN}|" \ "$CONFIG" rm -f "${CONFIG}.bak" ''' } } ``` Note the doubled backslashes — Groovy strings eat one layer before the shell sees them. The first time you use sed inside a Jenkinsfile this looks like a sed bug. It isn't. Retry-safety matters more in Jenkins than anywhere else, because Jenkins re-runs failed stages by default and the "Replay" button re-runs the whole thing. Every sed in a Jenkinsfile must be idempotent. If it isn't, your second-to-third retry will eat the file. ### CodeBuild: rewriting `appspec.yml` before CodeDeploy In a CodeBuild project that produces a CodeDeploy-compatible artifact, `appspec.yml` often needs the build's image URI baked in. The buildspec uses sed in a `pre_build` phase. ```yaml version: 0.2 env: shell: bash phases: pre_build: commands: - set -euo pipefail - SHORT_SHA="${CODEBUILD_RESOLVED_SOURCE_VERSION:0:7}" - IMAGE_URI="${ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/myapp:${SHORT_SHA}" - grep -qE '^[[:space:]]*image:' deploy/appspec.yml - | diff -u deploy/appspec.yml \ <(sed -E "s|^([[:space:]]*image:[[:space:]]+).*|\1${IMAGE_URI}|" deploy/appspec.yml) || true - | sed -i.bak -E "s|^([[:space:]]*image:[[:space:]]+).*|\1${IMAGE_URI}|" deploy/appspec.yml rm -f deploy/appspec.yml.bak artifacts: files: - deploy/appspec.yml - deploy/scripts/**/* ``` The image URI uses `:` characters, so the sed delimiter is `|`. The short-SHA tag from `CODEBUILD_RESOLVED_SOURCE_VERSION` is idempotent on retry — the same commit produces the same URI, so a re-run of the build phase rewrites the file to the same content. For more on the buildspec format around this sed call, see [Advanced buildspec.yml: Python, Go, ECR Push, EKS Deploy](https://karandeepsingh.ca/posts/aws-codebuild-advanced-buildspec-guide/). ## What the Fix Looks Like The way to harden a pipeline against this failure mode is four specific changes to every step that uses sed: 1. **Precede every sed call with a `grep -q` precheck.** If the field isn't there, the pipeline fails loudly instead of silently no-op'ing. 2. **Anchor every regex to the field name.** No more `s/version: 1/version: 2/` patterns. Always `s/^(version:).*/\1 2/`. 3. **Emit a `diff -u` before any step that mutates a file.** The build logs get slightly bigger. That's fine. 4. **Make every `sed -i` a `sed -i.bak` with an explicit `rm` of the backup.** Portability and a free rollback hook in one change. Under these rules, a change that would previously fail silently fails loudly instead. If a developer renames `version` to `appVersion` in a YAML file, the next pipeline run fails at the `grep -q` step with a clear error message rather than shipping the wrong config. One more rule worth baking into every CI review: if your sed command is more than two substitutions long, or if the file nests more than two levels deep, you should probably be using `yq`, `jq`, or a real templating tool. Sed is a scalpel. The patterns above keep it from slipping. They don't make sed the right tool for every job. If you take one thing from this article: never run `sed -i` in a pipeline without a `grep -q` precheck and a `diff` log line. That alone catches the failure described at the start. ## Related - [Sed Cheat Sheet: 30 One-Liners from Real Production Logs](https://karandeepsingh.ca/posts/sed-command-cheat-sheet-30-essential-one-liners/) - [Sed Gotchas: GNU vs BSD, In-Place Backup, and Safety Patterns](https://karandeepsingh.ca/posts/sed-gotchas-gnu-vs-bsd/) - [Advanced buildspec.yml: Python, Go, ECR Push, EKS Deploy](https://karandeepsingh.ca/posts/aws-codebuild-advanced-buildspec-guide/) - [Sed for Log Analysis: Extract Errors, Filter by Time, Find Patterns](https://karandeepsingh.ca/posts/sed-for-log-analysis/)