# Filename Extraction: basename to a Production File Pipeline

- URL: https://karandeepsingh.ca/posts/bash-extract-filename-from-path-guide/
- Author: Karandeep Singh
- Published: 2024-04-30
- Updated: 2026-07-16
- Series: Bash for Production
- Tags: bash
- Summary: Work through a log-filename processing exercise from scratch. Start with simple basename, encounter the symlink bug that breaks extraction, fix it, then benchmark all four approaches yourself (basename vs parameter expansion vs awk vs sed).

Log processing is where filename handling quietly breaks. Picture services all writing logs to `/var/log/<service>/<timestamp>-<hostname>.log`, and a job that has to extract just the service name from each path.

This kind of exercise teaches filename extraction through edge cases: filenames with spaces, symlinks, process-spawning overhead, multiple extensions. This article walks through that progression: start simple, hit the bugs, fix them, and end with a benchmark script you can run to see which approach performs best.

![Linux terminal running a program that processes file paths](https://karandeepsingh.ca/images/posts/bash-extract-filename.jpg)

## Step 1: The Simple Approach (basename)

The requirement: given `/var/log/auth-api/2024-03-15-prod01.log`, extract `auth-api`.

The obvious approach is `basename`:

```text
$ path="/var/log/auth-api/2024-03-15-prod01.log"
$ service=$(basename $(dirname "$path"))
$ echo "$service"
auth-api
```

This works. `dirname` gives `/var/log/auth-api`, then `basename` extracts `auth-api`. Simple, readable, done.

But with a dozen services writing logs, you need it to work for all of them. Test across the set:

```text
$ for path in /var/log/*/2024-03-15-*.log; do
>     service=$(basename $(dirname "$path"))
>     echo "$service"
> done
auth-api
payment-svc
user-svc
notification-svc
```

This works in dev. It works in staging. It can still break in production.

## Step 2: The Symlink Deploy Bug

A common failure report looks like this: "Service name extraction broken for auth-api on prod03."

Check the log paths on the affected host:

```text
$ ls -la /var/log/ | grep auth
lrwxrwxrwx  1 root root   28 Mar 15 10:30 auth-api -> /mnt/shared-logs/auth-api
```

Symlink. The log directory is a symlink to shared NFS storage. On most servers, `/var/log/auth-api` is a real directory. On some it is a symlink — and that is enough to break extraction inconsistently across a fleet.

Try the extraction on a symlink:

```text
$ path="/var/log/auth-api/2024-03-15-prod03.log"
$ service=$(basename $(dirname "$path"))
$ echo "$service"
auth-api
```

That worked. Try a more realistic test:

```text
# Create a symlink test
$ mkdir -p /tmp/real-logs/service-a
$ ln -s /tmp/real-logs/service-a /tmp/logs-link
$ path="/tmp/logs-link/2024-03-15.log"

$ service=$(basename $(dirname "$path"))
$ echo "$service"
logs-link
```

There's the bug. `basename` gives the symlink name, not the target. In production, some paths look like:

```
/var/log/auth-api -> /mnt/shared/logs/auth-api
```

The dirname gave `/var/log/auth-api`, basename gave `auth-api`. But for resolved paths:

```
/mnt/shared/logs/auth-api/2024-03-15.log
```

The dirname gave `/mnt/shared/logs/auth-api`, basename gave `auth-api`. Same result.

But the actual problem is more subtle. Suppose the log path on the affected host isn't `/var/log/auth-api/file.log` but the resolved real path: `/mnt/shared/logs/auth-api/file.log`. That happens when something normalizes paths with `readlink -f` before processing:

```bash
path=$(readlink -f "/var/log/auth-api/2024-03-15.log")
# path is now /mnt/shared/logs/auth-api/2024-03-15.log

service=$(basename $(dirname "$path"))
# service is "auth-api" — correct!
```

That case still resolves correctly. So trace what actually breaks. Consider a monitoring script doing:

```bash
for log_file in /var/log/*/2024-03-15-*.log; do
    # log_file here is the symlink path: /var/log/auth-api/2024-03-15-prod03.log
    service=$(basename $(dirname "$log_file"))
    echo "$service"
done
```

That should still work because `log_file` contains the literal path. The breakage shows up in a different shape of the monitoring code:

```bash
find /var/log -name "*.log" | while read -r path; do
    service=$(basename $(dirname "$path"))
    echo "$service"
done
```

And here's the problem. On an affected host:

```bash
find /var/log -name "*.log" -print
```

**Output (truncated):**
```
/var/log/syslog
/var/log/auth-api/2024-03-15-prod03.log
```

But with `-L` (follow symlinks), which can be set globally via `alias find='find -L'` on the affected servers:

```text
$ find -L /var/log -name "*.log" -print
/mnt/shared/logs/auth-api/2024-03-15-prod03.log
```

The symlink got resolved by `find -L`, the path became `/mnt/shared/logs/...`, and the extraction logic expected `/var/log/...`.

## Step 3: The Fix — Use Basename on the Parent Directory Name

The real issue: you need the directory name directly under `/var/log`, not the resolved path. The fix:

```bash
path="/mnt/shared/logs/auth-api/2024-03-15.log"

# Extract the service name from the original symlink location
# by not using readlink and not using find -L
service=$(basename $(dirname "$path"))
```

But if the paths are already resolved because of a global alias, that isn't enough. One option is to remove `alias find='find -L'` from the server configs — but that risks breaking other scripts. A safer option is to change the extraction logic to look for the pattern directly:

```bash
# Match /var/log/<service>/ or /mnt/shared/logs/<service>/
service=$(echo "$path" | sed 's|.*/log[s]*/\([^/]*\)/.*|\1|')
echo "$service"
```

**Test:**
```text
$ echo "/var/log/auth-api/2024-03-15.log" | sed 's|.*/log[s]*/\([^/]*\)/.*|\1|'
$ echo "/mnt/shared/logs/auth-api/2024-03-15.log" | sed 's|.*/log[s]*/\([^/]*\)/.*|\1|'
auth-api
auth-api
```

Fixed. Both paths now extract correctly.

## Step 4: Handling Files with Spaces

This sed fix works — until a service appears with a space in the directory name, e.g. `reporting service`.

Its logs go to `/var/log/reporting service/2024-04-01.log`.

The extraction breaks:

```text
$ path="/var/log/reporting service/2024-04-01.log"
$ service=$(echo "$path" | sed 's|.*/log[s]*/\([^/]*\)/.*|\1|')
$ echo "$service"
reporting service
```

That part works. But a downstream script that consumes the value can still break:

```text
$ service="reporting service"
$ mkdir "/backup/$service"
mkdir: cannot create directory '/backup/reporting': No such file or directory
mkdir: cannot create directory 'service': No such file or directory
```

No quotes around `$service` would split the space into two arguments — a classic bash mistake. But here the quotes are already present, so the real problem is elsewhere. The extraction is piped into `xargs`:

```text
$ echo "/var/log/reporting service/2024-04-01.log" | \
>     sed 's|.*/log[s]*/\([^/]*\)/.*|\1|' | \
>     xargs -I {} mkdir "/backup/{}"
mkdir: cannot create directory '/backup/reporting': No such file or directory
```

`xargs` split on whitespace. The `{}` got replaced with `reporting` only. The solution:

```bash
echo "/var/log/reporting service/2024-04-01.log" | \
    sed 's|.*/log[s]*/\([^/]*\)/.*|\1|' | \
    xargs -d '\n' -I {} mkdir "/backup/{}"
```

`-d '\n'` tells xargs to split on newlines only, not whitespace. Now:

```text
$ ls /backup/
reporting service/
```

Fixed. But this exposed another question: is there a faster way than spawning `sed` for every path?

## Step 5: Performance — basename vs Parameter Expansion vs awk

When a script processes a lot of files, the per-file cost adds up. Not huge, but enough that spawning a process for each filename matters.

Four approaches:

1. **basename (spawn process)**
2. **Parameter expansion (pure bash)**
3. **awk (spawn process)**
4. **sed (already tested)**

Benchmark them yourself with the script below. I won't quote exact timings — they vary too much by machine to be meaningful. What is consistent is the relative ordering: pure-bash parameter expansion avoids the per-call process spawn, so it comes out far ahead of the tools that fork a process per path.

### Test Setup

```bash
#!/bin/bash

# Generate 10,000 sample paths
paths=()
for i in $(seq 1 10000); do
    paths+=("/var/log/service-$((i % 50))/2024-03-15-host$i.log")
done

echo "Testing 10,000 paths..."
```

### Approach 1: basename + dirname

```bash
start=$(date +%s%N)
for path in "${paths[@]}"; do
    service=$(basename $(dirname "$path"))
done
end=$(date +%s%N)
elapsed=$(( (end - start) / 1000000 ))
echo "basename: ${elapsed}ms"
```

This forks a subshell and two processes (`dirname`, `basename`) per path, so expect it to take seconds for 10,000 paths.

### Approach 2: Parameter Expansion

```bash
start=$(date +%s%N)
for path in "${paths[@]}"; do
    service="${path%/*}"     # Remove everything after last /
    service="${service##*/}" # Remove everything before last /
done
end=$(date +%s%N)
elapsed=$(( (end - start) / 1000000 ))
echo "parameter expansion: ${elapsed}ms"
```

This one runs entirely inside the shell — no process spawning at all. On any machine it finishes in a small fraction of the time the other approaches take.

### Approach 3: awk

```bash
start=$(date +%s%N)
for path in "${paths[@]}"; do
    service=$(echo "$path" | awk -F'/' '{print $(NF-1)}')
done
end=$(date +%s%N)
elapsed=$(( (end - start) / 1000000 ))
echo "awk: ${elapsed}ms"
```

A subshell plus an `awk` process per path — in the same slow ballpark as basename, often a bit slower.

### Approach 4: sed

```bash
start=$(date +%s%N)
for path in "${paths[@]}"; do
    service=$(echo "$path" | sed 's|.*/\([^/]*\)/[^/]*$|\1|')
done
end=$(date +%s%N)
elapsed=$(( (end - start) / 1000000 ))
echo "sed: ${elapsed}ms"
```

Same story as awk: a process per path.

### Results Summary

Run the four blocks yourself — the exact milliseconds depend on your machine, but the ordering is consistent every time:

| Method | Process spawns per path | Relative Speed |
|--------|-----------------|----------------|
| Parameter expansion | 0 | fastest by a wide margin |
| basename | subshell + dirname + basename | slow |
| sed | subshell + sed | slow |
| awk | subshell + awk | slow |

**Winner: parameter expansion.** Pure bash, no process spawning, handles spaces correctly when quoted.

The final version of the script:

```bash
for log_file in /var/log/*/202*.log; do
    dir="${log_file%/*}"      # /var/log/auth-api/2024-03-15.log -> /var/log/auth-api
    service="${dir##*/}"      # /var/log/auth-api -> auth-api

    # Process with proper quoting
    mkdir -p "/backup/$service"
    cp "$log_file" "/backup/$service/"
done
```

Parameter expansion deals with spaces, symlinks (as long as you don’t resolve them), and avoids a process spawn per file — the more files you process, the more that matters.

## Step 6: Removing Extensions (The Next Requirement)

Once filename extraction is stable, the next requirement usually follows: strip file extensions.

Input: `2024-03-15-prod01.log`
Output: `2024-03-15-prod01`

### Approach 1: basename with Suffix

```text
$ filename="2024-03-15-prod01.log"
$ base=$(basename "$filename" .log)
$ echo "$base"
2024-03-15-prod01
```

Works. But what if the extension varies? Some files were `.log`, others `.log.gz`.

```bash
basename "2024-03-15-prod01.log.gz" .log.gz  # Works
basename "2024-03-15-prod01.log.gz" .gz      # Gives "2024-03-15-prod01.log"
```

You need to know the exact extension.

### Approach 2: Parameter Expansion

```text
$ filename="2024-03-15-prod01.log"
$ base="${filename%.*}"
$ echo "$base"
2024-03-15-prod01
```

Removes everything after the last `.`. Handles any extension:

```text
$ filename="2024-03-15-prod01.log.gz"
$ base="${filename%.*}"
$ echo "$base"
2024-03-15-prod01.log
```

Removes only `.gz`. To remove all extensions:

```text
$ filename="2024-03-15-prod01.log.gz"
$ base="${filename%%.*}"
$ echo "$base"
2024-03-15-prod01
```

`%%` removes the longest match, so it strips everything after the first `.`. But this breaks filenames with dots:

```text
$ filename="service.v2.log"
$ base="${filename%%.*}"
$ echo "$base"
service
```

Lost `v2`. The correct pattern depends on your naming convention.

For logs that always end with `.log` or `.log.gz`, the correct pattern:

```bash
filename="${filename%.log.gz}"
filename="${filename%.log}"
echo "$filename"
```

Apply both removals in sequence. If `.log.gz` exists, remove it. Otherwise remove `.log`.

### The Bug: Order Matters

Try this:

```text
$ filename="2024-03-15.log"
$ filename="${filename%.log}"
$ filename="${filename%.log.gz}"
$ echo "$filename"
2024-03-15
```

Works. Now reverse the order:

```text
$ filename="2024-03-15.log.gz"
$ filename="${filename%.log.gz}"
$ filename="${filename%.log}"
$ echo "$filename"
2024-03-15
```

Still works. Both orders work because:
- `.log.gz` file: first pattern removes `.log.gz`, second pattern finds no `.log`, does nothing
- `.log` file: first pattern finds no `.log.gz`, does nothing, second pattern removes `.log`

Good. Final version:

```text
# Extract service name and remove extensions
$ path="/var/log/auth-api/2024-03-15-prod01.log.gz"
$ dir="${path%/*}"
$ service="${dir##*/}"
$ filename="${path##*/}"
$ filename="${filename%.log.gz}"
$ filename="${filename%.log}"

$ echo "Service: $service"
$ echo "Base filename: $filename"
Service: auth-api
Base filename: 2024-03-15-prod01
```

## What We Built: A Robust Log-Processing Script

Starting from a simple requirement (extract service names from log paths), we hit the common real-world edge cases:

1. **Simple basename** — works until symlinks show up
2. **Symlink bug** — `find -L` resolved paths, breaking extraction assumptions
3. **Sed fix** — pattern matching worked regardless of symlink resolution
4. **Spaces in filenames** — broke xargs, fixed with `-d '\n'`
5. **Performance** — parameter expansion beats every process-spawning approach by a wide margin
6. **Extension removal** — multiple approaches, parameter expansion won again

The final script:

```bash
#!/bin/bash

# Process all logs, extract service name, archive by service
for log_file in /var/log/*/202*.log*; do
    [ -f "$log_file" ] || continue

    # Extract service name (pure bash, fast)
    dir="${log_file%/*}"
    service="${dir##*/}"

    # Extract and clean filename
    filename="${log_file##*/}"
    filename="${filename%.log.gz}"
    filename="${filename%.log}"

    # Archive (with proper quoting for spaces)
    mkdir -p "/backup/$service"
    cp "$log_file" "/backup/$service/"
done
```

With the fixes in place, this handles spaces, symlinks, and mixed extensions without surprises.

## Cheat Sheet

**Extract directory from path:**
```bash
dir="${path%/*}"              # /var/log/auth-api/file.log → /var/log/auth-api
```

**Extract filename from path:**
```bash
filename="${path##*/}"        # /var/log/auth-api/file.log → file.log
```

**Extract parent directory name:**
```bash
dir="${path%/*}"             # Get directory
service="${dir##*/}"          # Get last component
# /var/log/auth-api/file.log → auth-api
```

**Remove file extension:**
```bash
base="${filename%.*}"         # file.log → file (last extension)
base="${filename%%.*}"        # file.tar.gz → file (all extensions)
```

**Remove specific extension:**
```bash
filename="${filename%.log.gz}"  # Try .log.gz first
filename="${filename%.log}"     # Then try .log
```

**Process spawning comparison (per path processed):**
- Parameter expansion: no processes — fastest by far
- basename: subshell + dirname + basename
- sed: subshell + sed
- awk: subshell + awk

## Key Rules

1. **Use parameter expansion for performance** — no process spawn per file
2. **Quote everything** — `"$variable"` prevents word splitting on spaces
3. **Use `xargs -d '\n'`** when piping filenames with spaces
4. **Don’t use `find -L`** unless you actually need symlink resolution
5. **`%` removes from end, `#` removes from beginning** — `%%` and `##` are longest match
6. **Test with edge cases** — spaces, symlinks, dots in names, multiple extensions

## FAQ

**Q: When should I use basename vs parameter expansion?**
A: Use parameter expansion (`${var##*/}`) for scripts that process many files — it avoids a process spawn per file. Use basename for interactive one-liners where readability matters more than performance.

**Q: How do I handle filenames with spaces?**
A: Always quote variables: `"$filename"`. When using xargs, add `-d '\n'` to split on newlines instead of whitespace.

**Q: What’s the difference between `%` and `%%`?**
A: Both strip from the end, but they differ by how much of a *glob pattern* they match: `%` removes the shortest trailing match, `%%` removes the longest. For a literal suffix like `.log` (no wildcards) there's nothing to match short vs long, so `${var%.log}` and `${var%%.log}` behave identically. The difference only shows up with a pattern: on `file.tar.gz`, `${var%.*}` removes the shortest match (`.gz`, giving `file.tar`), while `${var%%.*}` removes the longest (`.tar.gz`, giving `file`). For trimming a single extension, you usually want `%.*` (shortest).

**Q: Why did symlinks break my script?**
A: `find -L` and `readlink -f` resolve symlinks to real paths. If your extraction logic expects the symlink path, disable symlink resolution or adjust the pattern to work with both.

**Q: Which is faster: awk or sed?**
A: For filename extraction, they’re roughly the same speed (both slow compared to parameter expansion). Both spawn a process per path, and that spawn dominates the runtime. Run the benchmark script above if you want numbers for your own machine.

## Keep Reading

- [Mastering Bash: The Ultimate Guide to Command Line Productivity](https://karandeepsingh.ca/posts/mastering-bash-scripting/) — more bash patterns and productivity techniques
- [Linux Automation: From Cron to a Go Task Runner](https://karandeepsingh.ca/posts/linux-automation-tools/) — apply these filename patterns in automation pipelines
- [Sed Cheat Sheet: 30 Essential One-Liners](https://karandeepsingh.ca/posts/sed-command-cheat-sheet-30-essential-one-liners/) — when you need more power than parameter expansion