Posts on Karandeep Singh - DevOps in Calgary

Sed for Log Analysis: Extract Errors, Filter by Time, Find Patterns

Tue, 05 May 2026 00:00:00 +0000

Practical sed recipes for working with production logs — error extraction, time-range filtering, multi-line stack trace handling, anonymization, and pre-pipeline cleanup before grep/awk.

Sed Gotchas: GNU vs BSD, In-Place Backup, and Safety Patterns

Tue, 05 May 2026 00:00:00 +0000

The sed gotchas every DevOps engineer hits — GNU vs BSD differences, in-place editing dangers, escape traps, and the portable patterns that work on Linux, macOS, and Alpine without modification.

Sed in CI/CD Pipelines: Safe Patterns for GitHub Actions and Jenkins

Tue, 05 May 2026 00:00:00 +0000

A safety-first guide to running sed in automated pipelines — idempotency, exit codes, dry-run flags, in-place backup, and patterns that survive multiple runs without corrupting config files.

Sed Multiline Patterns: How to Match Across Lines

Tue, 05 May 2026 00:00:00 +0000

A walkthrough of sed multiline editing — pattern space, hold space, N/D/P commands, address ranges — built up step by step with real DevOps examples (stack traces, YAML blocks, SQL).

Sed vs Awk vs Grep: When to Use Which (with Decision Matrix)

Tue, 05 May 2026 00:00:00 +0000

A practical comparison of sed, awk, and grep with a decision tree, performance observations on real log data, and ten side-by-side examples of the same task in all three tools.

What I Got Wrong About Kubernetes in 2025

Mon, 04 May 2026 10:00:00 -0600

A year-end retrospective on Kubernetes mistakes from a DevOps engineer. Five things I overinvested in, underinvested in, or simply got wrong while running EKS in 2025.

The DevOps Stack I'd Pick If I Started Over in 2026

Mon, 04 May 2026 08:00:00 -0600

An opinionated DevOps stack pick from an engineer with years of production experience. Each layer is a vote, with what I would avoid and why.

Master tmux: From Terminal Multiplexer to a Go Session Manager

Sat, 21 Feb 2026 14:00:00 +0000

Learn tmux one concept at a time — sessions, windows, panes, configuration — then build a Go CLI tool that reads a YAML file and launches your entire dev environment automatically.

Log Aggregator From Scratch in Go

Mon, 16 Feb 2026 14:00:00 +0000

Build a log aggregator from scratch in Go. Tail files with inotify, survive log rotation, parse syslog and JSON, fan in from multiple sources, and wire up graceful shutdown.

Terraform From Scratch: Provision AWS Infrastructure Step by Step

Mon, 16 Feb 2026 10:00:00 +0000

Learn Terraform with AWS step by step. Start with one S3 bucket, hit real errors, fix them, then build a full VPC + EC2 setup. Each step shows the command, the output, and the mistake.

Nginx Log Analysis: From grep to a Go Log Parser

Sun, 15 Feb 2026 23:00:00 +0000

Nginx log analysis from grep to Go — each step shows the command-line approach first, then builds it in Go. Parse access logs, find slow endpoints, detect error spikes, build a log analysis dashboard.

AWS CLI Automation: From Bash Scripts to Go

Sun, 15 Feb 2026 22:00:00 +0000

AWS automation from CLI to Go code. Each step shows the AWS CLI command first, then builds it in Go. List EC2 instances, manage S3, check costs, build an infrastructure report tool.

Config Templating: From envsubst to Go

Sun, 15 Feb 2026 21:00:00 +0000

Config templating from envsubst to Go: each step shows the shell way first, then builds it in Go. Handle missing variables, add conditionals, build a config generator for dev/staging/prod.

Process Management: From Linux Commands to a Go Supervisor

Sun, 15 Feb 2026 20:00:00 +0000

Process management from command line to Go code. Each step shows the Linux command first, then builds it in Go. Kill processes, send signals, set resource limits, build a process supervisor with auto-restart.

Linux Access Control: From sudo to a Go Security Scanner

Sun, 15 Feb 2026 19:00:00 +0000

Linux access control from command line to Go code. Each step shows the Linux command first, then builds it in Go. Parse sudoers, audit permissions, build a security scanner that flags dangerous configs.

Timezones in Production: From Linux Commands to Go

Sun, 15 Feb 2026 18:00:00 +0000

Timezone handling from command line to Go code — each step shows the Linux command first, then builds it in Go. Hit the DST parsing trap, fix it, normalize log timestamps, end with a multi-zone clock.

CPU Monitoring: From Linux Commands to a Go Dashboard

Sun, 15 Feb 2026 17:00:00 +0000

CPU monitoring from command line to Go code — each step shows the Linux command first, then builds it in Go. Hit the jiffies trap, fix it, add colored bars, end with a live dashboard.

Unix Power Tools Every DevOps Engineer Should Know

Sun, 15 Feb 2026 16:00:00 +0000

Learn Unix commands that actually matter in production: pv for progress bars, xargs for parallel execution, watch for live monitoring, column for readable output. Then build a Go log watcher using the same patterns.

Go + Nginx: Deploy a Go API Behind a Reverse Proxy

Sun, 15 Feb 2026 14:00:00 +0000

Deploy a Go HTTP API behind Nginx. Start with a bare server, hit the port 80 permission trap, proxy through Nginx, lose client IPs, fix header forwarding, add rate limiting, and wire up graceful shutdown with systemd.

Containers From Scratch in Go

Sat, 14 Feb 2026 14:00:00 +0000

Build your own container runtime in Go from scratch. Start by running a command, then add PID isolation, filesystem isolation, hostname, cgroups, one concept at a time.

Go + SQS: Build a Message Queue Processor

Sat, 14 Feb 2026 13:00:00 +0000

Build an SQS message processor in Go step by step. Send messages, hit the visibility timeout trap, add batch operations, and build a production-ready polling loop.

Go + DynamoDB: Build a Simple CRUD App

Sat, 14 Feb 2026 12:00:00 +0000

Build a DynamoDB CRUD app in Go step by step. Start raw, hit real pain points, and end up with clean struct-based operations.

Build a Go CLI Tool for AWS S3

Sat, 14 Feb 2026 11:00:00 +0000

Build a practical Go CLI tool for AWS S3. Start with listing buckets, then add uploads, downloads, and concurrent transfers step by step.

Build and Deploy a Go Lambda Function

Sat, 14 Feb 2026 10:00:00 +0000

Build your first Go Lambda function from scratch. We start simple, hit real errors, fix them, and work our way up to a production-ready API endpoint.

Jenkins Configuration as Code: The Definitive JCasC Guide

Sat, 14 Feb 2026 02:01:00 +0000

Learn how to manage your entire Jenkins setup through version-controlled YAML files using Jenkins Configuration as Code (JCasC), with practical examples for Kubernetes, security, and shared libraries.

Deploy Jenkins on Amazon EKS: A Practical Tutorial

Tue, 24 Jun 2025 20:01:00 +0000

Step-by-step tutorial for deploying Jenkins on Amazon EKS. Learn pods vs deployments and why each matters for your CI/CD setup.

Jenkins LTS vs Weekly: Which Version Should You Use?

Thu, 13 Mar 2025 14:45:00 +0000

A practical guide to Jenkins version selection, covering JVM compatibility, version evolution, agent requirements, and implementation steps.

Linux ls Command: History, Tricks, and Examples

Mon, 10 Mar 2025 14:00:00 +0000

The definitive guide to the Linux ls command. Covers its five-decade history from early Unix to GNU coreutils, core options, task-based examples for real-world scenarios, quirky hidden tricks, and a quick-reference cheat sheet.

Jenkins UserRemoteConfig: Dynamic Git in Pipelines

Tue, 25 Feb 2025 00:00:00 +0000

Unlock advanced Git workflows with Jenkins UserRemoteConfig. Perfect for DevOps engineers managing multi-repository pipelines.

Linux Automation Tools: From Cron to a Custom Go Runner

Wed, 05 Feb 2025 00:00:00 +0000

Master Linux task automation with cron, make, and systemd timers. Then build a complete Go task runner with scheduling, retries, and a status dashboard.

Deployment Automation: From SSH Scripts to a Go Deploy Tool

Tue, 28 Jan 2025 02:01:00 +0000

Master deployment automation with Linux commands and Go. From SSH and rsync basics to building a complete deploy tool with health checks, rollbacks, and multi-server support.

Jenkinsfile with envsubst: Simplifying CI/CD Pipeline Configuration

Tue, 28 Jan 2025 02:01:00 +0000

Discover how combining Jenkinsfile with envsubst creates cleaner, more maintainable CI/CD pipelines through powerful environment variable substitution techniques.

Sed Cheat Sheet: 30 One-Liners from Real Production Logs

Tue, 28 Jan 2025 02:01:00 +0000

Thirty sed one-liners from actual log analysis at production scale. Each pattern solves a real problem (extract errors, clean configs, parse timestamps, transform JSON). Organized by use case with input/output examples and performance notes.

envsubst Security: How to Protect Your CI/CD Pipeline

Sat, 28 Dec 2024 02:01:00 +0000

Explore critical security vulnerabilities, mitigation strategies, and best practices when implementing envsubst for environment variable substitution in your deployment pipelines.

How to Replace Text in Multiple Files with Sed: A Step-by-Step Guide

Sat, 28 Dec 2024 02:01:00 +0000

Learn reliable techniques for finding and replacing text across multiple files using sed, including file selection methods, safety practices, and advanced pattern matching with real-world examples.

Bash for SRE: Google's Reliability in Shell Scripts

Wed, 28 Aug 2024 02:01:00 +0000

Learn to integrate Google’s SRE principles into your Bash scripts with practical functions for implementing SLOs, reducing toil, and building more reliable systems through automation.

Filename Extraction: From basename to a 500K File/Day Pipeline

Tue, 30 Apr 2024 07:00:00 +0000

Build a production log aggregation pipeline from scratch. Start with simple basename, encounter the symlink bug that breaks extraction, fix it, then benchmark all four approaches (basename vs parameter expansion vs awk vs sed) on 500K files.

Sed for JSON Manipulation: Emergency Patterns When jq is Unavailable

Sun, 28 Apr 2024 02:01:00 +0000

Emergency JSON manipulation patterns using sed when jq is unavailable. Based on real production incident requiring urgent config changes on locked-down server. Always use jq for JSON - this is the emergency backup plan.

Mastering NGINX Logs: A Detailed Guide to Configuration and Analysis

Fri, 19 Apr 2024 20:01:00 +0000

A complete guide to NGINX logs. Learn how to configure, manage, and analyze access and error logs to optimize your NGINX server’s performance and troubleshoot issues effectively.

User Management: From Simple useradd to SOC 2 Compliance

Tue, 16 Apr 2024 02:01:00 +0000

Learn user management the way production incidents teach it. Create a user, realize they can access sensitive files, lock it down with groups and permissions, encounter the audit requirement, implement proper access controls. Based on a real SOC 2 compliance project in Calgary.

Advanced Guide to Using the top Command in Linux

Tue, 26 Mar 2024 02:01:00 +0000

The Linux top command is an essential tool for real-time system monitoring, providing a dynamic overview of running processes and their resource consumption. This guide digs deep into the functionalities of the top command, offering users of all skill levels, from novices to seasoned system administrators, a detailed look at how to utilize this tool effectively. Starting with the basics of launching and reading top, the guide progresses through intermediate techniques like sorting and filtering processes, and advances into complex uses such as batch mode operation and remote system monitoring. Practical examples in each section demonstrate how to apply these techniques in real-world scenarios, enhancing understanding and skill. Additionally, the guide addresses common questions through a thorough FAQ section, equipping users with the knowledge to troubleshoot common issues and optimize system performance. The conclusion ties everything together, suggesting further resources for expanding beyond top to other diagnostic tools, thereby providing a rounded approach to system management.

Advanced Guide to Using the vmstat Command in Linux

Sun, 24 Mar 2024 02:01:00 +0000

This article provides an in-depth exploration of vmstat, a powerful tool for monitoring virtual memory and system performance. Starting with basic commands and moving through advanced functionalities, it offers practical examples to demonstrate each option of vmstat. Learn how to combine different options for thorough monitoring, automate tasks with scripting, and interpret outputs to diagnose and address system issues effectively. This guide also includes pro tips and best practices to enhance monitoring strategies, helping both new and experienced users maximize the utility of vmstat for system management and troubleshooting.

AWS Security Audit: From AWS CLI to a Go Security Scanner

Fri, 01 Mar 2024 07:00:00 +0000

Master AWS security auditing with CLI commands and Go. From checking IAM users and security groups to building a complete scanner that finds and reports misconfigurations.

Database Scaling: From 100K to 5M Users in 18 Months

Tue, 09 Jan 2024 02:01:00 +0000

Watch a database scale from a single Postgres instance to a sharded architecture supporting 5M users. Start simple, hit real bottlenecks (connection limits, replica lag, query timeouts), fix each one, measure the improvement. Based on scaling a Calgary SaaS from 100K to 5M users over 18 months.

Docker Log Management: From docker logs to a Go Log Collector

Sun, 10 Sep 2023 20:01:00 +0000

Master Docker container logging with Linux commands and a real Go log collector. From docker logs basics to a streaming aggregator that processes multiple containers.

Advanced Bash Scripting for Automation