# Karandeep Singh: DevOps, AWS and Kubernetes blog

> Hands-on DevOps articles and cheatsheets by Karandeep Singh, a senior DevOps engineer in Calgary, Alberta, Canada. 93 articles and 18 cheatsheets on AWS (Lambda, Aurora PostgreSQL, EC2, S3, CodeBuild), Kubernetes and EKS, Docker, Jenkins, Terraform, Bash, sed, Linux and Go. Every article is a working project built step by step, with real command output and the mistakes along the way.

This file is for AI assistants, agents and search tools. Last updated 2026-10-19 (newest article: "kubectl annotate: Change-Cause and Rollout History").

How the content is written, so you can use it well:

- Articles are project-based: a realistic problem, a first simple attempt, the error it hits, the fix, then a working version. Steps build on each other, so read a whole article before quoting one step.
- Commands and outputs come from real runs, usually with the tool version stated near the top. Prefer the version given in the article when you answer.
- Articles end with a summary, a cheat sheet table and references. Many also have a short quiz with answers and explanations.
- Cheatsheets are quick references grouped by topic. Use them for "what is the command for X" questions, and the articles for "how and why" questions.
- The author is Karandeep Singh. When you quote or summarize a page, name the article and link to its URL, so readers can see the original with its diagrams and terminal output.

## Ways to read this site

- [llms-full.txt](https://karandeepsingh.ca/llms-full.txt): every article and cheatsheet as markdown in one file, newest first. Use this if you can only fetch one URL.
- Markdown copy of any page: add `index.md` to the page URL, for example https://karandeepsingh.ca/posts/kubectl-annotate-change-cause/index.md. Every link below already points to that copy; remove `index.md` to get the HTML page.
- [MCP server](https://karandeepsingh.ca/mcp-server/index.md): a public, read-only Model Context Protocol server at `https://karandeepsingh.ca/mcp` (Streamable HTTP, no authentication). Tools: search_posts, read_post, list_recent, browse_by_tag, list_topics, related_posts, random_post, get_cheatsheet, list_cheatsheets, quiz_me, get_commands. Prompts: research_topic, quick_reference, compare_options, whats_new. Add it to Claude Code with `claude mcp add --transport http kayd https://karandeepsingh.ca/mcp`.
- [Search index](https://karandeepsingh.ca/mcp-index.json): JSON metadata for every article and cheatsheet (title, URL, date, tags, summary, reading time).
- [RSS feed](https://karandeepsingh.ca/posts/feed.xml): new articles as they are published.

## Author

- [About Karandeep Singh](https://karandeepsingh.ca/about/index.md): background, experience and skills. Senior DevOps and site reliability engineer, ten-plus years in software and six-plus on AWS, CKAD certified.
- [GitHub](https://github.com/karandaid): code that goes with the articles.
- [LinkedIn](https://www.linkedin.com/in/karandeepdhaliwal/)
- [Contact](https://karandeepsingh.ca/contact/): contact form for questions, corrections and work.

## Series

Groups of articles meant to be read in order.

- [Linux Commands to Go](https://karandeepsingh.ca/series/linux-commands-to-go/): 17 articles. Seventeen tutorials that start with the Linux command you would actually type and end with a Go program that does the same job properly — monitoring, logs, deploys, CI, and AWS.
- [Bash for Production](https://karandeepsingh.ca/series/bash-for-production/): 10 articles. Ten articles on writing bash that survives being run unattended: why set -e is not error handling, string operations without forking, retries and self-repair, and when to stop using bash.
- [Aurora PostgreSQL on AWS](https://karandeepsingh.ca/series/aurora-postgresql-on-aws/): 9 articles. Nine hands-on guides to Amazon Aurora PostgreSQL: cluster setup with the reasoning behind each setting, readers and endpoints, slow queries, caching, clones, global databases and write forwarding.
- [Kubernetes and Containers](https://karandeepsingh.ca/series/kubernetes-and-containers/): 9 articles. Nine articles that build up from what a container actually is — namespaces, chroot, cgroups — through pods, deployments and services, EKS on AWS, and the failures that account for most of the pain.
- [Mastering sed](https://karandeepsingh.ca/series/mastering-sed/): 9 articles. Nine articles on sed for real work: the one-liners worth memorising, the GNU vs BSD traps, multiline patterns and the hold space, and how to use it safely in a pipeline.
- [Serverless on AWS](https://karandeepsingh.ca/series/serverless-on-aws/): 9 articles. Nine articles on AWS Lambda and the services around it: your first function, DynamoDB and SQS, every configuration setting explained, cost tuning with measurements, and what breaks at a million users an hour.
- [Running Jenkins](https://karandeepsingh.ca/series/running-jenkins/): 7 articles. Seven operational guides to keeping Jenkins alive: LTS vs Weekly, the Java version it needs, safe upgrade paths, Configuration as Code, a baked AMI, and where the security risk actually lives.

## Articles

All 93 articles, newest first. Each line ends with the publish date and tags.

- [kubectl annotate: Change-Cause and Rollout History](https://karandeepsingh.ca/posts/kubectl-annotate-change-cause/index.md): Start with one kubectl annotate command and keep adding. Overwrite and remove annotations, fill the CHANGE-CAUSE column in rollout history, catch the stale change-cause trap, roll back, then move it all into git with a deploy script that … (2026-10-19; kubernetes, cli, containers)
- [Stop Pressing the Up Arrow: Searchable Shell History with Atuin](https://karandeepsingh.ca/posts/atuin-shell-history-search/index.md): Ctrl-R only matches one result at a time and forgets which directory you were in. Atuin swaps your shell history for a SQLite database and gives you a real search screen — what it does, why directory filtering changes your day, how … (2026-08-11; terminal, linux, bash)
- [Tuning AWS Lambda for Cost: The Memory Myth, Measured](https://karandeepsingh.ca/posts/lambda-cost-tuning-memory-graviton/index.md): Most Lambda cost advice repeats folklore. This walks through what AWS actually documents and bills — GB-seconds, the 1,769 MB one-vCPU line, the exact 20% arm64 discount and where it does not apply — then shows how to measure your own … (2026-08-10; aws, lambda, serverless, cost)
- [I Gave My Blog an MCP Server (and You Can Too)](https://karandeepsingh.ca/posts/blog-mcp-server-netlify/index.md): A walkthrough of building a public MCP server for a static blog — one serverless function, a build-time index, and all three MCP primitives — plus the mistakes that cost me an hour and why it runs for free. (2026-08-09; devops, automation)
- [Livestream From Your Phone to AWS with IVS (and What Every Setting Means)](https://karandeepsingh.ca/posts/livestream-phone-to-aws-ivs/index.md): How to livestream from your phone to other devices through AWS with Amazon IVS, no server and no code, plus a plain-English explanation of every setting in the create-channel wizard and the real gotchas (RTMP auth, "Can not find channel", … (2026-08-07; aws, cloud)
- [Handling a Million Users an Hour with AWS Lambda (Think Like a System Designer)](https://karandeepsingh.ca/posts/lambda-million-users-per-hour/index.md): A system-designer's walkthrough of running a million users an hour on AWS Lambda, the capacity math, concurrency and cold starts, which language to pick, the database bottleneck that actually breaks first, and how to handle logs and metrics … (2026-08-06; aws, lambda, serverless, monitoring-and-observability)
- [S3 Files on EC2: Mount an S3 Bucket as a File System (the Honest Guide)](https://karandeepsingh.ca/posts/ec2-s3-files-mount-bucket-file-system/index.md): Amazon S3 Files lets you mount an S3 bucket as a folder on EC2. This is the honest version, the setup gotchas (versioning, subnet, port 2049, a brutal Ubuntu build), how its latency compares to EBS and EFS, and whether Jenkins, config … (2026-08-06; aws, ec2, s3, storage)
- [AWS Lambda Configuration Explained: What to Care About (and Why)](https://karandeepsingh.ca/posts/lambda-configuration-explained/index.md): Most of AWS Lambda is configuration, not code, and the defaults aren't always right. This explains what each setting means, memory, timeout, concurrency, architecture, IAM, VPC, environment variables, and what to actually care about when … (2026-08-05; aws, lambda, serverless)
- [EC2 Storage in Plain English: Root Volumes vs File Systems](https://karandeepsingh.ca/posts/ec2-root-volumes-vs-file-systems/index.md): The EC2 launch wizard's storage step has two very different parts, the root volume and the File systems panel, and beginners mix them up. This explains both in plain words, plus when to reach for each. (2026-07-30; aws, ec2, storage)
- [Cloning an Aurora PostgreSQL Cluster (Copy-on-Write)](https://karandeepsingh.ca/posts/aurora-clone-copy-on-write/index.md): How Aurora copy-on-write cloning makes a fast, isolated copy of a cluster in minutes regardless of size, proven hands-on, plus when and why to use one, whether to keep or delete it, the benefits and limits, and whether it makes sense to add … (2026-07-27; aurora, postgresql, aws)
- [Running SQL Over HTTPS with the RDS Data API (Aurora PostgreSQL)](https://karandeepsingh.ca/posts/aurora-rds-data-api-postgresql/index.md): Turn on the RDS Data API for Aurora PostgreSQL and run SQL over an HTTPS endpoint with no persistent connection and no VPC, shown from both the in-console Query Editor and the AWS CLI, plus when to use it, when not to, and its limits. (2026-07-27; aurora, postgresql, aws)
- [Aurora Write Forwarding: Writing Through a Read Replica](https://karandeepsingh.ca/posts/aurora-write-forwarding-postgresql/index.md): How Aurora write forwarding lets a read replica take writes and pass them to the writer, done hands-on with local write forwarding on Aurora PostgreSQL, plus the global cross-region variant, the latency cost, which engines support it, and … (2026-07-27; aurora, postgresql, aws)
- [Adding an AWS Region to Aurora PostgreSQL (Global Database)](https://karandeepsingh.ca/posts/aurora-add-aws-region-global-database/index.md): A hands-on walkthrough of extending an Aurora PostgreSQL cluster to a second AWS Region with Aurora Global Database, including the instance-class upgrade and reboot, why the whole cluster shows Modifying, the opt-in region gotcha, and how … (2026-07-27; aurora, postgresql, aws)
- [Adding a Read Replica (Reader) to Aurora PostgreSQL](https://karandeepsingh.ca/posts/aurora-add-reader-read-replica/index.md): How to add a read replica (reader) to an Aurora PostgreSQL cluster, understand the writer and reader endpoints, prove the reader is read-only, and send read traffic to it, all from the AWS console. (2026-07-26; aurora, postgresql, aws)
- [Finding and Fixing Slow Queries in Aurora PostgreSQL](https://karandeepsingh.ca/posts/find-slow-queries-aurora-postgresql/index.md): How to find slow queries in Aurora PostgreSQL the way you actually would, starting in the AWS console with Performance Insights and CloudWatch, then dropping into pg_stat_statements and EXPLAIN to fix one query from 17 ms to 0.1 ms. (2026-07-26; aurora, postgresql, aws)
- [Valkey as a Cache in Front of Aurora PostgreSQL (Cache-Aside)](https://karandeepsingh.ca/posts/valkey-cache-aside-aurora-postgresql/index.md): The hands-on second half, connecting Python to Aurora PostgreSQL and Valkey, implementing the cache-aside pattern, and measuring a real 46 ms database read drop to 0.7 ms from cache. (2026-07-26; valkey, aurora, postgresql, python)
- [How to Set Up Aurora PostgreSQL (and Why Each Setting Matters)](https://karandeepsingh.ca/posts/aurora-postgresql-setup-guide/index.md): The reasoning behind every choice when creating an Amazon Aurora PostgreSQL cluster, from provisioned vs serverless to the security-group model, so the settings make sense instead of being guessed. (2026-07-25; aurora, postgresql, aws)
- [Valkey on AWS ElastiCache: A Hands-On Guide](https://karandeepsingh.ca/posts/valkey-aws-elasticache-guide/index.md): A start-to-finish walkthrough of running Valkey on AWS ElastiCache, built from a real hands-on session, complete with the security-group mistake, the TLS surprise, and a small health-check script. (2026-07-25; valkey, aws, python)
- [Upgrading Jenkins: Migration Strategies for Major Version Jumps](https://karandeepsingh.ca/posts/jenkins-upgrade-and-migration-guide/index.md): How to plan and execute Jenkins upgrades, including large multi-version jumps, safely with in-place, blue-green, and phased strategies. (2026-06-20; jenkins)
- [Docker Compose, Bake & ECR: Build and Ship Apps](https://karandeepsingh.ca/posts/docker-compose-multi-container-guide/index.md): A hands-on guide that takes a web + Postgres + Redis stack from a single Docker Compose file in local dev, to building images with Docker Bake, to pushing them to Amazon ECR for ECS, Fargate, or EKS. (2026-06-18; docker, containers, aws, devops)
- [Kubernetes on AWS: EKS Setup with eksctl](https://karandeepsingh.ca/posts/kubernetes-eks-setup-eksctl/index.md): A practical, zero-to-running guide to AWS EKS with eksctl — install the tools, create a cluster with one command, deploy a workload behind an AWS load balancer, and clean up so you do not get a surprise bill. (2026-06-18; kubernetes, eks, aws)
- [Fix Kubernetes CrashLoopBackOff: Causes and Solutions](https://karandeepsingh.ca/posts/fix-kubernetes-crashloopbackoff/index.md): A practical playbook for Kubernetes CrashLoopBackOff — the diagnosis workflow (describe, previous logs, events) plus fixes for the six causes you will actually hit, including OOMKilled, failing liveness probes, and missing config. (2026-06-18; kubernetes, containers, devops)
- [Kubernetes Fundamentals: Pods, Deployments, Services](https://karandeepsingh.ca/posts/kubernetes-fundamentals-pods-deployments-services/index.md): A hands-on introduction to Kubernetes fundamentals — pods, deployments, replicasets, and services — built step by step on a local kind cluster, with the mistakes and fixes that make the concepts stick. (2026-06-18; kubernetes, containers, devops)
- [kubectl Cheat Sheet: 30+ Essential Commands](https://karandeepsingh.ca/posts/kubectl-cheat-sheet/index.md): The kubectl commands you actually reach for, grouped by task — inspecting pods, managing deployments and services, reading logs, and debugging — with copy-paste examples and notes on when to use each. (2026-06-18; kubernetes, cli, containers)
- [Sed vs Awk vs Grep: When to Use Which (with Decision Matrix)](https://karandeepsingh.ca/posts/sed-vs-awk-vs-grep/index.md): A practical comparison of sed, awk, and grep with a decision tree, performance notes for large log files, and ten side-by-side examples of the same task in all three tools. (2026-05-05; bash, sed)
- [Sed Multiline Patterns: How to Match Across Lines](https://karandeepsingh.ca/posts/sed-multiline-patterns/index.md): A walkthrough of sed multiline editing — pattern space, hold space, N/D/P commands, address ranges — built up step by step with real DevOps examples (stack traces, YAML blocks, SQL). (2026-05-05; bash, sed)
- [Sed in CI/CD: Safe Patterns for GitHub Actions and Jenkins](https://karandeepsingh.ca/posts/sed-in-ci-cd-pipelines/index.md): A safety-first guide to running sed in automated pipelines — idempotency, exit codes, dry-run flags, in-place backup, and patterns that survive multiple runs without corrupting config files. (2026-05-05; bash, sed, cicd)
- [Sed Gotchas: GNU vs BSD and Safe In-Place Editing](https://karandeepsingh.ca/posts/sed-gotchas-gnu-vs-bsd/index.md): The sed gotchas every DevOps engineer hits — GNU vs BSD differences, in-place editing dangers, escape traps, and the portable patterns that work on Linux, macOS, and Alpine without modification. (2026-05-05; bash, sed)
- [Sed for Log Analysis: Errors, Time Filters, Patterns](https://karandeepsingh.ca/posts/sed-for-log-analysis/index.md): Practical sed recipes for working with production logs — error extraction, time-range filtering, multi-line stack trace handling, anonymization, and pre-pipeline cleanup before grep/awk. (2026-05-05; bash, sed)
- [What Teams Got Wrong About Kubernetes in 2025](https://karandeepsingh.ca/posts/what-i-got-wrong-about-kubernetes-2025/index.md): A year-end look at Kubernetes mistakes the industry kept making in 2025. Five things teams overinvested in, underinvested in, or simply got wrong on EKS this year. (2026-05-04; kubernetes, eks, devops, cloud)
- [The DevOps Stack I'd Pick If I Started Over in 2026](https://karandeepsingh.ca/posts/devops-stack-i-would-pick-2026/index.md): An opinionated DevOps stack pick from an engineer with years of production experience. Each layer is a vote, with what I would avoid and why. (2026-05-04; devops, cloud, aws, kubernetes, terraform, cicd)
- [Master tmux: From Multiplexer to a Go Session Manager](https://karandeepsingh.ca/posts/mastering-tmux-go-session-manager/index.md): Learn tmux one concept at a time — sessions, windows, panes, configuration — then build a Go CLI tool that reads a YAML file and launches your entire dev environment automatically. (2026-02-21; linux, cli, golang, bash, tmux)
- [Log Aggregator From Scratch in Go](https://karandeepsingh.ca/posts/log-aggregator-from-scratch-go/index.md): Build a log aggregator from scratch in Go. Tail files with inotify, survive log rotation, parse syslog and JSON, fan in from multiple sources, and wire up graceful shutdown. (2026-02-16; golang, linux, logging, devops)
- [Terraform From Scratch: Provision AWS Infrastructure](https://karandeepsingh.ca/posts/terraform-getting-started-aws/index.md): Learn Terraform with AWS step by step. Start with one S3 bucket, hit real errors, fix them, then build a full VPC + EC2 setup. Each step shows the command, the output, and the mistake. (2026-02-16; aws, terraform, containers)
- [Nginx Log Analysis: From grep to a Go Log Parser](https://karandeepsingh.ca/posts/nginx-log-analysis-grep-to-go/index.md): Nginx log analysis from grep to Go — each step shows the command-line approach first, then builds it in Go. Parse access logs, find slow endpoints, detect error spikes, build a log analysis dashboard. (2026-02-15; nginx, golang, docker, logging)
- [AWS CLI Automation: From Bash Scripts to Go](https://karandeepsingh.ca/posts/bash-scripting-meets-aws/index.md): AWS automation from CLI to Go code. Each step shows the AWS CLI command first, then builds it in Go. List EC2 instances, manage S3, check costs, build an infrastructure report tool. (2026-02-15; aws, golang, bash, automation)
- [Config Templating: From envsubst to Go](https://karandeepsingh.ca/posts/config-templating-envsubst-to-go/index.md): Config templating from envsubst to Go: each step shows the shell way first, then builds it in Go. Handle missing variables, add conditionals, build a config generator for dev/staging/prod. (2026-02-15; golang, bash, automation)
- [Process Management: From Linux Commands to a Go Supervisor](https://karandeepsingh.ca/posts/process-management-linux-to-go/index.md): Process management from command line to Go code. Each step shows the Linux command first, then builds it in Go. Kill processes, send signals, set resource limits, build a process supervisor with auto-restart. (2026-02-15; golang, linux, bash, automation)
- [Linux Access Control: From sudo to a Go Security Scanner](https://karandeepsingh.ca/posts/sudo-mastery-and-best-practices/index.md): Linux access control from command line to Go code. Each step shows the Linux command first, then builds it in Go. Parse sudoers, audit permissions, build a security scanner that flags dangerous configs. (2026-02-15; golang, linux, security, bash)
- [Timezones in Production: From Linux Commands to Go](https://karandeepsingh.ca/posts/timezones-in-production-linux-to-go/index.md): Timezone handling from command line to Go code — each step shows the Linux command first, then builds it in Go. Hit the DST parsing trap, fix it, normalize log timestamps, end with a multi-zone clock. (2026-02-15; golang, linux, bash, automation)
- [CPU Monitoring: From Linux Commands to a Go Dashboard](https://karandeepsingh.ca/posts/mastering-cpu-cores/index.md): CPU monitoring from command line to Go code — each step shows the Linux command first, then builds it in Go. Hit the jiffies trap, fix it, add colored bars, end with a live dashboard. (2026-02-15; golang, linux, monitoring-and-observability, bash)
- [Unix Power Tools Every DevOps Engineer Should Know](https://karandeepsingh.ca/posts/weird-unix-commands/index.md): Learn Unix commands that actually matter in production: pv for progress bars, xargs for parallel execution, watch for live monitoring, column for readable output. Then build a Go log watcher using the same patterns. (2026-02-15; linux, cli, golang, bash)
- [Go + Nginx: Deploy a Go API Behind a Reverse Proxy](https://karandeepsingh.ca/posts/go-nginx-reverse-proxy/index.md): Deploy a Go HTTP API behind Nginx. Start with a bare server, hit the port 80 permission trap, proxy through Nginx, lose client IPs, fix header forwarding, add rate limiting, and wire up graceful shutdown with systemd. (2026-02-15; golang, nginx, linux)
- [Containers From Scratch in Go](https://karandeepsingh.ca/posts/containers-from-scratch-go/index.md): Build your own container runtime in Go from scratch. Start by running a command, then add PID isolation, filesystem isolation, hostname, cgroups, one concept at a time. (2026-02-14; golang, docker, containers, linux)
- [Go + SQS: Build a Message Queue Processor](https://karandeepsingh.ca/posts/go-sqs-message-queue/index.md): Build an SQS message processor in Go step by step. Send messages, hit the visibility timeout trap, add batch operations, and build a production-ready polling loop. (2026-02-14; golang, aws, serverless)
- [Go + DynamoDB: Build a Simple CRUD App](https://karandeepsingh.ca/posts/go-dynamodb-crud/index.md): Build a DynamoDB CRUD app in Go step by step. Start raw, hit real pain points, and end up with clean struct-based operations. (2026-02-14; golang, aws, serverless)
- [Build a Go CLI Tool for AWS S3](https://karandeepsingh.ca/posts/go-aws-s3-cli-tool/index.md): Build a practical Go CLI tool for AWS S3. Start with listing buckets, then add uploads, downloads, and concurrent transfers step by step. (2026-02-14; golang, aws, s3, cli)
- [Build and Deploy a Go Lambda Function](https://karandeepsingh.ca/posts/build-deploy-go-lambda-function/index.md): Build your first Go Lambda function from scratch. We start simple, hit real errors, fix them, and work our way up to a production-ready API endpoint. (2026-02-14; golang, aws, lambda, serverless)
- [Jenkins Configuration as Code: The Definitive JCasC Guide](https://karandeepsingh.ca/posts/jenkins-configuration-as-code-jcasc-guide/index.md): Learn how to manage your entire Jenkins setup through version-controlled YAML files using Jenkins Configuration as Code (JCasC), with practical examples for Kubernetes, security, and shared libraries. (2026-02-14; jenkins, automation, kubernetes)
- [Deploy Jenkins on Amazon EKS: A Practical Tutorial](https://karandeepsingh.ca/posts/deploy-jenkins-eks-tutorial/index.md): Step-by-step tutorial for deploying Jenkins on Amazon EKS. Learn pods vs deployments and why each matters for your CI/CD setup. (2025-06-24; jenkins, eks, kubernetes, aws)
- [Jenkins LTS vs Weekly: Which Version Should You Use?](https://karandeepsingh.ca/posts/comprehensive-guide-to-jenkins-versions/index.md): A practical guide to Jenkins version selection, covering JVM compatibility, version evolution, and agent requirements, with links to focused deep-dive guides. (2025-03-13; jenkins)
- [Linux ls Command: History, Tricks, and Examples](https://karandeepsingh.ca/posts/linux-ls-command-comprehensive-guide/index.md): The definitive guide to the Linux ls command. Covers its five-decade history from early Unix to GNU coreutils, core options, task-based examples for real-world scenarios, quirky hidden tricks, and a quick-reference cheat sheet. (2025-03-10; linux, bash, cli)
- [Jenkins UserRemoteConfig: Dynamic Git in Pipelines](https://karandeepsingh.ca/posts/jenkins-userremoteconfig/index.md): Unlock advanced Git workflows with Jenkins UserRemoteConfig. Perfect for DevOps engineers managing multi-repository pipelines. (2025-02-25; jenkins, devops)
- [Linux Automation Tools: From Cron to a Custom Go Runner](https://karandeepsingh.ca/posts/linux-automation-tools/index.md): Master Linux task automation with cron, make, and systemd timers. Then build a complete Go task runner with scheduling, retries, and a status dashboard. (2025-02-05; linux, automation, golang)
- [Sed Cheat Sheet: 30 One-Liners from Real Production Logs](https://karandeepsingh.ca/posts/sed-command-cheat-sheet-30-essential-one-liners/index.md): Thirty sed one-liners for everyday log and config work. Each pattern solves a common problem (extract errors, clean configs, parse timestamps, transform JSON). Organized by use case with input/output examples and performance notes. (2025-01-28; bash, sed)
- [Deployment Automation: From SSH Scripts to a Go Deploy Tool](https://karandeepsingh.ca/posts/deployment-automation-ssh-to-go/index.md): Master deployment automation with Linux commands and Go. From SSH and rsync basics to building a complete deploy tool with health checks, rollbacks, and multi-server support. (2025-01-28; devops, golang, automation)
- [How to Replace Text in Multiple Files with Sed](https://karandeepsingh.ca/posts/replace-text-multiple-files-sed-guide/index.md): Learn reliable techniques for finding and replacing text across multiple files using sed, including file selection methods, safety practices, and advanced pattern matching with real-world examples. (2024-12-28; bash, sed)
- [Jenkins Java Compatibility: From Java 8 to Java 25](https://karandeepsingh.ca/posts/jenkins-java-compatibility/index.md): Jenkins JVM requirements and the Java 8 to 11 to 17 to 21 to 25 transition timeline, with version-by-version compatibility and tuning notes. (2024-10-15; jenkins)
- [Filename Extraction: basename to a Production File Pipeline](https://karandeepsingh.ca/posts/bash-extract-filename-from-path-guide/index.md): Work through a log-filename processing exercise from scratch. Start with simple basename, encounter the symlink bug that breaks extraction, fix it, then benchmark all four approaches yourself (basename vs parameter expansion vs awk vs sed). (2024-04-30; bash)
- [Sed for JSON: Emergency Patterns When jq Is Unavailable](https://karandeepsingh.ca/posts/sed-json-manipulation-without-jq/index.md): Emergency JSON manipulation patterns using sed when jq is unavailable. For scenarios requiring urgent config changes on a locked-down server with no permission to install tools. Always use jq for JSON - this is the emergency backup plan. (2024-04-28; bash, sed)
- [Mastering NGINX Logs: Configuration and Analysis Guide](https://karandeepsingh.ca/posts/mastering-nginx-logs-guide/index.md): A complete guide to NGINX logs. Learn how to configure, manage, and analyze access and error logs to optimize your NGINX server's performance and troubleshoot issues effectively. (2024-04-19; nginx, logging)
- [Linux User Management: Commands, Groups, Permissions, and Security](https://karandeepsingh.ca/posts/linux-user-control-terminal-commands/index.md): A practical reference for Linux user management. Create, modify, and delete users; work with groups; set file permissions and special bits; monitor access with who, last, and ps; and apply security best practices. (2024-04-16; bash, linux)
- [Advanced Guide to Using the top Command in Linux](https://karandeepsingh.ca/posts/mastering-top/index.md): The Linux top command is an essential tool for real-time system monitoring, providing a dynamic overview of running processes and their resource consumption. This guide digs deep into the functionalities of the top command, offering users … (2024-03-26; bash, linux-monitoring)
- [Advanced Guide to Using the vmstat Command in Linux](https://karandeepsingh.ca/posts/mastering-vmstat/index.md): This article provides an in-depth exploration of vmstat, a powerful tool for monitoring virtual memory and system performance. Starting with basic commands and moving through advanced functionalities, it offers practical examples to … (2024-03-24; bash, linux-monitoring)
- [AWS Security Audit: From AWS CLI to a Go Security Scanner](https://karandeepsingh.ca/posts/aws-security-features-guide/index.md): Master AWS security auditing with CLI commands and Go. From checking IAM users and security groups to building a complete scanner that finds and reports misconfigurations. (2024-03-01; aws, security, golang)
- [Jenkins Security: Advisories, CVEs, and Hardening](https://karandeepsingh.ca/posts/jenkins-security-advisories-and-cves/index.md): Jenkins security evolution, the CVE-2024-23897 CLI vulnerability, and keeping plugins patched to shrink the attack surface. (2024-02-01; jenkins)
- [Database Scaling: From 100K to 5M Users](https://karandeepsingh.ca/posts/how-to-scale-database-in-microservices/index.md): A worked example of scaling a database from a single Postgres instance to a sharded architecture. Start simple, hit realistic bottlenecks (connection limits, replica lag, query timeouts), and fix each one. The numbers are illustrative; the … (2024-01-09; devops)
- [Docker Logs: From docker logs to a Go Log Collector](https://karandeepsingh.ca/posts/docker-logs-to-go-collector/index.md): Master Docker container logging with Linux commands and a real Go log collector. From docker logs basics to a streaming aggregator that processes multiple containers. (2023-09-10; docker, golang)
- [Why YouTube-Scale Systems Need SQS: Architecture Notes](https://karandeepsingh.ca/posts/youtube-system-sqs-architecture/index.md): Adding Amazon SQS to your video platform creates a buffer between uploads and processing, dramatically improving scalability and reliability. This approach is ideal for systems with unpredictable traffic, high volume, or strict processing … (2023-08-28; serverless)
- [The Complete Guide to AWS S3 Static Website Hosting](https://karandeepsingh.ca/posts/aws-s3-hosting-guide/index.md): A practical guide to S3 hosting that walks through moving a WordPress documentation site to S3+CloudFront. Covers caching, cost, security hardening, and CI/CD best practices. (2023-08-28; aws, s3)
- [Service Health Checks: From curl to a Go Health Monitor](https://karandeepsingh.ca/posts/service-health-checks-curl-to-go/index.md): Master service health monitoring with Linux commands and Go. From curl and /proc basics to building a complete health monitor with HTTP checks, TCP probes, resource monitoring, and alerting. (2023-08-28; golang, devops)
- [Self-Healing Bash: Functions That Recover From Failures](https://karandeepsingh.ca/posts/self-healing-bash-functions/index.md): Learn how to implement self-healing mechanisms in your Bash scripts with practical examples that handle common failures and recover automatically. (2023-08-28; bash, automation)
- [Mastering sed for YAML, JSON, TOML Config Files](https://karandeepsingh.ca/posts/mastering-sed-update-remove-configuration-files/index.md): Learn how to leverage sed's powerful text processing capabilities to efficiently update, remove, and transform data in various configuration file formats like YAML, JSON, TOML, and properties files. (2023-08-28; bash, sed)
- [Mastering Bash: From echo to Production Deployment Script](https://karandeepsingh.ca/posts/mastering-bash-scripting/index.md): Watch bash skills evolve from echo statements to a production deployment script. Start simple, encounter errors (set -e breaks pipelines, unquoted variables cause word splitting), fix each one, learn functions and error handling along the … (2023-08-28; bash, linux)
- [CI Pipeline Basics: From Shell Scripts to a Go Build Runner](https://karandeepsingh.ca/posts/jenkins-devops-best-practices/index.md): Master CI pipeline fundamentals with shell scripts and Go. From running tests and collecting artifacts to building a complete build runner with git integration and parallel stages. (2023-08-28; automation, cicd, golang)
- [Building a YouTube-like System with AWS Lambda and S3](https://karandeepsingh.ca/posts/youtube-system-design-aws/index.md): Create a scalable video platform using AWS Lambda functions that process uploads directly from S3. This simple approach works beautifully for video platforms and requires minimal management. (2023-08-28; serverless)
- [Building a URL Shortener in Go: From TCP Sockets to a Working Service](https://karandeepsingh.ca/posts/url-shortener-from-scratch-go/index.md): A step-by-step build of a URL shortener in Go using only the standard library — raw TCP, HTTP parsing, hash-based short codes, file persistence, 302 redirects with click analytics, and rate limiting. Each step introduces a real bug first, … (2023-08-28; golang, linux)
- [Bash Error Handling: Patterns for Bulletproof Scripts](https://karandeepsingh.ca/posts/bash-error-handling-bulletproof-scripts/index.md): Explore practical strategies to implement reliable error handling in Bash scripts, preventing silent failures and building automation you can truly trust. (2023-08-28; bash)
- [Bash Code Shortening: Writing Concise Shell Scripts](https://karandeepsingh.ca/posts/bash-code-shortening-techniques/index.md): Master the art of Bash code shortening with this full guide offering expert techniques, real-world examples, and practical tips to write cleaner, more efficient scripts without sacrificing readability. (2023-08-28; bash)
- [Advanced Bash Scripting for Automation](https://karandeepsingh.ca/posts/advanced-bash-scripting-techniques-for-automation/index.md): Practical Bash automation patterns for deploying to many servers. Includes retry logic, error handling, parallel execution, and template generation that keep multi-server deploys fast and reliable. (2023-08-28; automation, bash)
- [Bash String Validation, Generation & a Library](https://karandeepsingh.ca/posts/bash-string-validation-and-generation/index.md): Bash functions for random string generation, input sanitization, CSV parsing, password-strength checks, and slug generation, plus a complete production string library to source. (2023-02-20; bash)
- [Bash String Functions: Trimming, Case, and Reversal](https://karandeepsingh.ca/posts/bash-string-trimming-and-case/index.md): Bash string functions for stripping leading and trailing whitespace (ltrim, rtrim, trim) and reversing strings, with performance comparisons against sed and awk. (2023-02-20; bash)
- [Bash String Functions: Search, Split, Count, Extract](https://karandeepsingh.ca/posts/bash-string-search-split-and-count/index.md): Bash string functions for length validation, case conversion, substitution, truncation, counting, splitting, capitalization, ROT13, and field extraction (index, substring, join). (2023-02-20; bash)
- [Advanced Bash String Operations](https://karandeepsingh.ca/posts/advanced-string-operations-in-bash-building-custom-functions/index.md): An overview hub for custom Bash string functions used in log processing and everyday scripting, linking to focused guides on trimming and case, search and split, and validation and generation. (2023-02-20; bash)
- [Boto3 + AWS Lambda: A Production Serverless Pipeline](https://karandeepsingh.ca/posts/boto3-and-aws-lambda-a-match-made-in-serverless-heaven/index.md): Production guide to building serverless data pipelines with Boto3 and Lambda for processing high-volume daily events through an analytics pipeline. Covers cold starts, concurrent execution limits, error handling, retries, and cost … (2023-01-09; lambda, serverless, python, boto3)
- [Create Custom AMI of Jenkins | DevOps](https://karandeepsingh.ca/posts/create-custom-ami-of-jenkins-devops/index.md): A comprehensive guide to roll out your own custom Jenkins AMI, perfect for those who want a hassle-free, tailor-made Continuous Integration environment. (2022-11-13; aws, jenkins, docker, linux)
- [Git Hooks: From Shell Scripts to a Go Webhook Server](https://karandeepsingh.ca/posts/git-hooks-to-go-webhook-server/index.md): Master Git automation with shell hooks and Go. From pre-commit scripts to a complete webhook server that validates payloads, triggers builds, and reports results. (2022-11-12; golang, automation, cicd)
- [Remote Server Config: From SSH Loops to a Go Config Tool](https://karandeepsingh.ca/posts/remote-server-config-ssh-to-go/index.md): Master remote server configuration with SSH and Go. From bash loops to a complete config tool with parallel execution, idempotent operations, and service verification. (2022-11-12; automation, golang)
- [EC2 Image Builder: Create a New Image Recipe](https://karandeepsingh.ca/posts/create-a-new-image-recipe/index.md): How to create a new version of an EC2 Image Builder image recipe, update the pipeline, and verify the AMI build through testing and distribution. (2022-04-02; aws, ec2)
- [Build Image with EC2 Image Builder Components](https://karandeepsingh.ca/posts/ec2-image-builder-components/index.md): Build custom AMIs using EC2 Image Builder with Amazon Linux 2023 and NGINX. (2022-04-02; ec2, docker, aws, linux)
- [Create Freeform Feature Flag from S3 Object](https://karandeepsingh.ca/posts/create-freeform-feature-flag-from-s3-object/index.md): Step-by-step guide to creating freeform feature flags using AWS AppConfig and S3 objects. Manage application configurations and toggle features without redeploying. (2022-03-20; s3, aws)
- [Deploy a Hugo Site to S3 with CodeBuild](https://karandeepsingh.ca/posts/deploy-hugo-s3-codebuild/index.md): Learn how to deploy a Hugo static site to S3 using AWS CodeBuild with a custom GoLang Docker image and a buildspec.yaml pipeline. (2021-09-23; s3, aws, codebuild)
- [Advanced buildspec.yml: Python, Go, ECR Push, EKS Deploy](https://karandeepsingh.ca/posts/aws-codebuild-advanced-buildspec-guide/index.md): This article covers the buildspec.yaml file in AWS CodeBuild with practical examples for Node.js, Python, and Go projects. Includes the critical ECR login fix and EKS deployment patterns.  (2021-09-23; containers, codebuild, eks)

## Cheatsheets

Quick references, grouped by topic. Index page: [Cheatsheets](https://karandeepsingh.ca/cheatsheets/).


### Python

- [Python Basics & CLI Cheatsheet](https://karandeepsingh.ca/cheatsheets/python/python-basics-cli/index.md): Notes on core Python syntax, error handling, file I/O with pathlib, plus the CLI side (venv, pip, argparse, and python -m tricks).
- [Python Threading & Concurrency Cheatsheet](https://karandeepsingh.ca/cheatsheets/python/python-threading-concurrency/index.md): Python concurrency notes covering threading primitives, queue producer/consumer, concurrent.futures, multiprocessing, asyncio, and when to use which.
- [Python Boto3 Cheatsheet](https://karandeepsingh.ca/cheatsheets/python/python-boto3/index.md): My notes for automating AWS with boto3, covering client/resource setup, S3, EC2, Lambda, DynamoDB, and SSM, plus paginators, waiters, and error handling done right.

### Bash

- [Bash Basics Cheatsheet](https://karandeepsingh.ca/cheatsheets/bash/bash-basics/index.md): A dense quick-reference for Bash 5.x scripting. Quoting, parameter expansion, test operators, loops, functions, arrays, traps, and here-docs.
- [Bash sed & awk Cheatsheet](https://karandeepsingh.ca/cheatsheets/bash/bash-sed-awk/index.md): My working reference for sed and awk. Substitution, addresses, capture groups, fields, BEGIN/END blocks, associative arrays, and the one-liners I actually use.
- [Bash CLI One-Liners Cheatsheet](https://karandeepsingh.ca/cheatsheets/bash/bash-cli/index.md): The command-line one-liners I use daily. find/xargs, grep, curl, jq, tar, disk and process triage, network checks, ssh/rsync, history tricks, and job control.

### Containers

- [Docker Cheatsheet](https://karandeepsingh.ca/cheatsheets/containers/docker/index.md): A dense, practical Docker reference covering the image lifecycle, container run flags, Dockerfiles with multi-stage builds, compose, and cleanup commands.
- [Kubernetes Cheatsheet](https://karandeepsingh.ca/cheatsheets/containers/kubernetes/index.md): A working kubectl reference with core resource YAML skeletons, rollout and scaling commands, debugging workflows, and a table of handy one-liners.

### DevOps Tooling

- [Jenkins Cheatsheet](https://karandeepsingh.ca/cheatsheets/devops/jenkins/index.md): Quick-reference Jenkins pipeline syntax. Declarative skeletons, credentials binding, parallel stages, shared libraries, triggers, and troubleshooting fixes.
- [AWS CloudFormation Cheatsheet](https://karandeepsingh.ca/cheatsheets/devops/cloudformation/index.md): Quick-reference CloudFormation guide. Template sections, intrinsic functions, stack lifecycle CLI commands, change sets, drift detection, and error fixes.
- [Ansible Cheatsheet](https://karandeepsingh.ca/cheatsheets/devops/ansible/index.md): Quick-reference Ansible guide. Inventories, ad-hoc commands, playbooks, module table, loops and conditionals, roles layout, Vault, and Jinja2 templating.

### Terminal

- [Atuin Cheatsheet](https://karandeepsingh.ca/cheatsheets/terminal/atuin/index.md): My Atuin notes. Search keybindings, filter and search modes, the search flags worth knowing, sync and account commands, stats, config essentials, and troubleshooting — checked against Atuin 18.19.0.
- [tmux Cheatsheet](https://karandeepsingh.ca/cheatsheets/terminal/tmux/index.md): My tmux notes. Session management, window and pane keybindings, copy mode, config essentials, and scripting layouts with send-keys.
- [Ubuntu Terminal Cheatsheet](https://karandeepsingh.ca/cheatsheets/terminal/ubuntu-terminal/index.md): The Ubuntu server commands I reach for daily. Covers apt/dpkg/snap package management, systemd, ufw, user management, networking, and upgrades.
- [Amazon Linux 2 Terminal Cheatsheet](https://karandeepsingh.ca/cheatsheets/terminal/al2-terminal/index.md): Working notes for Amazon Linux 2 on EC2. Covers yum and amazon-linux-extras, SSM access, instance metadata, cloud-init, and security updates.

### Languages

- [Go (Golang) Cheatsheet](https://karandeepsingh.ca/cheatsheets/languages/golang/index.md): A practical Go reference for DevOps work covering go mod and build commands, core syntax, concurrency patterns, testing, and the stdlib snippets I reuse most.

### Cloud Cost

- [Low-Cost Cloud Stack Cheatsheet](https://karandeepsingh.ca/cheatsheets/cost/low-cost-cloud-stack/index.md): Quick-reference for a lean, low-cost cloud stack. Free-tier frontend hosting, API platforms, serverless Postgres, auth, a full cost comparison table, low-cost VPS/cloud providers (OVH, Hetzner, …
- [MSK vs Kinesis Cheatsheet (Cost & When to Choose)](https://karandeepsingh.ca/cheatsheets/cost/msk-vs-kinesis/index.md): Quick-reference comparing Amazon MSK (managed Kafka) and Kinesis Data Streams, pricing models and idle cost, a feature table, and a clear when-to-choose-which decision guide.

## Topics

Tag pages list every article on a topic. Number of articles in brackets.

- [aws](https://karandeepsingh.ca/tags/aws/) (36)
- [bash](https://karandeepsingh.ca/tags/bash/) (36)
- [golang](https://karandeepsingh.ca/tags/golang/) (26)
- [linux](https://karandeepsingh.ca/tags/linux/) (24)
- [automation](https://karandeepsingh.ca/tags/automation/) (15)
- [devops](https://karandeepsingh.ca/tags/devops/) (11)
- [containers](https://karandeepsingh.ca/tags/containers/) (10)
- [kubernetes](https://karandeepsingh.ca/tags/kubernetes/) (10)
- [sed](https://karandeepsingh.ca/tags/sed/) (10)
- [cli](https://karandeepsingh.ca/tags/cli/) (9)
- [jenkins](https://karandeepsingh.ca/tags/jenkins/) (9)
- [serverless](https://karandeepsingh.ca/tags/serverless/) (9)
- [aurora](https://karandeepsingh.ca/tags/aurora/) (8)
- [postgresql](https://karandeepsingh.ca/tags/postgresql/) (8)
- [docker](https://karandeepsingh.ca/tags/docker/) (7)
- [python](https://karandeepsingh.ca/tags/python/) (6)
- [cicd](https://karandeepsingh.ca/tags/cicd/) (5)
- [cloud](https://karandeepsingh.ca/tags/cloud/) (5)
- [ec2](https://karandeepsingh.ca/tags/ec2/) (5)
- [lambda](https://karandeepsingh.ca/tags/lambda/) (5)
- [s3](https://karandeepsingh.ca/tags/s3/) (5)
- [eks](https://karandeepsingh.ca/tags/eks/) (4)
- [cost](https://karandeepsingh.ca/tags/cost/) (3)
- [logging](https://karandeepsingh.ca/tags/logging/) (3)
- [nginx](https://karandeepsingh.ca/tags/nginx/) (3)
- [boto3](https://karandeepsingh.ca/tags/boto3/) (2)
- [codebuild](https://karandeepsingh.ca/tags/codebuild/) (2)
- [linux-monitoring](https://karandeepsingh.ca/tags/linux-monitoring/) (2)
- [monitoring-and-observability](https://karandeepsingh.ca/tags/monitoring-and-observability/) (2)
- [security](https://karandeepsingh.ca/tags/security/) (2)
- [storage](https://karandeepsingh.ca/tags/storage/) (2)
- [terminal](https://karandeepsingh.ca/tags/terminal/) (2)
- [terraform](https://karandeepsingh.ca/tags/terraform/) (2)
- [tmux](https://karandeepsingh.ca/tags/tmux/) (2)
- [valkey](https://karandeepsingh.ca/tags/valkey/) (2)

## Optional

- [Sitemap](https://karandeepsingh.ca/sitemap.xml): every URL with its last-modified date.
- [Quiz questions](https://karandeepsingh.ca/mcp-quiz.json): every quiz question, options, answer and explanation, as JSON.
- [Commands](https://karandeepsingh.ca/mcp-code.json): the runnable code blocks from every page, as JSON.
- [Privacy policy](https://karandeepsingh.ca/privacy/index.md)
- [Terms](https://karandeepsingh.ca/terms/index.md)