Posts on Karandeep Singh - DevOps in Calgary

Tuning AWS Lambda for Cost: The Memory Myth, Measured

Mon, 10 Aug 2026 19:45:00 +0000

Most Lambda cost advice repeats folklore. This walks through what AWS actually documents and bills — GB-seconds, the 1,769 MB one-vCPU line, the exact 20% arm64 discount and where it does not apply — then shows how to measure your own function instead of guessing.

I Gave My Blog an MCP Server (and You Can Too)

Sun, 09 Aug 2026 19:30:00 +0000

A walkthrough of building a public MCP server for a static blog — one serverless function, a build-time index, and all three MCP primitives — plus the mistakes that cost me an hour and why it runs for free.

Livestream From Your Phone to AWS with IVS (and What Every Setting Means)

Fri, 07 Aug 2026 10:00:00 +0000

How to livestream from your phone to other devices through AWS with Amazon IVS, no server and no code, plus a plain-English explanation of every setting in the create-channel wizard and the real gotchas (RTMP auth, “Can not find channel”, why Chrome won’t play the URL).

Handling a Million Users an Hour with AWS Lambda (Think Like a System Designer)

Thu, 06 Aug 2026 10:00:00 +0000

A system-designer’s walkthrough of running a million users an hour on AWS Lambda, the capacity math, concurrency and cold starts, which language to pick, the database bottleneck that actually breaks first, and how to handle logs and metrics without the bill or the noise getting out of hand.

S3 Files on EC2: Mount an S3 Bucket as a File System (the Honest Guide)

Thu, 06 Aug 2026 09:00:00 +0000

Amazon S3 Files lets you mount an S3 bucket as a folder on EC2. This is the honest version, the setup gotchas (versioning, subnet, port 2049, a brutal Ubuntu build), how its latency compares to EBS and EFS, and whether Jenkins, config files, and logs belong on it.

AWS Lambda Configuration Explained: What to Care About (and Why)

Wed, 05 Aug 2026 10:00:00 +0000

Most of AWS Lambda is configuration, not code, and the defaults aren’t always right. This explains what each setting means, memory, timeout, concurrency, architecture, IAM, VPC, environment variables, and what to actually care about when you set them.

EC2 Storage in Plain English: Root Volumes vs File Systems

Thu, 30 Jul 2026 09:00:00 +0000

The EC2 launch wizard’s storage step has two very different parts, the root volume and the File systems panel, and beginners mix them up. This explains both in plain words, plus when to reach for each.

Cloning an Aurora PostgreSQL Cluster (Copy-on-Write)

Mon, 27 Jul 2026 04:15:00 +0000

How Aurora copy-on-write cloning makes a fast, isolated copy of a cluster in minutes regardless of size, proven hands-on, plus when and why to use one, whether to keep or delete it, the benefits and limits, and whether it makes sense to add readers or Regions to a clone.

Running SQL Over HTTPS with the RDS Data API (Aurora PostgreSQL)

Mon, 27 Jul 2026 04:00:00 +0000

Turn on the RDS Data API for Aurora PostgreSQL and run SQL over an HTTPS endpoint with no persistent connection and no VPC, shown from both the in-console Query Editor and the AWS CLI, plus when to use it, when not to, and its limits.

Aurora Write Forwarding: Writing Through a Read Replica

Mon, 27 Jul 2026 03:00:00 +0000

How Aurora write forwarding lets a read replica take writes and pass them to the writer, done hands-on with local write forwarding on Aurora PostgreSQL, plus the global cross-region variant, the latency cost, which engines support it, and why you still need an instance in every Region you serve.

Adding an AWS Region to Aurora PostgreSQL (Global Database)

Mon, 27 Jul 2026 00:30:00 +0000

A hands-on walkthrough of extending an Aurora PostgreSQL cluster to a second AWS Region with Aurora Global Database, including the instance-class upgrade and reboot, why the whole cluster shows Modifying, the opt-in region gotcha, and how far regions and readers scale.

Adding a Read Replica (Reader) to Aurora PostgreSQL

Sun, 26 Jul 2026 03:30:00 +0000

How to add a read replica (reader) to an Aurora PostgreSQL cluster, understand the writer and reader endpoints, prove the reader is read-only, and send read traffic to it, all from the AWS console.

Finding and Fixing Slow Queries in Aurora PostgreSQL

Sun, 26 Jul 2026 01:00:00 +0000

How to find slow queries in Aurora PostgreSQL the way you actually would, starting in the AWS console with Performance Insights and CloudWatch, then dropping into pg_stat_statements and EXPLAIN to fix one query from 17 ms to 0.1 ms.

Valkey as a Cache in Front of Aurora PostgreSQL (Cache-Aside)

Sun, 26 Jul 2026 00:30:00 +0000

The hands-on second half, connecting Python to Aurora PostgreSQL and Valkey, implementing the cache-aside pattern, and measuring a real 46 ms database read drop to 0.7 ms from cache.

How to Set Up Aurora PostgreSQL (and Why Each Setting Matters)

Sat, 25 Jul 2026 20:00:00 +0000

The reasoning behind every choice when creating an Amazon Aurora PostgreSQL cluster, from provisioned vs serverless to the security-group model, so the settings make sense instead of being guessed.

Valkey on AWS ElastiCache: A Hands-On Guide

Sat, 25 Jul 2026 02:00:00 +0000

A start-to-finish walkthrough of running Valkey on AWS ElastiCache, built from a real hands-on session, complete with the security-group mistake, the TLS surprise, and a small health-check script.

Upgrading Jenkins: Migration Strategies for Major Version Jumps

Sat, 20 Jun 2026 09:00:00 +0000

How to plan and execute Jenkins upgrades, including large multi-version jumps, safely with in-place, blue-green, and phased strategies.

Docker Compose, Bake & ECR: Build and Ship Apps

Thu, 18 Jun 2026 06:01:00 +0000

A hands-on guide that takes a web + Postgres + Redis stack from a single Docker Compose file in local dev, to building images with Docker Bake, to pushing them to Amazon ECR for ECS, Fargate, or EKS.

Kubernetes on AWS: EKS Setup with eksctl

Thu, 18 Jun 2026 05:01:00 +0000

A practical, zero-to-running guide to AWS EKS with eksctl — install the tools, create a cluster with one command, deploy a workload behind an AWS load balancer, and clean up so you do not get a surprise bill.

Fix Kubernetes CrashLoopBackOff: Causes and Solutions

Thu, 18 Jun 2026 04:01:00 +0000

A practical playbook for Kubernetes CrashLoopBackOff — the diagnosis workflow (describe, previous logs, events) plus fixes for the six causes you will actually hit, including OOMKilled, failing liveness probes, and missing config.

Kubernetes Fundamentals: Pods, Deployments, Services

Thu, 18 Jun 2026 03:01:00 +0000

A hands-on introduction to Kubernetes fundamentals — pods, deployments, replicasets, and services — built step by step on a local kind cluster, with the mistakes and fixes that make the concepts stick.

kubectl Cheat Sheet: 30+ Essential Commands

Thu, 18 Jun 2026 02:01:00 +0000

The kubectl commands you actually reach for, grouped by task — inspecting pods, managing deployments and services, reading logs, and debugging — with copy-paste examples and notes on when to use each.

Sed for Log Analysis: Errors, Time Filters, Patterns

Tue, 05 May 2026 00:00:00 +0000

Practical sed recipes for working with production logs — error extraction, time-range filtering, multi-line stack trace handling, anonymization, and pre-pipeline cleanup before grep/awk.

Sed Gotchas: GNU vs BSD and Safe In-Place Editing

Tue, 05 May 2026 00:00:00 +0000

The sed gotchas every DevOps engineer hits — GNU vs BSD differences, in-place editing dangers, escape traps, and the portable patterns that work on Linux, macOS, and Alpine without modification.

Sed in CI/CD: Safe Patterns for GitHub Actions and Jenkins

Tue, 05 May 2026 00:00:00 +0000

A safety-first guide to running sed in automated pipelines — idempotency, exit codes, dry-run flags, in-place backup, and patterns that survive multiple runs without corrupting config files.

Sed Multiline Patterns: How to Match Across Lines

Tue, 05 May 2026 00:00:00 +0000

A walkthrough of sed multiline editing — pattern space, hold space, N/D/P commands, address ranges — built up step by step with real DevOps examples (stack traces, YAML blocks, SQL).

Sed vs Awk vs Grep: When to Use Which (with Decision Matrix)

Tue, 05 May 2026 00:00:00 +0000

A practical comparison of sed, awk, and grep with a decision tree, performance notes for large log files, and ten side-by-side examples of the same task in all three tools.

What Teams Got Wrong About Kubernetes in 2025

Mon, 04 May 2026 10:00:00 -0600

A year-end look at Kubernetes mistakes the industry kept making in 2025. Five things teams overinvested in, underinvested in, or simply got wrong on EKS this year.

The DevOps Stack I'd Pick If I Started Over in 2026

Mon, 04 May 2026 08:00:00 -0600

An opinionated DevOps stack pick from an engineer with years of production experience. Each layer is a vote, with what I would avoid and why.

Master tmux: From Multiplexer to a Go Session Manager

Sat, 21 Feb 2026 14:00:00 +0000

Learn tmux one concept at a time — sessions, windows, panes, configuration — then build a Go CLI tool that reads a YAML file and launches your entire dev environment automatically.

Log Aggregator From Scratch in Go

Mon, 16 Feb 2026 14:00:00 +0000

Build a log aggregator from scratch in Go. Tail files with inotify, survive log rotation, parse syslog and JSON, fan in from multiple sources, and wire up graceful shutdown.

Terraform From Scratch: Provision AWS Infrastructure

Mon, 16 Feb 2026 10:00:00 +0000

Learn Terraform with AWS step by step. Start with one S3 bucket, hit real errors, fix them, then build a full VPC + EC2 setup. Each step shows the command, the output, and the mistake.

Nginx Log Analysis: From grep to a Go Log Parser

Sun, 15 Feb 2026 23:00:00 +0000

Nginx log analysis from grep to Go — each step shows the command-line approach first, then builds it in Go. Parse access logs, find slow endpoints, detect error spikes, build a log analysis dashboard.

AWS CLI Automation: From Bash Scripts to Go

Sun, 15 Feb 2026 22:00:00 +0000

AWS automation from CLI to Go code. Each step shows the AWS CLI command first, then builds it in Go. List EC2 instances, manage S3, check costs, build an infrastructure report tool.

Config Templating: From envsubst to Go

Sun, 15 Feb 2026 21:00:00 +0000

Config templating from envsubst to Go: each step shows the shell way first, then builds it in Go. Handle missing variables, add conditionals, build a config generator for dev/staging/prod.

Process Management: From Linux Commands to a Go Supervisor

Sun, 15 Feb 2026 20:00:00 +0000

Process management from command line to Go code. Each step shows the Linux command first, then builds it in Go. Kill processes, send signals, set resource limits, build a process supervisor with auto-restart.

Linux Access Control: From sudo to a Go Security Scanner

Sun, 15 Feb 2026 19:00:00 +0000

Linux access control from command line to Go code. Each step shows the Linux command first, then builds it in Go. Parse sudoers, audit permissions, build a security scanner that flags dangerous configs.

Timezones in Production: From Linux Commands to Go

Sun, 15 Feb 2026 18:00:00 +0000

Timezone handling from command line to Go code — each step shows the Linux command first, then builds it in Go. Hit the DST parsing trap, fix it, normalize log timestamps, end with a multi-zone clock.

CPU Monitoring: From Linux Commands to a Go Dashboard

Sun, 15 Feb 2026 17:00:00 +0000

CPU monitoring from command line to Go code — each step shows the Linux command first, then builds it in Go. Hit the jiffies trap, fix it, add colored bars, end with a live dashboard.

Unix Power Tools Every DevOps Engineer Should Know

Sun, 15 Feb 2026 16:00:00 +0000

Learn Unix commands that actually matter in production: pv for progress bars, xargs for parallel execution, watch for live monitoring, column for readable output. Then build a Go log watcher using the same patterns.

Go + Nginx: Deploy a Go API Behind a Reverse Proxy

Sun, 15 Feb 2026 14:00:00 +0000

Deploy a Go HTTP API behind Nginx. Start with a bare server, hit the port 80 permission trap, proxy through Nginx, lose client IPs, fix header forwarding, add rate limiting, and wire up graceful shutdown with systemd.

Containers From Scratch in Go

Sat, 14 Feb 2026 14:00:00 +0000

Build your own container runtime in Go from scratch. Start by running a command, then add PID isolation, filesystem isolation, hostname, cgroups, one concept at a time.

Go + SQS: Build a Message Queue Processor

Sat, 14 Feb 2026 13:00:00 +0000

Build an SQS message processor in Go step by step. Send messages, hit the visibility timeout trap, add batch operations, and build a production-ready polling loop.

Go + DynamoDB: Build a Simple CRUD App

Sat, 14 Feb 2026 12:00:00 +0000

Build a DynamoDB CRUD app in Go step by step. Start raw, hit real pain points, and end up with clean struct-based operations.

Build a Go CLI Tool for AWS S3

Sat, 14 Feb 2026 11:00:00 +0000

Build a practical Go CLI tool for AWS S3. Start with listing buckets, then add uploads, downloads, and concurrent transfers step by step.

Build and Deploy a Go Lambda Function

Sat, 14 Feb 2026 10:00:00 +0000

Build your first Go Lambda function from scratch. We start simple, hit real errors, fix them, and work our way up to a production-ready API endpoint.

Jenkins Configuration as Code: The Definitive JCasC Guide

Sat, 14 Feb 2026 02:01:00 +0000

Learn how to manage your entire Jenkins setup through version-controlled YAML files using Jenkins Configuration as Code (JCasC), with practical examples for Kubernetes, security, and shared libraries.

Deploy Jenkins on Amazon EKS: A Practical Tutorial

Tue, 24 Jun 2025 20:01:00 +0000

Step-by-step tutorial for deploying Jenkins on Amazon EKS. Learn pods vs deployments and why each matters for your CI/CD setup.

Jenkins LTS vs Weekly: Which Version Should You Use?

Thu, 13 Mar 2025 14:45:00 +0000

A practical guide to Jenkins version selection, covering JVM compatibility, version evolution, and agent requirements, with links to focused deep-dive guides.

Linux ls Command: History, Tricks, and Examples

Mon, 10 Mar 2025 14:00:00 +0000

The definitive guide to the Linux ls command. Covers its five-decade history from early Unix to GNU coreutils, core options, task-based examples for real-world scenarios, quirky hidden tricks, and a quick-reference cheat sheet.

Jenkins UserRemoteConfig: Dynamic Git in Pipelines

Tue, 25 Feb 2025 00:00:00 +0000

Unlock advanced Git workflows with Jenkins UserRemoteConfig. Perfect for DevOps engineers managing multi-repository pipelines.

Linux Automation Tools: From Cron to a Custom Go Runner

Wed, 05 Feb 2025 00:00:00 +0000

Master Linux task automation with cron, make, and systemd timers. Then build a complete Go task runner with scheduling, retries, and a status dashboard.

Deployment Automation: From SSH Scripts to a Go Deploy Tool

Tue, 28 Jan 2025 02:01:00 +0000

Master deployment automation with Linux commands and Go. From SSH and rsync basics to building a complete deploy tool with health checks, rollbacks, and multi-server support.

Sed Cheat Sheet: 30 One-Liners from Real Production Logs

Tue, 28 Jan 2025 02:01:00 +0000

Thirty sed one-liners for everyday log and config work. Each pattern solves a common problem (extract errors, clean configs, parse timestamps, transform JSON). Organized by use case with input/output examples and performance notes.

How to Replace Text in Multiple Files with Sed

Sat, 28 Dec 2024 02:01:00 +0000

Learn reliable techniques for finding and replacing text across multiple files using sed, including file selection methods, safety practices, and advanced pattern matching with real-world examples.

Jenkins Java Compatibility: From Java 8 to Java 25

Tue, 15 Oct 2024 09:00:00 +0000

Jenkins JVM requirements and the Java 8 to 11 to 17 to 21 to 25 transition timeline, with version-by-version compatibility and tuning notes.

Filename Extraction: basename to a Production File Pipeline

Tue, 30 Apr 2024 07:00:00 +0000

Work through a log-filename processing exercise from scratch. Start with simple basename, encounter the symlink bug that breaks extraction, fix it, then benchmark all four approaches yourself (basename vs parameter expansion vs awk vs sed).

Sed for JSON: Emergency Patterns When jq Is Unavailable

Sun, 28 Apr 2024 02:01:00 +0000

Emergency JSON manipulation patterns using sed when jq is unavailable. For scenarios requiring urgent config changes on a locked-down server with no permission to install tools. Always use jq for JSON - this is the emergency backup plan.

Mastering NGINX Logs: Configuration and Analysis Guide

Fri, 19 Apr 2024 20:01:00 +0000

A complete guide to NGINX logs. Learn how to configure, manage, and analyze access and error logs to optimize your NGINX server’s performance and troubleshoot issues effectively.

Linux User Management: Commands, Groups, Permissions, and Security

Tue, 16 Apr 2024 02:01:00 +0000

A practical reference for Linux user management. Create, modify, and delete users; work with groups; set file permissions and special bits; monitor access with who, last, and ps; and apply security best practices.

Advanced Guide to Using the top Command in Linux

Tue, 26 Mar 2024 02:01:00 +0000

The Linux top command is an essential tool for real-time system monitoring, providing a dynamic overview of running processes and their resource consumption. This guide digs deep into the functionalities of the top command, offering users of all skill levels, from novices to seasoned system administrators, a detailed look at how to utilize this tool effectively. Starting with the basics of launching and reading top, the guide progresses through intermediate techniques like sorting and filtering processes, and advances into complex uses such as batch mode operation and remote system monitoring. Practical examples in each section demonstrate how to apply these techniques in real-world scenarios, enhancing understanding and skill. Additionally, the guide addresses common questions through a thorough FAQ section, equipping users with the knowledge to troubleshoot common issues and optimize system performance. The conclusion ties everything together, suggesting further resources for expanding beyond top to other diagnostic tools, thereby providing a rounded approach to system management.

Advanced Guide to Using the vmstat Command in Linux

Sun, 24 Mar 2024 02:01:00 +0000

This article provides an in-depth exploration of vmstat, a powerful tool for monitoring virtual memory and system performance. Starting with basic commands and moving through advanced functionalities, it offers practical examples to demonstrate each option of vmstat. Learn how to combine different options for thorough monitoring, automate tasks with scripting, and interpret outputs to diagnose and address system issues effectively. This guide also includes pro tips and best practices to enhance monitoring strategies, helping both new and experienced users maximize the utility of vmstat for system management and troubleshooting.

AWS Security Audit: From AWS CLI to a Go Security Scanner

Fri, 01 Mar 2024 07:00:00 +0000

Master AWS security auditing with CLI commands and Go. From checking IAM users and security groups to building a complete scanner that finds and reports misconfigurations.

Jenkins Security: Advisories, CVEs, and Hardening

Thu, 01 Feb 2024 09:00:00 +0000

Jenkins security evolution, the CVE-2024-23897 CLI vulnerability, and keeping plugins patched to shrink the attack surface.

Database Scaling: From 100K to 5M Users

Tue, 09 Jan 2024 02:01:00 +0000

A worked example of scaling a database from a single Postgres instance to a sharded architecture. Start simple, hit realistic bottlenecks (connection limits, replica lag, query timeouts), and fix each one. The numbers are illustrative; the progression is the real lesson.

Docker Logs: From docker logs to a Go Log Collector

Sun, 10 Sep 2023 20:01:00 +0000

Master Docker container logging with Linux commands and a real Go log collector. From docker logs basics to a streaming aggregator that processes multiple containers.

Advanced Bash Scripting for Automation