A URL shortener takes a long URL and gives back a short one. When someone visits the short URL, the server redirects them to the original. That is the whole idea. But building one teaches you TCP connections, HTTP headers, redirects, persistent storage, and analytics.
We will build this step by step. Each step starts with Linux commands so you can see the concept in your terminal. Then we write the Go equivalent. Along the way, we will make mistakes on purpose, see what breaks, and fix it.
You need a Linux terminal and Go installed. Nothing else.
Step 1: TCP Connections and HTTP Basics
Before writing any Go code, we need to understand what happens when a browser talks to a server. It all starts with TCP.
Linux: Listen on a Port
Open two terminal windows. In the first one, start a listener on port 8080.
nc -l 8080
This tells netcat to listen for a TCP connection on port 8080. It will wait until someone connects.
In the second terminal, send an HTTP request.
curl -v http://localhost:8080/test
Now look at the first terminal. You will see something like this.
GET /test HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.81.0
Accept: */*
That is a raw HTTP request. It is plain text sent over a TCP connection. The first line is the request line: the method (GET), the path (/test), and the protocol version. The lines after that are headers. The empty line at the end marks the end of the headers.
Type a response in the netcat terminal and press Enter.
HTTP/1.1 200 OK
Content-Type: text/plain
Hello from netcat
Press Ctrl+C to close the connection. In the curl terminal, you will see your response appear. HTTP is just text over TCP. Nothing magical.
Go: Build a TCP Server
Now let us do the same thing in Go. We will listen on a TCP port and read the raw bytes that arrive.
package main
import (
"fmt"
"net"
"os"
)
func main() {
listener, err := net.Listen("tcp", ":8080")
if err != nil {
fmt.Println("Error starting server:", err)
os.Exit(1)
}
defer listener.Close()
fmt.Println("Listening on :8080")
for {
conn, err := listener.Accept()
if err != nil {
fmt.Println("Error accepting connection:", err)
continue
}
go handleConnection(conn)
}
}
func handleConnection(conn net.Conn) {
defer conn.Close()
buf := make([]byte, 1024)
n, err := conn.Read(buf)
if err != nil {
fmt.Println("Error reading:", err)
return
}
fmt.Println("--- Received ---")
fmt.Println(string(buf[:n]))
response := "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\n\r\nHello from Go TCP server\n"
conn.Write([]byte(response))
}
Save this as main.go and run it.
go run main.go
In another terminal, test it.
curl -v http://localhost:8080/test
You will see the raw HTTP request printed in the server terminal, and curl will show the response. This is the same thing netcat did, but now your Go program controls everything.
The Bug: Missing POST Body
Try sending a POST request with a body.
curl -X POST -d '{"url":"https://example.com"}' http://localhost:8080/shorten
The server prints the request, but where is the body? Sometimes you see it. Sometimes you do not. The problem is our buffer. We read only 1024 bytes in a single Read call. The TCP stack might deliver the headers and body in separate packets. Our single Read call might only get the headers.
Look at the output carefully. You will see a Content-Length header in the request.
POST /shorten HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.81.0
Accept: */*
Content-Length: 28
Content-Type: application/x-www-form-urlencoded
The body is 28 bytes. But our code did not wait for it.
The Fix: Read the Content-Length
We need to parse the headers, find Content-Length, and then read exactly that many bytes for the body.
package main
import (
"bufio"
"fmt"
"net"
"os"
"strconv"
"strings"
)
func main() {
listener, err := net.Listen("tcp", ":8080")
if err != nil {
fmt.Println("Error starting server:", err)
os.Exit(1)
}
defer listener.Close()
fmt.Println("Listening on :8080")
for {
conn, err := listener.Accept()
if err != nil {
fmt.Println("Error accepting connection:", err)
continue
}
go handleConnection(conn)
}
}
func handleConnection(conn net.Conn) {
defer conn.Close()
reader := bufio.NewReader(conn)
// Read the request line
requestLine, err := reader.ReadString('\n')
if err != nil {
fmt.Println("Error reading request line:", err)
return
}
fmt.Print("Request: ", requestLine)
// Read headers
contentLength := 0
for {
line, err := reader.ReadString('\n')
if err != nil {
fmt.Println("Error reading header:", err)
return
}
line = strings.TrimSpace(line)
if line == "" {
break // Empty line means end of headers
}
fmt.Println("Header:", line)
if strings.HasPrefix(strings.ToLower(line), "content-length:") {
parts := strings.SplitN(line, ":", 2)
contentLength, _ = strconv.Atoi(strings.TrimSpace(parts[1]))
}
}
// Read the body if there is one
if contentLength > 0 {
body := make([]byte, contentLength)
bytesRead := 0
for bytesRead < contentLength {
n, err := reader.Read(body[bytesRead:])
if err != nil {
fmt.Println("Error reading body:", err)
return
}
bytesRead += n
}
fmt.Println("Body:", string(body))
}
response := "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\n\r\nReceived your request\n"
conn.Write([]byte(response))
}
Now run this version and send the POST request again.
curl -X POST -d '{"url":"https://example.com"}' http://localhost:8080/shorten
This time the server prints the body correctly. The key change: we use a buffered reader, parse headers line by line, find the Content-Length value, and then read exactly that many bytes. No more missing data.
This is how HTTP works underneath. Every web framework does this same parsing. Now that you understand it, we can move to the standard library and let it handle the details.
Step 2: HTTP Server with Routing
Now we need a real HTTP server with proper routing. Our URL shortener needs two endpoints: one to create short URLs and one to redirect.
Linux: Test HTTP Methods
Before writing the server, let us understand the HTTP methods we will use.
Create a short URL.
curl -X POST -d '{"url":"https://example.com"}' http://localhost:8080/shorten
Check a redirect without following it.
curl -I http://localhost:8080/abc123
The -I flag sends a HEAD request. You will see the response headers without the body. This is useful for checking if a redirect is set up correctly.
Follow the redirect.
curl -L http://localhost:8080/abc123
The -L flag tells curl to follow Location headers automatically. This is what browsers do.
Go: Two Routes with net/http
Let us build a proper HTTP server using the standard library.
package main
import (
"encoding/json"
"fmt"
"net/http"
"strings"
"sync"
)
type URLStore struct {
mu sync.RWMutex
urls map[string]string // short code -> original URL
}
var store = &URLStore{
urls: make(map[string]string),
}
type ShortenRequest struct {
URL string `json:"url"`
}
type ShortenResponse struct {
ShortURL string `json:"short_url"`
Code string `json:"code"`
}
func main() {
http.HandleFunc("/", handler)
fmt.Println("Server running on :8080")
err := http.ListenAndServe(":8080", nil)
if err != nil {
fmt.Println("Error:", err)
}
}
func handler(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/shorten" && r.Method == http.MethodPost {
handleShorten(w, r)
return
}
code := strings.TrimPrefix(r.URL.Path, "/")
if code != "" {
handleRedirect(w, r, code)
return
}
w.WriteHeader(http.StatusOK)
fmt.Fprintf(w, "URL Shortener is running\n")
}
func handleShorten(w http.ResponseWriter, r *http.Request) {
var req ShortenRequest
err := json.NewDecoder(r.Body).Decode(&req)
if err != nil || req.URL == "" {
http.Error(w, "Invalid request. Send JSON with a 'url' field.", http.StatusBadRequest)
return
}
code := "abc123" // Hardcoded for now. We fix this in Step 3.
store.mu.Lock()
store.urls[code] = req.URL
store.mu.Unlock()
resp := ShortenResponse{
ShortURL: fmt.Sprintf("http://localhost:8080/%s", code),
Code: code,
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(resp)
}
func handleRedirect(w http.ResponseWriter, r *http.Request, code string) {
store.mu.RLock()
originalURL, exists := store.urls[code]
store.mu.RUnlock()
if !exists {
http.Error(w, "Short URL not found", http.StatusNotFound)
return
}
http.Redirect(w, r, originalURL, http.StatusFound)
}
Run it and test.
go run main.go
Create a short URL.
curl -X POST -H "Content-Type: application/json" \
-d '{"url":"https://example.com"}' \
http://localhost:8080/shorten
You get back something like this.
{"short_url":"http://localhost:8080/abc123","code":"abc123"}
Now test the redirect.
curl -I http://localhost:8080/abc123
HTTP/1.1 302 Found
Location: https://example.com
The server sends back a 302 status with a Location header pointing to the original URL.
The Bug: No 404 for Unknown Paths
Try requesting a path that is not a short code and not /shorten.
curl -v http://localhost:8080/favicon.ico
HTTP/1.1 404 Not Found
That looks right. But try this.
curl -v http://localhost:8080/shorten/extra
The handler receives this request because the pattern / matches everything. The path /shorten/extra does not match /shorten exactly, so it falls through to the redirect logic. The code extracted is shorten/extra. That code does not exist, so you get a 404.
This seems fine on the surface, but it means any path that is not /shorten exactly will be treated as a short code lookup. What if someone requests /shorten?url=test with a GET method? It falls through to the redirect handler and tries to look up shorten as a code.
curl -v "http://localhost:8080/shorten?url=test"
HTTP/1.1 404 Not Found
Short URL not found
That is confusing. The user tried to shorten a URL using the wrong method, but got “Short URL not found” instead of a helpful error.
The Fix: Explicit Route Checking
func handler(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
if path == "/shorten" {
if r.Method != http.MethodPost {
http.Error(w, "Method not allowed. Use POST.", http.StatusMethodNotAllowed)
return
}
handleShorten(w, r)
return
}
if path == "/" {
w.WriteHeader(http.StatusOK)
fmt.Fprintf(w, "URL Shortener is running\n")
return
}
// Only treat single-segment paths as short codes
code := strings.TrimPrefix(path, "/")
if strings.Contains(code, "/") {
http.NotFound(w, r)
return
}
handleRedirect(w, r, code)
}
Now test the edge cases.
curl -v "http://localhost:8080/shorten?url=test"
HTTP/1.1 405 Method Not Allowed
Method not allowed. Use POST.
curl -v http://localhost:8080/shorten/extra
HTTP/1.1 404 Not Found
404 page not found
Each path gets a clear and correct response. The server only treats single-segment paths as short code lookups. Multi-segment paths get a 404. GET requests to /shorten get a 405 Method Not Allowed.
Step 3: Generating Short Codes
Right now every URL gets the code abc123. That is not useful. We need unique short codes.
Linux: Two Ways to Generate Codes
Hash-based. Take the URL, hash it, use the first few characters.
echo -n "https://example.com" | sha256sum | cut -c1-8
f0e6a6a3
The same URL always produces the same hash. That means the same URL always gets the same short code. This is useful if you want to avoid duplicates.
Random. Pull bytes from the system random source.
cat /dev/urandom | tr -dc 'a-zA-Z0-9' | head -c 8
kQ7mBx2p
Every run gives a different result. This is useful if you want different short codes even for the same URL.
Try running the hash command twice with the same URL.
echo -n "https://example.com" | sha256sum | cut -c1-8
echo -n "https://example.com" | sha256sum | cut -c1-8
You get f0e6a6a3 both times. Now run the random command twice.
cat /dev/urandom | tr -dc 'a-zA-Z0-9' | head -c 8 && echo
cat /dev/urandom | tr -dc 'a-zA-Z0-9' | head -c 8 && echo
Two different results. Each approach has tradeoffs. We will start with hash-based.
Go: Hash-Based Code Generation
package main
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"net/http"
"strings"
"sync"
)
type URLStore struct {
mu sync.RWMutex
urls map[string]string // short code -> original URL
}
var store = &URLStore{
urls: make(map[string]string),
}
type ShortenRequest struct {
URL string `json:"url"`
}
type ShortenResponse struct {
ShortURL string `json:"short_url"`
Code string `json:"code"`
}
func generateCode(url string) string {
hash := sha256.Sum256([]byte(url))
return hex.EncodeToString(hash[:])
}
func main() {
http.HandleFunc("/", handler)
fmt.Println("Server running on :8080")
http.ListenAndServe(":8080", nil)
}
func handler(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
if path == "/shorten" {
if r.Method != http.MethodPost {
http.Error(w, "Method not allowed. Use POST.", http.StatusMethodNotAllowed)
return
}
handleShorten(w, r)
return
}
if path == "/" {
fmt.Fprintf(w, "URL Shortener is running\n")
return
}
code := strings.TrimPrefix(path, "/")
if strings.Contains(code, "/") {
http.NotFound(w, r)
return
}
handleRedirect(w, r, code)
}
func handleShorten(w http.ResponseWriter, r *http.Request) {
var req ShortenRequest
err := json.NewDecoder(r.Body).Decode(&req)
if err != nil || req.URL == "" {
http.Error(w, "Invalid request. Send JSON with a 'url' field.", http.StatusBadRequest)
return
}
code := generateCode(req.URL)
store.mu.Lock()
store.urls[code] = req.URL
store.mu.Unlock()
resp := ShortenResponse{
ShortURL: fmt.Sprintf("http://localhost:8080/%s", code),
Code: code,
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(resp)
}
func handleRedirect(w http.ResponseWriter, r *http.Request, code string) {
store.mu.RLock()
originalURL, exists := store.urls[code]
store.mu.RUnlock()
if !exists {
http.Error(w, "Short URL not found", http.StatusNotFound)
return
}
http.Redirect(w, r, originalURL, http.StatusFound)
}
Test it.
curl -X POST -H "Content-Type: application/json" \
-d '{"url":"https://example.com"}' \
http://localhost:8080/shorten
{"short_url":"http://localhost:8080/f0e6a6a3a0d6...","code":"f0e6a6a3a0d6..."}
The Bug: Codes Are Too Long
The code is the full SHA256 hash. That is 64 hexadecimal characters. The whole point of a URL shortener is to make URLs short. A 64-character code defeats the purpose.
http://localhost:8080/f0e6a6a3a0d6b6c1e2d3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5
That is longer than most original URLs.
The Fix: Truncate and Handle Collisions
Take only the first 7 characters of the hash. But now two different URLs might produce the same 7-character prefix. We need to check for collisions.
func generateCode(url string) string {
hash := sha256.Sum256([]byte(url))
fullHash := hex.EncodeToString(hash[:])
return fullHash[:7]
}
func handleShorten(w http.ResponseWriter, r *http.Request) {
var req ShortenRequest
err := json.NewDecoder(r.Body).Decode(&req)
if err != nil || req.URL == "" {
http.Error(w, "Invalid request. Send JSON with a 'url' field.", http.StatusBadRequest)
return
}
code := generateCode(req.URL)
store.mu.Lock()
// Check for collision: same code but different URL
if existingURL, exists := store.urls[code]; exists && existingURL != req.URL {
// Collision. Try longer prefixes until we find a unique one.
hash := sha256.Sum256([]byte(req.URL))
fullHash := hex.EncodeToString(hash[:])
for length := 8; length <= len(fullHash); length++ {
code = fullHash[:length]
if existing, exists := store.urls[code]; !exists || existing == req.URL {
break
}
}
}
store.urls[code] = req.URL
store.mu.Unlock()
resp := ShortenResponse{
ShortURL: fmt.Sprintf("http://localhost:8080/%s", code),
Code: code,
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(resp)
}
Now the same URL always returns the same short code. Different URLs get different codes. If two URLs happen to share the same 7-character prefix, the code grows longer until it is unique.
Test it.
curl -X POST -H "Content-Type: application/json" \
-d '{"url":"https://example.com"}' \
http://localhost:8080/shorten
{"short_url":"http://localhost:8080/f0e6a6a","code":"f0e6a6a"}
Seven characters. Short and clean.
Submit the same URL again.
curl -X POST -H "Content-Type: application/json" \
-d '{"url":"https://example.com"}' \
http://localhost:8080/shorten
Same code: f0e6a6a. No duplicates.
Step 4: Storing URLs Persistently
Right now all URLs are stored in memory. Restart the server and they are gone. We need persistent storage.
Linux: File-Based Storage
The simplest storage is a text file. One line per URL mapping.
echo "f0e6a6a https://example.com" >> urls.txt
echo "b3c8d9e https://golang.org" >> urls.txt
Look up a code.
grep "^f0e6a6a " urls.txt | cut -d' ' -f2-
https://example.com
The ^ anchors the match to the start of the line. The space after the code ensures we match the whole code and not a prefix. The cut command extracts everything after the first space.
Check how fast this is with many entries. Generate 10,000 lines.
for i in $(seq 1 10000); do
echo "code${i} https://example.com/${i}" >> urls_big.txt
done
Now search for the last entry.
time grep "^code10000 " urls_big.txt | cut -d' ' -f2-
It is fast for 10,000 entries. But grep scans the entire file from top to bottom. With millions of entries, every lookup reads the whole file. That gets slow.
Go: File-Based Store with In-Memory Lookup
We will write URLs to a file for persistence but keep them in a map in memory for fast lookups. On startup, we load the file into the map.
package main
import (
"bufio"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"net/http"
"os"
"strings"
"sync"
)
const dataFile = "urls.txt"
type URLStore struct {
mu sync.RWMutex
urls map[string]string
file *os.File
}
type ShortenRequest struct {
URL string `json:"url"`
}
type ShortenResponse struct {
ShortURL string `json:"short_url"`
Code string `json:"code"`
}
var store *URLStore
func newURLStore(filename string) (*URLStore, error) {
s := &URLStore{
urls: make(map[string]string),
}
// Load existing URLs from file
if _, err := os.Stat(filename); err == nil {
file, err := os.Open(filename)
if err != nil {
return nil, fmt.Errorf("cannot open data file: %w", err)
}
scanner := bufio.NewScanner(file)
count := 0
for scanner.Scan() {
line := scanner.Text()
parts := strings.SplitN(line, " ", 2)
if len(parts) == 2 {
s.urls[parts[0]] = parts[1]
count++
}
}
file.Close()
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("error reading data file: %w", err)
}
fmt.Printf("Loaded %d URLs from %s\n", count, filename)
}
// Open file for appending new URLs
file, err := os.OpenFile(filename, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
return nil, fmt.Errorf("cannot open data file for writing: %w", err)
}
s.file = file
return s, nil
}
func (s *URLStore) Save(code, url string) error {
s.mu.Lock()
defer s.mu.Unlock()
s.urls[code] = url
_, err := fmt.Fprintf(s.file, "%s %s\n", code, url)
if err != nil {
return fmt.Errorf("failed to write to file: %w", err)
}
return s.file.Sync()
}
func (s *URLStore) Get(code string) (string, bool) {
s.mu.RLock()
defer s.mu.RUnlock()
url, exists := s.urls[code]
return url, exists
}
func (s *URLStore) Exists(code string) (string, bool) {
s.mu.RLock()
defer s.mu.RUnlock()
url, exists := s.urls[code]
return url, exists
}
func generateCode(url string) string {
hash := sha256.Sum256([]byte(url))
fullHash := hex.EncodeToString(hash[:])
return fullHash[:7]
}
func main() {
var err error
store, err = newURLStore(dataFile)
if err != nil {
fmt.Println("Error initializing store:", err)
os.Exit(1)
}
defer store.file.Close()
http.HandleFunc("/", handler)
fmt.Println("Server running on :8080")
err = http.ListenAndServe(":8080", nil)
if err != nil {
fmt.Println("Error:", err)
}
}
func handler(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
if path == "/shorten" {
if r.Method != http.MethodPost {
http.Error(w, "Method not allowed. Use POST.", http.StatusMethodNotAllowed)
return
}
handleShorten(w, r)
return
}
if path == "/" {
fmt.Fprintf(w, "URL Shortener is running\n")
return
}
code := strings.TrimPrefix(path, "/")
if strings.Contains(code, "/") {
http.NotFound(w, r)
return
}
handleRedirect(w, r, code)
}
func handleShorten(w http.ResponseWriter, r *http.Request) {
var req ShortenRequest
err := json.NewDecoder(r.Body).Decode(&req)
if err != nil || req.URL == "" {
http.Error(w, "Invalid request. Send JSON with a 'url' field.", http.StatusBadRequest)
return
}
code := generateCode(req.URL)
// Check for collision
if existingURL, exists := store.Exists(code); exists && existingURL != req.URL {
hash := sha256.Sum256([]byte(req.URL))
fullHash := hex.EncodeToString(hash[:])
for length := 8; length <= len(fullHash); length++ {
code = fullHash[:length]
if existing, exists := store.Exists(code); !exists || existing == req.URL {
break
}
}
}
// Check if this URL was already shortened
if existingURL, exists := store.Exists(code); exists && existingURL == req.URL {
resp := ShortenResponse{
ShortURL: fmt.Sprintf("http://localhost:8080/%s", code),
Code: code,
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(resp)
return
}
err = store.Save(code, req.URL)
if err != nil {
http.Error(w, "Failed to save URL", http.StatusInternalServerError)
return
}
resp := ShortenResponse{
ShortURL: fmt.Sprintf("http://localhost:8080/%s", code),
Code: code,
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(resp)
}
func handleRedirect(w http.ResponseWriter, r *http.Request, code string) {
originalURL, exists := store.Get(code)
if !exists {
http.Error(w, "Short URL not found", http.StatusNotFound)
return
}
http.Redirect(w, r, originalURL, http.StatusFound)
}
Run it and create some URLs.
go run main.go
curl -X POST -H "Content-Type: application/json" \
-d '{"url":"https://example.com"}' \
http://localhost:8080/shorten
curl -X POST -H "Content-Type: application/json" \
-d '{"url":"https://golang.org"}' \
http://localhost:8080/shorten
Check the data file.
cat urls.txt
f0e6a6a https://example.com
7e0e78c https://golang.org
Now stop the server with Ctrl+C and start it again.
go run main.go
Loaded 2 URLs from urls.txt
Server running on :8080
Test that the old codes still work.
curl -I http://localhost:8080/f0e6a6a
HTTP/1.1 302 Found
Location: https://example.com
The URLs survived the restart. The file gives us persistence. The map gives us fast lookups.
The Bug: Reading the Whole File on Every Lookup
Look at how we would have done it without the in-memory map. A naive approach might read the file on every request.
// BAD: Do not do this
func getURLFromFile(code string) (string, bool) {
file, err := os.Open(dataFile)
if err != nil {
return "", false
}
defer file.Close()
scanner := bufio.NewScanner(file)
for scanner.Scan() {
parts := strings.SplitN(scanner.Text(), " ", 2)
if len(parts) == 2 && parts[0] == code {
return parts[1], true
}
}
return "", false
}
This scans the entire file on every request. With 100,000 URLs, every redirect request reads 100,000 lines. If you get 100 requests per second, that is 10 million line reads per second. The server would crawl.
The Fix
We already fixed this in our implementation above. Load the file into a map on startup. Serve lookups from the map. Write new entries to the file for persistence. The map gives us O(1) lookup time. The file gives us durability.
This is the same pattern that databases like Redis use: serve reads from memory, persist writes to disk.
Step 5: HTTP Redirects and Click Analytics
The core feature of a URL shortener is the redirect. But we also want to know how many times each short URL is used.
Linux: Understanding Redirects
Let us see what a redirect looks like at the HTTP level.
curl -o /dev/null -s -w "Status: %{http_code}\nRedirect: %{redirect_url}\n" http://localhost:8080/f0e6a6a
Status: 302
Redirect: https://example.com
The -o /dev/null discards the body. The -s silences the progress bar. The -w flag formats the output to show just the status code and redirect URL.
There are two types of redirects in HTTP.
301 Moved Permanently. The browser caches this. Next time you visit the short URL, the browser goes straight to the destination without asking your server.
302 Found. The browser does not cache this. Every visit goes through your server first.
You can see the difference with curl.
curl -v -L http://localhost:8080/f0e6a6a
The -v flag shows headers. The -L flag follows redirects. You will see the 302 response with the Location header, then the final 200 response from the destination.
Go: Redirects with Analytics
We need to track every redirect. Add a click counter and timestamp tracking.
package main
import (
"bufio"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"net/http"
"os"
"strings"
"sync"
"time"
)
const dataFile = "urls.txt"
type ClickRecord struct {
Timestamp time.Time `json:"timestamp"`
UserAgent string `json:"user_agent"`
IP string `json:"ip"`
}
type URLEntry struct {
OriginalURL string
Clicks []ClickRecord
CreatedAt time.Time
}
type URLStore struct {
mu sync.RWMutex
entries map[string]*URLEntry
file *os.File
}
type ShortenRequest struct {
URL string `json:"url"`
}
type ShortenResponse struct {
ShortURL string `json:"short_url"`
Code string `json:"code"`
}
type StatsResponse struct {
Code string `json:"code"`
OriginalURL string `json:"original_url"`
ClickCount int `json:"click_count"`
CreatedAt time.Time `json:"created_at"`
LastClicked string `json:"last_clicked"`
}
var store *URLStore
func newURLStore(filename string) (*URLStore, error) {
s := &URLStore{
entries: make(map[string]*URLEntry),
}
if _, err := os.Stat(filename); err == nil {
file, err := os.Open(filename)
if err != nil {
return nil, fmt.Errorf("cannot open data file: %w", err)
}
scanner := bufio.NewScanner(file)
count := 0
for scanner.Scan() {
line := scanner.Text()
parts := strings.SplitN(line, " ", 2)
if len(parts) == 2 {
s.entries[parts[0]] = &URLEntry{
OriginalURL: parts[1],
Clicks: []ClickRecord{},
CreatedAt: time.Now(),
}
count++
}
}
file.Close()
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("error reading data file: %w", err)
}
fmt.Printf("Loaded %d URLs from %s\n", count, filename)
}
file, err := os.OpenFile(filename, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
return nil, fmt.Errorf("cannot open data file for writing: %w", err)
}
s.file = file
return s, nil
}
func (s *URLStore) Save(code, url string) error {
s.mu.Lock()
defer s.mu.Unlock()
s.entries[code] = &URLEntry{
OriginalURL: url,
Clicks: []ClickRecord{},
CreatedAt: time.Now(),
}
_, err := fmt.Fprintf(s.file, "%s %s\n", code, url)
if err != nil {
return fmt.Errorf("failed to write to file: %w", err)
}
return s.file.Sync()
}
func (s *URLStore) Get(code string) (*URLEntry, bool) {
s.mu.RLock()
defer s.mu.RUnlock()
entry, exists := s.entries[code]
return entry, exists
}
func (s *URLStore) RecordClick(code string, r *http.Request) {
s.mu.Lock()
defer s.mu.Unlock()
entry, exists := s.entries[code]
if !exists {
return
}
click := ClickRecord{
Timestamp: time.Now(),
UserAgent: r.UserAgent(),
IP: r.RemoteAddr,
}
entry.Clicks = append(entry.Clicks, click)
}
func generateCode(url string) string {
hash := sha256.Sum256([]byte(url))
fullHash := hex.EncodeToString(hash[:])
return fullHash[:7]
}
func main() {
var err error
store, err = newURLStore(dataFile)
if err != nil {
fmt.Println("Error initializing store:", err)
os.Exit(1)
}
defer store.file.Close()
http.HandleFunc("/", handler)
fmt.Println("Server running on :8080")
err = http.ListenAndServe(":8080", nil)
if err != nil {
fmt.Println("Error:", err)
}
}
func handler(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
if path == "/shorten" {
if r.Method != http.MethodPost {
http.Error(w, "Method not allowed. Use POST.", http.StatusMethodNotAllowed)
return
}
handleShorten(w, r)
return
}
if path == "/" {
fmt.Fprintf(w, "URL Shortener is running\n")
return
}
code := strings.TrimPrefix(path, "/")
if strings.Contains(code, "/") {
// Check for /stats/{code} pattern
if strings.HasPrefix(code, "stats/") {
statsCode := strings.TrimPrefix(code, "stats/")
if !strings.Contains(statsCode, "/") {
handleStats(w, r, statsCode)
return
}
}
http.NotFound(w, r)
return
}
handleRedirect(w, r, code)
}
func handleShorten(w http.ResponseWriter, r *http.Request) {
var req ShortenRequest
err := json.NewDecoder(r.Body).Decode(&req)
if err != nil || req.URL == "" {
http.Error(w, "Invalid request. Send JSON with a 'url' field.", http.StatusBadRequest)
return
}
code := generateCode(req.URL)
// Check if already exists with same URL
if entry, exists := store.Get(code); exists && entry.OriginalURL == req.URL {
resp := ShortenResponse{
ShortURL: fmt.Sprintf("http://localhost:8080/%s", code),
Code: code,
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(resp)
return
}
// Check for collision
if entry, exists := store.Get(code); exists && entry.OriginalURL != req.URL {
hash := sha256.Sum256([]byte(req.URL))
fullHash := hex.EncodeToString(hash[:])
for length := 8; length <= len(fullHash); length++ {
code = fullHash[:length]
if e, exists := store.Get(code); !exists || e.OriginalURL == req.URL {
break
}
}
}
err = store.Save(code, req.URL)
if err != nil {
http.Error(w, "Failed to save URL", http.StatusInternalServerError)
return
}
resp := ShortenResponse{
ShortURL: fmt.Sprintf("http://localhost:8080/%s", code),
Code: code,
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(resp)
}
func handleRedirect(w http.ResponseWriter, r *http.Request, code string) {
entry, exists := store.Get(code)
if !exists {
http.Error(w, "Short URL not found", http.StatusNotFound)
return
}
store.RecordClick(code, r)
http.Redirect(w, r, entry.OriginalURL, http.StatusMovedPermanently)
}
func handleStats(w http.ResponseWriter, r *http.Request, code string) {
entry, exists := store.Get(code)
if !exists {
http.Error(w, "Short URL not found", http.StatusNotFound)
return
}
lastClicked := "never"
if len(entry.Clicks) > 0 {
lastClicked = entry.Clicks[len(entry.Clicks)-1].Timestamp.Format(time.RFC3339)
}
stats := StatsResponse{
Code: code,
OriginalURL: entry.OriginalURL,
ClickCount: len(entry.Clicks),
CreatedAt: entry.CreatedAt,
LastClicked: lastClicked,
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(stats)
}
Test the analytics.
go run main.go
Create a URL and click it a few times.
curl -X POST -H "Content-Type: application/json" \
-d '{"url":"https://example.com"}' \
http://localhost:8080/shorten
curl -s -o /dev/null http://localhost:8080/f0e6a6a
curl -s -o /dev/null http://localhost:8080/f0e6a6a
curl -s -o /dev/null http://localhost:8080/f0e6a6a
Now check the stats.
curl -s http://localhost:8080/stats/f0e6a6a | python3 -m json.tool
{
"code": "f0e6a6a",
"original_url": "https://example.com",
"click_count": 3,
"created_at": "2024-01-15T10:30:00Z",
"last_clicked": "2024-01-15T10:30:05Z"
}
Three clicks recorded. The stats endpoint gives us click count, creation time, and last click time.
The Bug: 301 Permanent Redirect Kills Analytics
Look at the redirect handler. We used http.StatusMovedPermanently (301). Test this with curl’s verbose mode.
curl -v -L http://localhost:8080/f0e6a6a
< HTTP/1.1 301 Moved Permanently
< Location: https://example.com
That 301 tells the browser: “This URL has permanently moved. Cache this redirect and never ask me again.”
After the first visit, the browser goes straight to https://example.com without ever contacting your server. Your analytics counter stops incrementing because no requests reach your server.
To see this, open the short URL in a browser. Then check stats. Click count will be 1. Click the link again in the same browser. Click count stays at 1. The browser cached the redirect and bypassed your server.
The Fix: Use 302 Temporary Redirect
Change the redirect status to 302. This tells the browser: “Go to this URL, but ask me again next time.”
func handleRedirect(w http.ResponseWriter, r *http.Request, code string) {
entry, exists := store.Get(code)
if !exists {
http.Error(w, "Short URL not found", http.StatusNotFound)
return
}
store.RecordClick(code, r)
http.Redirect(w, r, entry.OriginalURL, http.StatusFound) // 302, not 301
}
Now every click goes through your server first. Your analytics are accurate.
The tradeoff: 302 redirects are slightly slower because the browser always contacts your server first. For a URL shortener, this is the right choice. You want accurate click counts more than you want to save a few milliseconds.
Step 6: Complete URL Shortener with Dashboard
Now we combine everything into the final service. We add rate limiting, colored terminal logging, and a stats dashboard.
Rate Limiting with Token Buckets
Before building the final version, understand the concept. A token bucket allows a fixed number of actions in a time window. Each IP address gets a bucket. Every request removes a token. Tokens refill over time.
# Simulate: send 15 requests rapidly to see rate limiting
for i in $(seq 1 15); do
STATUS=$(curl -s -o /dev/null -w "%{http_code}" -X POST \
-H "Content-Type: application/json" \
-d "{\"url\":\"https://example.com/$i\"}" \
http://localhost:8080/shorten)
echo "Request $i: HTTP $STATUS"
done
After the 10th request, you should see HTTP 429 (Too Many Requests).
The Complete Service
Here is the full URL shortener with all features combined.
package main
import (
"bufio"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"net/http"
"os"
"strings"
"sync"
"time"
)
// --- ANSI color codes for terminal output ---
const (
colorReset = "\033[0m"
colorRed = "\033[31m"
colorGreen = "\033[32m"
colorYellow = "\033[33m"
colorBlue = "\033[34m"
colorCyan = "\033[36m"
colorGray = "\033[90m"
)
// --- Data types ---
const dataFile = "urls.txt"
type ClickRecord struct {
Timestamp time.Time `json:"timestamp"`
UserAgent string `json:"user_agent"`
IP string `json:"ip"`
Referrer string `json:"referrer"`
}
type URLEntry struct {
OriginalURL string
Clicks []ClickRecord
CreatedAt time.Time
}
type URLStore struct {
mu sync.RWMutex
entries map[string]*URLEntry
file *os.File
}
type ShortenRequest struct {
URL string `json:"url"`
}
type ShortenResponse struct {
ShortURL string `json:"short_url"`
Code string `json:"code"`
}
type StatsResponse struct {
Code string `json:"code"`
OriginalURL string `json:"original_url"`
ClickCount int `json:"click_count"`
CreatedAt time.Time `json:"created_at"`
LastClicked string `json:"last_clicked"`
TopReferrers map[string]int `json:"top_referrers"`
RecentClicks []ClickSummary `json:"recent_clicks"`
}
type ClickSummary struct {
Timestamp string `json:"timestamp"`
UserAgent string `json:"user_agent"`
}
// --- Rate limiter ---
type RateLimiter struct {
mu sync.Mutex
buckets map[string]*Bucket
}
type Bucket struct {
tokens int
lastReset time.Time
}
const (
maxTokens = 10
resetPeriod = time.Minute
)
func newRateLimiter() *RateLimiter {
return &RateLimiter{
buckets: make(map[string]*Bucket),
}
}
func (rl *RateLimiter) Allow(ip string) bool {
rl.mu.Lock()
defer rl.mu.Unlock()
bucket, exists := rl.buckets[ip]
if !exists {
rl.buckets[ip] = &Bucket{
tokens: maxTokens - 1,
lastReset: time.Now(),
}
return true
}
// Refill tokens if the reset period has passed
if time.Since(bucket.lastReset) > resetPeriod {
bucket.tokens = maxTokens
bucket.lastReset = time.Now()
}
if bucket.tokens <= 0 {
return false
}
bucket.tokens--
return true
}
// --- URL Store ---
var store *URLStore
var limiter *RateLimiter
func newURLStore(filename string) (*URLStore, error) {
s := &URLStore{
entries: make(map[string]*URLEntry),
}
if _, err := os.Stat(filename); err == nil {
file, err := os.Open(filename)
if err != nil {
return nil, fmt.Errorf("cannot open data file: %w", err)
}
scanner := bufio.NewScanner(file)
count := 0
for scanner.Scan() {
line := scanner.Text()
parts := strings.SplitN(line, " ", 2)
if len(parts) == 2 {
s.entries[parts[0]] = &URLEntry{
OriginalURL: parts[1],
Clicks: []ClickRecord{},
CreatedAt: time.Now(),
}
count++
}
}
file.Close()
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("error reading data file: %w", err)
}
logInfo("Loaded %d URLs from %s", count, filename)
}
file, err := os.OpenFile(filename, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
return nil, fmt.Errorf("cannot open data file for writing: %w", err)
}
s.file = file
return s, nil
}
func (s *URLStore) Save(code, url string) error {
s.mu.Lock()
defer s.mu.Unlock()
s.entries[code] = &URLEntry{
OriginalURL: url,
Clicks: []ClickRecord{},
CreatedAt: time.Now(),
}
_, err := fmt.Fprintf(s.file, "%s %s\n", code, url)
if err != nil {
return fmt.Errorf("failed to write to file: %w", err)
}
return s.file.Sync()
}
func (s *URLStore) Get(code string) (*URLEntry, bool) {
s.mu.RLock()
defer s.mu.RUnlock()
entry, exists := s.entries[code]
return entry, exists
}
func (s *URLStore) RecordClick(code string, r *http.Request) {
s.mu.Lock()
defer s.mu.Unlock()
entry, exists := s.entries[code]
if !exists {
return
}
click := ClickRecord{
Timestamp: time.Now(),
UserAgent: r.UserAgent(),
IP: r.RemoteAddr,
Referrer: r.Referer(),
}
entry.Clicks = append(entry.Clicks, click)
}
func (s *URLStore) Count() int {
s.mu.RLock()
defer s.mu.RUnlock()
return len(s.entries)
}
// --- Code generation ---
func generateCode(url string) string {
hash := sha256.Sum256([]byte(url))
fullHash := hex.EncodeToString(hash[:])
return fullHash[:7]
}
func resolveCode(url string) string {
code := generateCode(url)
if entry, exists := store.Get(code); exists && entry.OriginalURL != url {
hash := sha256.Sum256([]byte(url))
fullHash := hex.EncodeToString(hash[:])
for length := 8; length <= len(fullHash); length++ {
code = fullHash[:length]
if e, exists := store.Get(code); !exists || e.OriginalURL == url {
break
}
}
}
return code
}
// --- Colored logging ---
func logInfo(format string, args ...interface{}) {
timestamp := time.Now().Format("15:04:05")
msg := fmt.Sprintf(format, args...)
fmt.Printf("%s[%s]%s %s%s%s\n", colorGray, timestamp, colorReset, colorCyan, msg, colorReset)
}
func logRedirect(code, destination, ip string) {
timestamp := time.Now().Format("15:04:05")
fmt.Printf("%s[%s]%s %sREDIRECT%s %s/%s%s -> %s%s%s from %s\n",
colorGray, timestamp, colorReset,
colorGreen, colorReset,
colorYellow, code, colorReset,
colorBlue, destination, colorReset,
ip,
)
}
func logCreate(code, url string) {
timestamp := time.Now().Format("15:04:05")
fmt.Printf("%s[%s]%s %sCREATE%s %s/%s%s -> %s%s%s\n",
colorGray, timestamp, colorReset,
colorGreen, colorReset,
colorYellow, code, colorReset,
colorBlue, url, colorReset,
)
}
func logRateLimit(ip string) {
timestamp := time.Now().Format("15:04:05")
fmt.Printf("%s[%s]%s %sRATE LIMIT%s %s%s%s\n",
colorGray, timestamp, colorReset,
colorRed, colorReset,
colorYellow, ip, colorReset,
)
}
func logRequest(method, path string, status int) {
timestamp := time.Now().Format("15:04:05")
statusColor := colorGreen
if status >= 400 {
statusColor = colorRed
} else if status >= 300 {
statusColor = colorYellow
}
fmt.Printf("%s[%s]%s %s%d%s %s %s\n",
colorGray, timestamp, colorReset,
statusColor, status, colorReset,
method, path,
)
}
// --- Extracting client IP ---
func getClientIP(r *http.Request) string {
// Check X-Forwarded-For header first (for proxied requests)
forwarded := r.Header.Get("X-Forwarded-For")
if forwarded != "" {
parts := strings.SplitN(forwarded, ",", 2)
return strings.TrimSpace(parts[0])
}
// Fall back to RemoteAddr
ip := r.RemoteAddr
// Remove port number
if colonIdx := strings.LastIndex(ip, ":"); colonIdx != -1 {
ip = ip[:colonIdx]
}
return ip
}
// --- HTTP handlers ---
func main() {
var err error
store, err = newURLStore(dataFile)
if err != nil {
fmt.Println("Error initializing store:", err)
os.Exit(1)
}
defer store.file.Close()
limiter = newRateLimiter()
http.HandleFunc("/", handler)
logInfo("URL Shortener running on :8080")
logInfo("Endpoints: POST /shorten, GET /{code}, GET /stats/{code}")
logInfo("Stored URLs: %d", store.Count())
err = http.ListenAndServe(":8080", nil)
if err != nil {
fmt.Println("Error:", err)
}
}
func handler(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
if path == "/shorten" {
if r.Method != http.MethodPost {
logRequest(r.Method, path, http.StatusMethodNotAllowed)
http.Error(w, "Method not allowed. Use POST.", http.StatusMethodNotAllowed)
return
}
handleShorten(w, r)
return
}
if path == "/" {
logRequest(r.Method, path, http.StatusOK)
w.Header().Set("Content-Type", "text/plain")
fmt.Fprintf(w, "URL Shortener is running. POST to /shorten to create a short URL.\n")
return
}
trimmed := strings.TrimPrefix(path, "/")
// Check for /stats/{code}
if strings.HasPrefix(trimmed, "stats/") {
statsCode := strings.TrimPrefix(trimmed, "stats/")
if statsCode != "" && !strings.Contains(statsCode, "/") {
handleStats(w, r, statsCode)
return
}
}
// Only single-segment paths are short codes
if strings.Contains(trimmed, "/") {
logRequest(r.Method, path, http.StatusNotFound)
http.NotFound(w, r)
return
}
handleRedirect(w, r, trimmed)
}
func handleShorten(w http.ResponseWriter, r *http.Request) {
clientIP := getClientIP(r)
// Rate limiting
if !limiter.Allow(clientIP) {
logRateLimit(clientIP)
http.Error(w, "Rate limit exceeded. Max 10 URLs per minute.", http.StatusTooManyRequests)
return
}
var req ShortenRequest
err := json.NewDecoder(r.Body).Decode(&req)
if err != nil || req.URL == "" {
logRequest(r.Method, "/shorten", http.StatusBadRequest)
http.Error(w, "Invalid request. Send JSON with a 'url' field.", http.StatusBadRequest)
return
}
// Basic URL validation
if !strings.HasPrefix(req.URL, "http://") && !strings.HasPrefix(req.URL, "https://") {
logRequest(r.Method, "/shorten", http.StatusBadRequest)
http.Error(w, "URL must start with http:// or https://", http.StatusBadRequest)
return
}
code := resolveCode(req.URL)
// Check if already exists with same URL
if entry, exists := store.Get(code); exists && entry.OriginalURL == req.URL {
resp := ShortenResponse{
ShortURL: fmt.Sprintf("http://localhost:8080/%s", code),
Code: code,
}
logRequest(r.Method, "/shorten", http.StatusOK)
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(resp)
return
}
err = store.Save(code, req.URL)
if err != nil {
logRequest(r.Method, "/shorten", http.StatusInternalServerError)
http.Error(w, "Failed to save URL", http.StatusInternalServerError)
return
}
logCreate(code, req.URL)
resp := ShortenResponse{
ShortURL: fmt.Sprintf("http://localhost:8080/%s", code),
Code: code,
}
logRequest(r.Method, "/shorten", http.StatusCreated)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusCreated)
json.NewEncoder(w).Encode(resp)
}
func handleRedirect(w http.ResponseWriter, r *http.Request, code string) {
entry, exists := store.Get(code)
if !exists {
logRequest(r.Method, "/"+code, http.StatusNotFound)
http.Error(w, "Short URL not found", http.StatusNotFound)
return
}
store.RecordClick(code, r)
logRedirect(code, entry.OriginalURL, getClientIP(r))
http.Redirect(w, r, entry.OriginalURL, http.StatusFound) // 302 temporary redirect
}
func handleStats(w http.ResponseWriter, r *http.Request, code string) {
entry, exists := store.Get(code)
if !exists {
logRequest(r.Method, "/stats/"+code, http.StatusNotFound)
http.Error(w, "Short URL not found", http.StatusNotFound)
return
}
store.mu.RLock()
clicks := entry.Clicks
store.mu.RUnlock()
lastClicked := "never"
if len(clicks) > 0 {
lastClicked = clicks[len(clicks)-1].Timestamp.Format(time.RFC3339)
}
// Count referrers
referrers := make(map[string]int)
for _, click := range clicks {
ref := click.Referrer
if ref == "" {
ref = "direct"
}
referrers[ref]++
}
// Get recent clicks (last 10)
recent := []ClickSummary{}
start := 0
if len(clicks) > 10 {
start = len(clicks) - 10
}
for _, click := range clicks[start:] {
recent = append(recent, ClickSummary{
Timestamp: click.Timestamp.Format(time.RFC3339),
UserAgent: click.UserAgent,
})
}
stats := StatsResponse{
Code: code,
OriginalURL: entry.OriginalURL,
ClickCount: len(clicks),
CreatedAt: entry.CreatedAt,
LastClicked: lastClicked,
TopReferrers: referrers,
RecentClicks: recent,
}
logRequest(r.Method, "/stats/"+code, http.StatusOK)
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(stats)
}
Running the Complete Service
Start the server.
go run main.go
You will see colored output in your terminal.
[10:30:00] URL Shortener running on :8080
[10:30:00] Endpoints: POST /shorten, GET /{code}, GET /stats/{code}
[10:30:00] Stored URLs: 0
Create a short URL.
curl -s -X POST -H "Content-Type: application/json" \
-d '{"url":"https://go.dev/doc/"}' \
http://localhost:8080/shorten | python3 -m json.tool
{
"short_url": "http://localhost:8080/a1b2c3d",
"code": "a1b2c3d"
}
The server terminal shows.
[10:30:05] CREATE /a1b2c3d -> https://go.dev/doc/
[10:30:05] 201 POST /shorten
Click the short URL a few times.
curl -s -o /dev/null http://localhost:8080/a1b2c3d
curl -s -o /dev/null http://localhost:8080/a1b2c3d
curl -s -o /dev/null http://localhost:8080/a1b2c3d
Each redirect appears in the terminal with color.
[10:30:10] REDIRECT /a1b2c3d -> https://go.dev/doc/ from 127.0.0.1
[10:30:11] REDIRECT /a1b2c3d -> https://go.dev/doc/ from 127.0.0.1
[10:30:12] REDIRECT /a1b2c3d -> https://go.dev/doc/ from 127.0.0.1
Check the stats.
curl -s http://localhost:8080/stats/a1b2c3d | python3 -m json.tool
{
"code": "a1b2c3d",
"original_url": "https://go.dev/doc/",
"click_count": 3,
"created_at": "2024-01-15T10:30:05Z",
"last_clicked": "2024-01-15T10:30:12Z",
"top_referrers": {
"direct": 3
},
"recent_clicks": [
{
"timestamp": "2024-01-15T10:30:10Z",
"user_agent": "curl/7.81.0"
},
{
"timestamp": "2024-01-15T10:30:11Z",
"user_agent": "curl/7.81.0"
},
{
"timestamp": "2024-01-15T10:30:12Z",
"user_agent": "curl/7.81.0"
}
]
}
Testing Rate Limiting
Send many requests quickly.
for i in $(seq 1 15); do
STATUS=$(curl -s -o /dev/null -w "%{http_code}" -X POST \
-H "Content-Type: application/json" \
-d "{\"url\":\"https://example.com/$i\"}" \
http://localhost:8080/shorten)
echo "Request $i: HTTP $STATUS"
done
Request 1: HTTP 201
Request 2: HTTP 201
Request 3: HTTP 201
Request 4: HTTP 201
Request 5: HTTP 201
Request 6: HTTP 201
Request 7: HTTP 201
Request 8: HTTP 201
Request 9: HTTP 201
Request 10: HTTP 201
Request 11: HTTP 429
Request 12: HTTP 429
Request 13: HTTP 429
Request 14: HTTP 429
Request 15: HTTP 429
The first 10 requests succeed. After that, the rate limiter kicks in and returns 429 Too Many Requests. The server terminal shows rate limit events in red.
[10:31:00] RATE LIMIT 127.0.0.1
Wait one minute and try again. The tokens refill and requests succeed.
Testing URL Validation
Try submitting an invalid URL.
curl -s -X POST -H "Content-Type: application/json" \
-d '{"url":"not-a-url"}' \
http://localhost:8080/shorten
URL must start with http:// or https://
Try an empty request.
curl -s -X POST -H "Content-Type: application/json" \
-d '{}' \
http://localhost:8080/shorten
Invalid request. Send JSON with a 'url' field.
The server validates input before processing.
Testing Persistence
Stop the server with Ctrl+C. Check the data file.
cat urls.txt
You will see all the URLs you created. Start the server again.
go run main.go
[10:35:00] Loaded 10 URLs from urls.txt
[10:35:00] URL Shortener running on :8080
[10:35:00] Endpoints: POST /shorten, GET /{code}, GET /stats/{code}
[10:35:00] Stored URLs: 10
All URLs are loaded from disk. The short codes still work. Click analytics are reset on restart because they are only stored in memory. If you need persistent analytics, you would write click events to a separate file or a database.
How the Pieces Fit Together
Here is the complete request flow for each operation.
Creating a short URL:
- Client sends POST /shorten with a JSON body containing the URL
- Rate limiter checks if the client IP has tokens remaining
- Server validates the URL format
- Server generates a 7-character hash code from the URL
- Server checks for hash collisions and extends the code if needed
- Server writes the mapping to urls.txt and stores it in memory
- Server returns the short URL as JSON
Redirecting:
- Client sends GET /{code}
- Server looks up the code in the in-memory map
- Server records a click with timestamp, User-Agent, IP, and referrer
- Server returns a 302 redirect with the Location header set to the original URL
- Client follows the redirect to the destination
Viewing stats:
- Client sends GET /stats/{code}
- Server looks up the code and its click history
- Server calculates referrer counts and formats recent clicks
- Server returns JSON with click count, timestamps, and referrer data
Every step in this guide built toward this final result. The TCP server showed how HTTP works under the hood. The routing showed how to dispatch requests. The code generation showed how to create unique identifiers. The file store showed how to persist data. The redirect analysis showed why 302 matters. And the rate limiter showed how to protect the service from abuse.
You now have a working URL shortener built from first principles. Every line of code uses only the Go standard library. No frameworks, no databases, no external services.
Keep Reading
- Go + Nginx: Deploy a Go API Behind a Reverse Proxy — put your URL shortener behind nginx for production use with TLS and load balancing.
- Containers From Scratch in Go — package your URL shortener into a container using the same Go patterns.
- Go + DynamoDB: Build a Simple CRUD App — replace the file-based storage with DynamoDB for a more scalable backend.