/user/kayd @ devops :~$ cat python-boto3.md

Python Boto3 Cheatsheet Python Boto3 Cheatsheet

QR Code linking to: Python Boto3 Cheatsheet
Karandeep Singh
Karandeep Singh
• 5 minutes

Summary

My notes for automating AWS with boto3, covering client/resource setup, S3, EC2, Lambda, DynamoDB, and SSM, plus paginators, waiters, and error handling done right.

Boto3 is how I automate everything in AWS that the console makes tedious: fleet-wide tag audits, S3 lifecycle scripts, Lambda deploys from CI. These are the calls I use constantly, plus the paginator/waiter/retry patterns that separate throwaway scripts from reliable ones. Brush up on the language itself in the Python Basics & CLI Cheatsheet if needed.

python -m pip install boto3

Session, Client, Resource

Docs: Boto3 credentials guide

import boto3

# Client - low-level, 1:1 with the AWS API. Prefer this; it covers every service.
s3 = boto3.client("s3", region_name="us-east-1")

# Resource - higher-level OO wrapper (S3, EC2, DynamoDB, and a few others only;
# in maintenance mode - no new services get resources)
s3r = boto3.resource("s3")

# Session - explicit credentials/profile/region; use for multi-account scripts
session = boto3.Session(profile_name="prod", region_name="eu-west-1")
ec2 = session.client("ec2")

# Cross-account via STS assume-role
sts = boto3.client("sts")
creds = sts.assume_role(
    RoleArn="arn:aws:iam::123456789012:role/Automation",
    RoleSessionName="audit",
)["Credentials"]
target = boto3.client(
    "s3",
    aws_access_key_id=creds["AccessKeyId"],
    aws_secret_access_key=creds["SecretAccessKey"],
    aws_session_token=creds["SessionToken"],
)

Credential Resolution Order

  1. Parameters passed to boto3.client() / Session()
  2. Environment: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN
  3. Shared credentials file ~/.aws/credentials (profile via AWS_PROFILE)
  4. Config file ~/.aws/config (incl. SSO and assume-role profiles)
  5. Container credentials (ECS task role)
  6. EC2 instance profile (IMDS)

S3

Docs: Boto3 S3 API reference

s3 = boto3.client("s3")

# Upload / download (managed transfers - multipart automatically for big files)
s3.upload_file("build.tar.gz", "my-bucket", "releases/build.tar.gz",
               ExtraArgs={"ServerSideEncryption": "aws:kms"})
s3.download_file("my-bucket", "releases/build.tar.gz", "/tmp/build.tar.gz")

# Small objects in memory
s3.put_object(Bucket="my-bucket", Key="cfg.json", Body=b'{"env":"prod"}')
body = s3.get_object(Bucket="my-bucket", Key="cfg.json")["Body"].read()

# Presigned URL - temporary access without credentials
url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": "my-bucket", "Key": "report.pdf"},
    ExpiresIn=3600,          # seconds
)
# For uploads: "put_object", or generate_presigned_post for browser forms

# List with pagination - list_objects_v2 caps at 1000, ALWAYS paginate
paginator = s3.get_paginator("list_objects_v2")
for page in paginator.paginate(Bucket="my-bucket", Prefix="logs/2026/"):
    for obj in page.get("Contents", []):
        print(obj["Key"], obj["Size"])

s3.delete_object(Bucket="my-bucket", Key="old.log")
s3.copy_object(Bucket="my-bucket", Key="dst",
               CopySource={"Bucket": "my-bucket", "Key": "src"})

EC2

Docs: Boto3 EC2 API reference

ec2 = boto3.client("ec2")

# Describe with filters
resp = ec2.describe_instances(Filters=[
    {"Name": "tag:Env", "Values": ["prod"]},
    {"Name": "instance-state-name", "Values": ["running"]},
])
for reservation in resp["Reservations"]:
    for inst in reservation["Instances"]:
        name = next((t["Value"] for t in inst.get("Tags", [])
                     if t["Key"] == "Name"), "-")
        print(inst["InstanceId"], inst["InstanceType"],
              inst.get("PrivateIpAddress"), name)

# Start / stop / reboot / terminate
ec2.stop_instances(InstanceIds=["i-0abc123"])
ec2.start_instances(InstanceIds=["i-0abc123"])
ec2.terminate_instances(InstanceIds=["i-0abc123"], DryRun=True)  # permission check

# Tags
ec2.create_tags(Resources=["i-0abc123"],
                Tags=[{"Key": "Owner", "Value": "platform-team"}])
ec2.delete_tags(Resources=["i-0abc123"], Tags=[{"Key": "Temp"}])

Lambda

Docs: Boto3 Lambda API reference

import json, base64
lam = boto3.client("lambda")

# Synchronous invoke
resp = lam.invoke(
    FunctionName="report-gen",
    InvocationType="RequestResponse",     # "Event" = async fire-and-forget
    Payload=json.dumps({"day": "2026-07-23"}),
    LogType="Tail",                       # last 4KB of logs in the response
)
result = json.loads(resp["Payload"].read())
logs = base64.b64decode(resp["LogResult"]).decode()
if resp.get("FunctionError"):             # function raised - payload holds the error
    raise RuntimeError(result)

# Deploy new code from a zip
with open("function.zip", "rb") as f:
    lam.update_function_code(FunctionName="report-gen", ZipFile=f.read())
lam.get_waiter("function_updated_v2").wait(FunctionName="report-gen")

lam.update_function_configuration(FunctionName="report-gen",
                                  MemorySize=512, Timeout=30,
                                  Environment={"Variables": {"ENV": "prod"}})

DynamoDB

Docs: Boto3 DynamoDB API reference

from boto3.dynamodb.conditions import Key, Attr

table = boto3.resource("dynamodb").Table("deployments")   # resource handles types for you

table.put_item(Item={
    "service": "api",              # partition key
    "deployed_at": "2026-07-23T10:00:00Z",   # sort key
    "version": "1.4.2",
    "healthy": True,
})

item = table.get_item(Key={"service": "api",
                           "deployed_at": "2026-07-23T10:00:00Z"}).get("Item")

# Query - partition key required, sort key optional
resp = table.query(
    KeyConditionExpression=Key("service").eq("api")
                           & Key("deployed_at").begins_with("2026-07"),
    FilterExpression=Attr("healthy").eq(True),
    ScanIndexForward=False,        # newest first
    Limit=10,
)
items = resp["Items"]

# Batch write
with table.batch_writer() as batch:
    for item in items:
        batch.put_item(Item=item)

SSM Parameter Store

Docs: Boto3 SSM API reference

ssm = boto3.client("ssm")

ssm.put_parameter(Name="/prod/api/db-url", Value="postgres://...",
                  Type="SecureString", Overwrite=True)

val = ssm.get_parameter(Name="/prod/api/db-url",
                        WithDecryption=True)["Parameter"]["Value"]

# All params under a path
paginator = ssm.get_paginator("get_parameters_by_path")
for page in paginator.paginate(Path="/prod/api/", Recursive=True,
                               WithDecryption=True):
    for p in page["Parameters"]:
        print(p["Name"], "=", p["Value"])

Error Handling with ClientError

Docs: Boto3 error handling guide

from botocore.exceptions import ClientError, NoCredentialsError

try:
    s3.head_object(Bucket="my-bucket", Key="maybe-missing.txt")
except ClientError as e:
    code = e.response["Error"]["Code"]
    if code == "404":                          # head_object returns 404, not NoSuchKey
        print("object does not exist")
    elif code in ("AccessDenied", "403"):
        print("check IAM policy")
    else:
        raise
except NoCredentialsError:
    print("no credentials found - check the resolution chain")

Common codes worth branching on: NoSuchKey, NoSuchBucket, AccessDenied, Throttling, ThrottlingException, ResourceNotFoundException (DynamoDB/Lambda), ParameterNotFound (SSM), ConditionalCheckFailedException (DynamoDB).

Paginators & Waiters

Docs: Boto3 paginators guide

# Any list/describe call with a NextToken has a paginator - never loop tokens by hand
ec2_pag = ec2.get_paginator("describe_instances")
ids = [i["InstanceId"]
       for page in ec2_pag.paginate()
       for r in page["Reservations"]
       for i in r["Instances"]]

# Waiters - poll until a state is reached, with sane backoff built in
ec2.get_waiter("instance_running").wait(
    InstanceIds=["i-0abc123"],
    WaiterConfig={"Delay": 15, "MaxAttempts": 40},
)
s3.get_waiter("object_exists").wait(Bucket="my-bucket", Key="flag.done")
Handy waitersService
instance_running, instance_stopped, instance_terminatedEC2
object_exists, bucket_existsS3
function_active_v2, function_updated_v2Lambda
table_exists, table_not_existsDynamoDB

Retries & Client Config

Docs: Boto3 retries guide

from botocore.config import Config

cfg = Config(
    region_name="us-east-1",
    retries={"max_attempts": 10, "mode": "adaptive"},  # adaptive throttles client-side
    connect_timeout=5,
    read_timeout=60,
    max_pool_connections=50,       # raise when fanning out across threads
)
s3 = boto3.client("s3", config=cfg)

Retry modes: legacy (old default), standard (sane default, jittered backoff), adaptive (standard + client-side rate limiting, best for scripts hammering one API). Clients are thread-safe; share one client across a ThreadPoolExecutor fan-out (see the Python Threading & Concurrency Cheatsheet) but give each process its own.

  • Python Threading & Concurrency: ThreadPoolExecutor patterns that turn slow serial AWS calls into fast parallel fan-outs
  • AWS CloudFormation: declaring these resources in templates instead, with anatomy, intrinsic functions, and the change-set workflow
  • Amazon Linux 2 Terminal: yum, amazon-linux-extras, IMDSv2, and SSM on the EC2 instances these scripts manage

More Python sheets are in the Python group; everything I’ve written up is in the cheatsheet library.

Similar Articles

More from development

Go (Golang) Cheatsheet

Go cheatsheet covering the go toolchain, syntax essentials, goroutines and channels, table-driven …

Bash Basics Cheatsheet

Quick-reference Bash basics cheatsheet: variables, parameter expansion, conditionals, loops, arrays, …