# MSK vs Kinesis Cheatsheet (Cost & When to Choose)

- URL: https://karandeepsingh.ca/cheatsheets/cost/msk-vs-kinesis/
- Author: Karandeep Singh
- Published: 2026-08-06
- Tags: aws, cloud, cost
- Summary: Quick-reference comparing Amazon MSK (managed Kafka) and Kinesis Data Streams, pricing models and idle cost, a feature table, and a clear when-to-choose-which decision guide.

Both **Amazon MSK** (managed Apache Kafka) and **Amazon Kinesis Data Streams** move high-throughput event streams. They overlap heavily, but they price very differently and pull you in different directions on lock-in and ops. This is the quick comparison, with the cost angle up front.

> **Note:** Prices are early-2026 ballparks and change often. Treat them as *relative* comparisons, confirm on the AWS pricing pages.

## What each one is

- **Amazon MSK** — real, open-source **Apache Kafka**, managed by AWS. Full Kafka API, ecosystem, and portability. Comes as **Provisioned** (you size brokers) or **Serverless** (no brokers to size).
- **Kinesis Data Streams** — AWS's **native** streaming service. Proprietary API, deeply integrated with Lambda/Firehose/Flink. Comes as **Provisioned** (you set shards) or **On-demand** (auto-scales).

## Cost comparison (the headline)

**info (rates):**
Every figure below is **us-east-1**, taken from AWS's public Price List API. Rates differ
by Region and do change — check the [Kinesis](https://aws.amazon.com/kinesis/data-streams/pricing/)
and [MSK](https://aws.amazon.com/msk/pricing/) pricing pages before you budget against them.
A month here is 730 hours.

| Service | Pricing model | Idle / floor cost | Scales by | Cheapest when |
|---|---|---|---|---|
| **Kinesis (On-demand)** | Per GB in + per-stream-hour | **Very low** (near-zero idle) | Automatic | Low, spiky, or unknown volume |
| **Kinesis (Provisioned)** | Per **shard-hour** (~$0.015) + PUT units | **~$11/shard/mo** (1 shard min) | Shards you set | Steady, known moderate volume |
| **MSK Serverless** | Per **cluster-hour** (~$0.75) + throughput + storage | **~$540/mo** baseline | Automatic | Want Kafka + no broker mgmt, higher volume |
| **MSK Provisioned** | Per **broker-hour** + storage + transfer | **~$70–300+/mo** (min brokers) | Brokers / partitions | Very high sustained throughput |

**The one-line cost takeaway:** Kinesis (especially **on-demand or a single provisioned shard**) is far cheaper at low or spiky volume; MSK carries a real **always-on floor**. At very high *sustained* throughput, MSK Provisioned's per-GB economics can win.

### Worked example: a ~1 MB/s stream

- **Kinesis Provisioned:** 1 shard handles 1 MB/s in → **~$11/mo** + small PUT-unit cost. Cheapest.
- **Kinesis On-demand:** 1 MB/s sustained ≈ 2.6 TB/mo. Watch which rate you apply — **ingest is $0.08/GB, double the $0.04/GB read rate**, and it is the ingest one that matters here: 2,592 GB × $0.08 = **$207**, plus $0.04/stream-hour (**$29**), plus another **$104** if one consumer reads it all back. Call it **$235–340/mo** against ~$11 provisioned. On-demand is for *spiky*, not sustained — and that 20–30× gap is why.
- **MSK:** overkill at this size, you're paying the broker/cluster floor regardless.

_Diagram: One provisioned shard covers 1 MB/s for **$11 a month**. The same load on Kinesis on-demand is **20–30× more**, because on-demand charges $0.08/GB to *write* — that is the number people get wrong. MSK is not competing at this size; you are paying its floor either way._

> Lesson: **on-demand for spiky, provisioned for sustained** — within Kinesis that choice alone can be a 10x cost swing.

## Feature comparison

| Dimension | MSK (Kafka) | Kinesis Data Streams |
|---|---|---|
| **API / protocol** | Apache Kafka (open, portable) | AWS proprietary SDK/API |
| **Ecosystem** | Full Kafka (Connect, Streams, MirrorMaker, 3rd-party) | AWS-native (Lambda, Firehose, Flink, KCL) |
| **Portability / lock-in** | Portable, multi-cloud, no lock-in | Locked to AWS |
| **Scaling unit** | Partitions / brokers | Shards (or auto in on-demand) |
| **Ordering** | Per-partition | Per-shard |
| **Max record size** | Configurable (default ~1 MB, can raise) | **1 MB hard limit** |
| **Retention** | Configurable, up to unlimited (tiered storage) | Default 24 h, up to 365 days |
| **Ops overhead** | Higher (Kafka concepts, even when managed) | Lower (esp. on-demand) |
| **Lambda trigger** | Yes (event source) | Yes (native) |

## Choosing like a senior DevOps engineer

### The default

**Default to Kinesis Data Streams (on-demand)** for greenfield AWS work: lowest ops, near-zero idle cost, native Lambda/Firehose/Flink integration, automatic scaling. Move to **MSK** only when a concrete need pulls you there. Don't adopt Kafka for its own sake, it's a standing operational commitment even when "managed."

### Questions to ask before you pick

Each "yes" is a reason to leave the default and consider MSK:

1. **Do we already run Kafka** (apps, skills, tooling, a migration)? → MSK
2. **Do we need the Kafka ecosystem** (Connect connectors, Kafka Streams, MirrorMaker)? → MSK
3. **Do we need portability / multi-cloud / no lock-in?** → MSK
4. **Are records ever larger than 1 MB**, or do we need very long / unlimited retention? → MSK
5. **Is throughput very high and sustained** (24/7, many MB/s)? → MSK Provisioned can win on unit cost
6. **Is volume low, spiky, or unknown?** → Kinesis on-demand
7. **Are consumers mostly AWS-native** (Lambda, Firehose, Flink)? → Kinesis
8. **How much ops do we want to own, and how mature is on-call?** → less / smaller team → Kinesis (or MSK Serverless)
9. **How budget-sensitive are we at low volume?** → Kinesis (MSK has a real floor)

If the answers mostly point at AWS-native, low-ops, cost-sensitive, stay on **Kinesis**. It takes a couple of firm "yes → MSK" answers to justify Kafka.

### Monolith vs microservices

- **Monolith:** you usually don't need a streaming backbone at all, reach for **SQS/SNS/EventBridge** first. If you genuinely need a durable, replayable stream (analytics ingest, event sourcing), **Kinesis** is the low-friction pick; you don't yet have the many independent consumers that justify Kafka.
- **Microservices:** many independent producers/consumers, event-driven choreography, per-consumer replay, this is where a durable log earns its keep. **Kinesis** fits AWS-native microservices well; **MSK/Kafka** grows compelling as the estate scales: many teams, a shared event backbone, rich Connect integrations to external systems. Kafka is the classic "central nervous system" for a large microservice org.

### Server/containers vs Lambda consumers

- **Lambda / serverless consumers → Kinesis.** Native event source, no VPC wiring, scales and bills per use; on-demand + Lambda is the lowest-friction combo.
- **Long-running server/container consumers (EC2/ECS/EKS) running Kafka clients, Kafka Streams, Flink, or Spark → MSK.** Standard Kafka clients, consumer groups, and stateful stream processors feel native there.
- **Rule:** serverless/event-driven consumers → **Kinesis**; long-running stateful stream processors on servers → **MSK**.

### First, is streaming even the right tool?

A senior instinct is *not* reaching for Kafka/Kinesis when something simpler fits:

- Just decouple + retry work → **SQS**
- Fan-out pub/sub → **SNS**
- Route/filter events across services → **EventBridge**
- Ordered, replayable, high-throughput log with multiple independent consumers / analytics → **Kinesis or MSK**

Don't pay for a streaming platform when a queue solves it.

### Quick decision

- **Greenfield, AWS-native, cost- and ops-sensitive?** → **Kinesis** (on-demand for spiky, provisioned for steady).
- **Already Kafka, need its ecosystem/portability, >1 MB records, or huge sustained throughput?** → **MSK** (Serverless to skip broker ops, Provisioned for max scale).

## References and Further Reading

- Amazon Web Services. [Amazon MSK pricing](https://aws.amazon.com/msk/pricing/).
- Amazon Web Services. [Amazon Kinesis Data Streams pricing](https://aws.amazon.com/kinesis/data-streams/pricing/).
- Amazon Web Services. [Amazon MSK: provisioned vs serverless](https://docs.aws.amazon.com/msk/latest/developerguide/what-is-msk.html).
- Amazon Web Services. [Kinesis Data Streams capacity modes](https://docs.aws.amazon.com/streams/latest/dev/how-do-i-size-a-stream.html).