/user/kayd @ devops :~$ cat msk-vs-kinesis.md

MSK vs Kinesis Cheatsheet (Cost & When to Choose) MSK vs Kinesis Cheatsheet (Cost & When to Choose)

QR Code linking to: MSK vs Kinesis Cheatsheet (Cost & When to Choose)
Karandeep Singh
Karandeep Singh
• 6 minutes

Summary

Quick-reference comparing Amazon MSK (managed Kafka) and Kinesis Data Streams, pricing models and idle cost, a feature table, and a clear when-to-choose-which decision guide.

Both Amazon MSK (managed Apache Kafka) and Amazon Kinesis Data Streams move high-throughput event streams. They overlap heavily, but they price very differently and pull you in different directions on lock-in and ops. This is the quick comparison, with the cost angle up front.

Note: Prices are early-2026 ballparks and change often. Treat them as relative comparisons, confirm on the AWS pricing pages.

What each one is

  • Amazon MSK — real, open-source Apache Kafka, managed by AWS. Full Kafka API, ecosystem, and portability. Comes as Provisioned (you size brokers) or Serverless (no brokers to size).
  • Kinesis Data Streams — AWS’s native streaming service. Proprietary API, deeply integrated with Lambda/Firehose/Flink. Comes as Provisioned (you set shards) or On-demand (auto-scales).

Cost comparison (the headline)

rates
Every figure below is us-east-1, taken from AWS’s public Price List API. Rates differ by Region and do change — check the Kinesis and MSK pricing pages before you budget against them. A month here is 730 hours.
ServicePricing modelIdle / floor costScales byCheapest when
Kinesis (On-demand)Per GB in + per-stream-hourVery low (near-zero idle)AutomaticLow, spiky, or unknown volume
Kinesis (Provisioned)Per shard-hour (~$0.015) + PUT units~$11/shard/mo (1 shard min)Shards you setSteady, known moderate volume
MSK ServerlessPer cluster-hour (~$0.75) + throughput + storage~$540/mo baselineAutomaticWant Kafka + no broker mgmt, higher volume
MSK ProvisionedPer broker-hour + storage + transfer~$70–300+/mo (min brokers)Brokers / partitionsVery high sustained throughput

The one-line cost takeaway: Kinesis (especially on-demand or a single provisioned shard) is far cheaper at low or spiky volume; MSK carries a real always-on floor. At very high sustained throughput, MSK Provisioned’s per-GB economics can win.

Worked example: a ~1 MB/s stream

  • Kinesis Provisioned: 1 shard handles 1 MB/s in → ~$11/mo + small PUT-unit cost. Cheapest.
  • Kinesis On-demand: 1 MB/s sustained ≈ 2.6 TB/mo. Watch which rate you apply — ingest is $0.08/GB, double the $0.04/GB read rate, and it is the ingest one that matters here: 2,592 GB × $0.08 = $207, plus $0.04/stream-hour ($29), plus another $104 if one consumer reads it all back. Call it $235–340/mo against ~$11 provisioned. On-demand is for spiky, not sustained — and that 20–30× gap is why.
  • MSK: overkill at this size, you’re paying the broker/cluster floor regardless.
1 MB/s sustained · us-east-1 · 730h month
KinesisMSK$0Kinesis provisioned · 1 shard$11MSK provisioned · 2 brokers$67Kinesis on-demand · ingest only$236Kinesis on-demand · +1 consumer$340MSK Serverless · baseline$548
One provisioned shard covers 1 MB/s for $11 a month. The same load on Kinesis on-demand is 20–30× more, because on-demand charges $0.08/GB to write — that is the number people get wrong. MSK is not competing at this size; you are paying its floor either way.

Lesson: on-demand for spiky, provisioned for sustained — within Kinesis that choice alone can be a 10x cost swing.

Deepen your understanding in Low-Cost Cloud Stack Cheatsheet

Get new articles by email

One DevOps article a week, plus the 18-cheatsheet PDF pack. No spam, one click to leave.

Feature comparison

DimensionMSK (Kafka)Kinesis Data Streams
API / protocolApache Kafka (open, portable)AWS proprietary SDK/API
EcosystemFull Kafka (Connect, Streams, MirrorMaker, 3rd-party)AWS-native (Lambda, Firehose, Flink, KCL)
Portability / lock-inPortable, multi-cloud, no lock-inLocked to AWS
Scaling unitPartitions / brokersShards (or auto in on-demand)
OrderingPer-partitionPer-shard
Max record sizeConfigurable (default ~1 MB, can raise)1 MB hard limit
RetentionConfigurable, up to unlimited (tiered storage)Default 24 h, up to 365 days
Ops overheadHigher (Kafka concepts, even when managed)Lower (esp. on-demand)
Lambda triggerYes (event source)Yes (native)

Choosing like a senior DevOps engineer

tip
The question that settles it is not throughput, it is who operates it. Kinesis has no brokers, no partitions to rebalance and no version upgrades; MSK gives you Kafka’s ecosystem and portability, and hands you the broker fleet. Pick the one whose failure modes you want to be on call for.
Question

If this stream doubled in volume overnight, which of the two would need a human — and is that person on your team?

The default

Default to Kinesis Data Streams (on-demand) for greenfield AWS work: lowest ops, near-zero idle cost, native Lambda/Firehose/Flink integration, automatic scaling. Move to MSK only when a concrete need pulls you there. Don’t adopt Kafka for its own sake, it’s a standing operational commitment even when “managed.”

Questions to ask before you pick

Each “yes” is a reason to leave the default and consider MSK:

  1. Do we already run Kafka (apps, skills, tooling, a migration)? → MSK
  2. Do we need the Kafka ecosystem (Connect connectors, Kafka Streams, MirrorMaker)? → MSK
  3. Do we need portability / multi-cloud / no lock-in? → MSK
  4. Are records ever larger than 1 MB, or do we need very long / unlimited retention? → MSK
  5. Is throughput very high and sustained (24/7, many MB/s)? → MSK Provisioned can win on unit cost
  6. Is volume low, spiky, or unknown? → Kinesis on-demand
  7. Are consumers mostly AWS-native (Lambda, Firehose, Flink)? → Kinesis
  8. How much ops do we want to own, and how mature is on-call? → less / smaller team → Kinesis (or MSK Serverless)
  9. How budget-sensitive are we at low volume? → Kinesis (MSK has a real floor)

If the answers mostly point at AWS-native, low-ops, cost-sensitive, stay on Kinesis. It takes a couple of firm “yes → MSK” answers to justify Kafka.

Monolith vs microservices

  • Monolith: you usually don’t need a streaming backbone at all, reach for SQS/SNS/EventBridge first. If you genuinely need a durable, replayable stream (analytics ingest, event sourcing), Kinesis is the low-friction pick; you don’t yet have the many independent consumers that justify Kafka.
  • Microservices: many independent producers/consumers, event-driven choreography, per-consumer replay, this is where a durable log earns its keep. Kinesis fits AWS-native microservices well; MSK/Kafka grows compelling as the estate scales: many teams, a shared event backbone, rich Connect integrations to external systems. Kafka is the classic “central nervous system” for a large microservice org.

Server/containers vs Lambda consumers

  • Lambda / serverless consumers → Kinesis. Native event source, no VPC wiring, scales and bills per use; on-demand + Lambda is the lowest-friction combo.
  • Long-running server/container consumers (EC2/ECS/EKS) running Kafka clients, Kafka Streams, Flink, or Spark → MSK. Standard Kafka clients, consumer groups, and stateful stream processors feel native there.
  • Rule: serverless/event-driven consumers → Kinesis; long-running stateful stream processors on servers → MSK.

First, is streaming even the right tool?

A senior instinct is not reaching for Kafka/Kinesis when something simpler fits:

  • Just decouple + retry work → SQS
  • Fan-out pub/sub → SNS
  • Route/filter events across services → EventBridge
  • Ordered, replayable, high-throughput log with multiple independent consumers / analytics → Kinesis or MSK

Don’t pay for a streaming platform when a queue solves it.

Discover related concepts in Git Cheatsheet

Quick decision

  • Greenfield, AWS-native, cost- and ops-sensitive? → Kinesis (on-demand for spiky, provisioned for steady).
  • Already Kafka, need its ecosystem/portability, >1 MB records, or huge sustained throughput? → MSK (Serverless to skip broker ops, Provisioned for max scale).

References and Further Reading

Similar Articles

More from cloud

Atuin Cheatsheet

Atuin cheatsheet: Ctrl-R search keys, filter and search modes, non-interactive flags, sync and …

Git Cheatsheet

Git cheatsheet: reset vs revert vs restore, rebase vs merge, reflog recovery, bisect, stash, …

Get new articles by email

One DevOps article a week, plus the 18-cheatsheet PDF pack. No spam, one click to leave.

🍪 We use cookies for analytics. Read the privacy policy