backpressure

A change-data-capture pipeline you can break on purpose. Source table, connector, partitioned log, consumer group, table-format sink.

insert update delete replayed consumer head

How to use it

  1. Start with a scenario on the left: Steady state, Hot key, Too many consumers, At-most-once, Replay storm or Exactly-once. Each one says what to watch for.
  2. Break it: Kill a consumer (K), Add a consumer (A) or Drop the connector (C).
  3. Change the shape: write rate, partitions, consumers, consumer throughput and hot-key share.
  4. Switch delivery between at-most-once, at-least-once and exactly-once, and change how often offsets commit and the sink snapshots.
  5. Watch the tiles for lag and latency, and Correctness for duplicates applied, records lost and stale overwrites. What's happening explains the cause in plain words.