Microservice

Maintains

Processing states

  • processing near-real time events
  • processing catch up events

Deterministic processing

= events get processed exactly the same way every time you would retry it

Problem:

  • real world: bugs/crashes etc.

How?

Timestamps

types

  • event time (creation)
  • broker ingestion time
  • consumer ingestion time
  • processing time
Watermark

??

Stream time
  • consumer maintains stream time for topology (= highest timestamp of processed events)
  • always pick lowest event timestamp for next process
  • stream time never decreases