Events on Apache Iggy

wakuwaku::iggy (the wakuwaku-iggy crate) is the event layer of wakuwaku. You describe each event type once, as a Rust type, and the framework handles the rest: which Iggy topic and partition it goes to, how its body is encoded, which events must be processed in order, which can be batched or folded, and what happens when a handler fails.

If you come from Spring, think of it as the role Spring for Apache Kafka plays there: a typed publisher plus listener methods. The difference is where the decisions live. With wakuwaku, ordering and batching rules are declared on the event type and checked by the compiler, and the consumer uses them to process a partition concurrently without breaking any order the events ask for.

Why order per key, not per partitionCopy Anchor Link

An Iggy partition is a totally ordered log. The simplest consumer processes it strictly one record at a time, which has two costs:

• One key stalls the others. A partition carries events of many keys (users, orders, …). If processing the events of order 17 is slow, or keeps failing, every other order in the partition waits behind it.
• Nothing can be combined. Ten consecutive "points granted" events for one user are ten handler calls, even when a single call with the sum would do.

Business logic almost never needs the whole partition in order. It needs the events of one key in order: an order must be placed before it is paid, but order 17 and order 18 have nothing to do with each other. wakuwaku therefore orders events per key:

• Events of different keys are never ordered against each other, even when they share a partition. Their keys are processed concurrently.
• Within one key, events of the same type always keep their log order.
• Between events of different types of one key, each event's EventAtomicOrdering (Relaxed, Acquire, Release, AcqRel) decides what may move past what. See Ordering.
• Inside one polled batch, all events of one type and one key between two ordering fences form a run, and the event type's algebra (NonAssociative, Commutative, Idempotent, IdempotentCommutative, Associative) says how a run may be reduced before the handler sees it. See Algebra.

Because ordering is scoped to a key, a failing key can also be moved aside: its unfinished work goes to a retry topic and the rest of the partition keeps flowing. See Error handling.

The modelCopy Anchor Link

Three types decide where an event goes and how it is recognised:

The stream is not part of the types. It comes from configuration: the stream name you pass to Publisher::new and to ConsumerConfig. One topic per key type means every event about orders (placed, paid, shipped, …) lives in one topic, keyed by OrderId, so a single consumer sees all of them in their log order and can respect the order between them.

What the consumer doesCopy Anchor Link

IggyConsumerRegisterCenter::start spawns one task per configured partition. Each task loops over these stages:

1. Retry first. It reads its partition of the retry topic retry_<topic> and re-runs the keys whose retry delay has elapsed.
2. Poll. It fetches up to poll_count records from its partition of the main topic.
3. Plan. It groups the batch by key hash, splits each key's events into segments at the ordering fences, and groups each segment's events by type into runs.
4. Execute. It processes keys concurrently; inside a key, segments run in order, and inside a segment the head run, then the body runs concurrently, then the tail run.
5. Reduce and handle. It decodes each run, reduces it with the event type's algebra, and calls the handler in chunks of at most MAX_BATCH events.
6. Isolate failures. A retryable failure moves the key's unfinished work to the retry topic and quarantines the key, so its later events queue behind it there.
7. Commit. It stores the consumer offset either before or after processing, depending on the DeliveryMode.

Consuming events walks through the API and every configuration option.

Enabling the featureCopy Anchor Link

Events are behind the iggy feature, which is off by default:

[dependencies]
wakuwaku = { version = "1.0.0-rc3", features = ["iggy"] }
tokio = { version = "1", features = ["full"] }

The feature enables:

• wakuwaku::iggy: the re-exported wakuwaku-iggy crate (events, publisher, handlers, consumer runtime).
• wakuwaku::deps::iggy: the Apache Iggy client SDK (iggy 0.11) that wakuwaku is built against. Use it for IggyClient and for creating streams and topics, instead of adding iggy yourself, so the versions always match.
• ServiceBuilder::iggy_consumer and ServiceBuilder::compose_iggy_consumer: building event handlers from your service's shared dependencies.

Event bodies are serialized with the kanau::message traits. Deriving them with serde JSON needs two extra dependencies; see Serializing the body.

In this sectionCopy Anchor Link