HorneSci

Use case 3 of 4 ยท Monitoring

Telemetry rollups over a fixed hierarchy

Sensors report to servers, servers to racks, racks to a facility. The totals at each level are sums or maxima over a fixed tree. When a few readings move, only the branches above them need to update.

A long row of supercomputer racks with perforated doors, seen at an angle.

What was measured

A windowed aggregation over 4,096 sensors in a fanout-4, six-level tree, rolling into 1,365 groups. The baseline is stock Kafka Streams, running the same aggregation. Both were run on the same machine.

Delta Routing pipeline against stock Kafka Streams
MeasureResult
Drain timeabout 15 times faster (21 times once warm)
Staleness, 99th percentilefrom 10.9 seconds to 107 milliseconds
CPU use38.2% lower
Resident memory60.5% lower
Energy for the same work2.1 times higher

How to read the result

Most of the drain gain comes from the pipeline design. The stock version fans out events, serializes them, moves them between processes over the network, and keeps its state in RocksDB. The Delta Routing version runs in one process over an in-memory array. We have not yet measured how much of the gain comes from the update rule alone, against an in-process dense accumulator. That comparison is the next measurement.

The energy result runs the other way. The in-process version uses more energy on this workload, and we report it here alongside the time result.

Conditions

  • Host: Intel Core i7-1165G7, performance governor, turbo disabled.
  • Runtime: Temurin 25 for the Kafka Streams baseline.
  • Scope: one aggregation pipeline on synthetic sensor events. Not a production deployment.

Where this applies

Use Delta Routing for rollups where the hierarchy is fixed, updates are small and frequent, and the current pipeline pays for fan-out, serialization or state storage on every step.

Next step

Send the topology, the update rate, and the current pipeline. We measure your pipeline and a version without the rewrite, so the result separates the architecture from the update rule. Start an assessment.

HorneSci