Skip to content

Persistent Write-Ahead Log

The write-ahead log ( WAL ) builds on circuit breaker and acknowledgement mechanisms to provide guaranteed delivery. The write-ahead log is useful in situations where sources/onramps do not offer guaranteed delivery themselves, but the data being distributed downstream can benefit from protection against loss and duplication.

In the configuration in this tutorial we configure a persistent WAL.


We configure a metronome as a source of data.

# File: etc/tremor/config/metronome.yaml
  - id: metronome
    type: metronome
      interval: 1000 # Every second

We configure a straight forward passthrough query to distribute the data to connected downstream sinks.

# File: etc/tremor/config/persistent.trickle
use tremor::system;

define qos::wal operator on_disk_wal
  read_count = 20,
  max_elements = 1000, # Capacity limit of 1000 stored events
  max_bytes = 10485760 # Capacity limit of 1MB of events

create operator on_disk_wal;

select patch event of
  insert hostname = system::hostname()
from in into on_disk_wal;

select event from on_disk_wal into out;

We then distribute the metronome events downstream to another websocket listener. We use websocat for this purpose in this example. We can invoke the server as follows:

$ websocat -s 8080
Listening on ws://

We configure the sink/offramp as follows:

  - id: ws
    type: ws
      url: ws://localhost:8080/

Finally, we interconnect the source, sink and pipeline logic into an active flow:

  - id: default
      "/onramp/metronome/{instance}/out": ["/pipeline/main/{instance}/in"]
      "/pipeline/main/{instance}/out": ["/offramp/ws/{instance}/in"]

    instance: "01"

Running the example via the tremor cli as follows:

$ tremor server run -f etc/tremor/config/*


If the tremor process restarts we pick up and re-send events that have not been acknowledged by the ws offramp and then carry on with new events coming from the metronome:

$ websocat -s 8080 ; websocat -s 8080
Listening on ws://


We restarted tremor after sending event with id 4. It did resend events 3 and 4 as they have not been acked from the perspective of the WAL yet.

If the downstream websocket service restarts we can recover up to 1000 events or any number of events worth 1MB. We may lose in flight events that were already acknowledged at the time the server went down and thus not fully delivered by the downstream system.

$ websocat -s 8080 ; websocat -s 8080
$  websocat -s 8080 ; websocat -s 8080
Listening on ws://


We killed the websocket server and restarted right afterwards. We in fact lost 1 event (id 5) which was acked inside tremor but not yet fully delivered to the console by websocat. Other events that the offramp was unable to send will be resent once the ws offramp can connect again.

In short, the persistent in memory wal can assist with partial recovery of downstream system or tremor itself and will actively reduce data loss within the configured retention but it is not guarenteed to be lossless.