Store and Forward

When a destination goes away — the historian's QuestDB, a logging database, an MQTT broker — QUBIQ does not drop the data and does not block the producer. It buffers to durable storage and replays in order once the destination returns.

What is covered

Store-and-forward protects outgoing data writes:

• Historian ingestion into QuestDB
• QuestDB, SQL and MongoDB writes from pipelines
• SQL and MongoDB writes from scripts and named queries
• Alarm journal rows on their way to the alarm store
• Values a pipeline publishes to an external MQTT broker
• Audit journal events (on their own durable stream)

It does not apply to reads, nor to control writes to devices — a tag write to a PLC that fails must fail visibly, not be replayed minutes later against a process that has moved on. Outbound REST requests are not buffered either: a REST call that fails, fails.

How it works

The buffer survives a restart, so an overnight outage does not cost you the night's data.

1. One durable, file-backed buffer per connection. Buffers are independent, so a failing database cannot stall the historian.
2. A single forwarder drains each buffer, one batch at a time: batch, write, flush, then acknowledge. Acknowledging only after a confirmed flush is what makes a crash mid-drain replay rather than lose data.
3. A latch enforces ordering. Once any write for a connection has been buffered, every subsequent write for it is buffered too — until the backlog fully drains. Without this, a live write taken on the fast path overtakes older buffered ones and the destination ends up out of order. This is the classic store-and-forward bug, and the latch is the fix.
4. Failures are classified. A connection/transport failure is transient: retried indefinitely, head-of-line, and never counted toward the quarantine cap — an outage must not dead-letter good data. A query or data error is permanent: replaying it would only fail again or duplicate.

Conservative classification

A plain timeout is not treated as a connection failure, because a timed-out write may in fact have committed. Treating it as transient and replaying it would duplicate rows.

Buffer pressure

Each buffer has a byte limit. As it fills:

Fill level	Behaviour
Normal	Buffer drains as fast as the destination accepts.
Above the high-water mark	A buffer-pressure alarm is raised, warning you well before data is at risk.
At the limit	Historian, pipeline QuestDB and MQTT broker buffers discard their oldest items to accept new ones. SQL, MongoDB (pipeline and script) and alarm-journal buffers refuse new writes instead and keep their backlog — dropping a buffered write would lose it and break the write order.

The high-water warning exists so that "my destination has been down for a day" reaches an operator before the oldest data starts falling off, or new writes start being refused.

Monitor it in Settings → Store & Forward (needs View connections).

Quarantine (dead-letter queue)

An item that fails permanently, or exceeds its delivery attempts, is moved to quarantine rather than being retried forever or silently dropped. Quarantine is an operator queue, not a graveyard:

Action	Permission
List quarantined items	View connections
Retry — put items back on the buffer	Edit connections
Drop — discard permanently	Edit connections

Investigate before retrying. A quarantined batch usually means a schema mismatch, a constraint violation or a credential change — retrying without fixing the cause just re-quarantines it.

Liveness gating

Forwarders consult the connection monitor's state. While a connection is known to be down, the forwarder idles instead of hammering it with retries. An unknown connection is treated as up — the sink's own error is the authority, not a stale assumption.

This is why connection health matters operationally: it is not only an indicator, it throttles retry traffic.

Historian specifics

The historian's sink adapts its QuestDB writer to the same forwarder, so history inherits ordered, ack-after-flush, no-drop semantics. Two extra behaviours are historian-only:

• A live tail buffer holds the most recent numeric samples in memory and is merged into live chart queries, closing the sub-second lag between ingestion and QuestDB's flush. Charts look live even though storage is batched.
• A write-ahead-log supervisor watches for QuestDB tables suspended by a failed WAL apply, and distinguishes recoverable stalls from poison segments — resuming the former, alarming on the latter.

→ Historian

What this means operationally

• A short outage is invisible. Data lands late, in order, complete.
• A long outage is visible early — via the buffer-pressure alarm — while there is still time to act.
• A bad payload is isolated, in quarantine, instead of blocking every good write behind it.
• Nothing is silently lost. Every drop is either an explicit operator action or a time-series buffer that exceeded its configured limit after alarming.

Next

→ Connections · Historian