Store and Forward

When a destination goes away — the historian's QuestDB, a logging database, a REST endpoint — QUBIQ does not drop the data and does not block the producer. It buffers to durable storage and replays in order once the destination returns.

What is covered

Store-and-forward protects outgoing data writes:

• Historian ingestion into QuestDB
• SQL and MongoDB writes from pipelines
• Outbound REST data pushes
• Audit journal events (on their own durable stream)

It does not apply to reads, nor to control writes to devices — a tag write to a PLC that fails must fail visibly, not be replayed minutes later against a process that has moved on.

How it works

The buffer survives a restart, so an overnight outage does not cost you the night's data.

1. One durable, file-backed buffer per connection. Buffers are independent, so a failing database cannot stall the historian.
2. A single forwarder drains each buffer, one batch at a time: batch, write, flush, then acknowledge. Acknowledging only after a confirmed flush is what makes a crash mid-drain replay rather than lose data.
3. A latch enforces ordering. Once any write for a connection has been buffered, every subsequent write for it is buffered too — until the backlog fully drains. Without this, a live write taken on the fast path overtakes older buffered ones and the destination ends up out of order. This is the classic store-and-forward bug, and the latch is the fix.
4. Failures are classified. A connection/transport failure is transient: retried indefinitely, head-of-line, and never counted toward the quarantine cap — an outage must not dead-letter good data. A query or data error is permanent: replaying it would only fail again or duplicate.

Conservative classification

A plain timeout is not treated as a connection failure, because a timed-out write may in fact have committed. Treating it as transient and replaying it would duplicate rows.

Buffer pressure

Each buffer has a byte limit. As it fills:

Fill level	Behaviour
Normal	Buffer drains as fast as the destination accepts.
Above the high-water mark	A buffer-pressure alarm is raised, warning you well before data is at risk.
At the limit	The oldest buffered items are discarded to accept new ones.

The high-water warning exists so that "my destination has been down for a day" reaches an operator before the oldest data starts falling off.

Monitor it in the Status panel (needs View connections).

Quarantine (dead-letter queue)

An item that fails permanently, or exceeds its delivery attempts, is moved to quarantine rather than being retried forever or silently dropped. Quarantine is an operator queue, not a graveyard:

Action	Permission
List quarantined items	View connections
Retry — put items back on the buffer	Edit connections
Drop — discard permanently	Edit connections

Investigate before retrying. A quarantined batch usually means a schema mismatch, a constraint violation or a credential change — retrying without fixing the cause just re-quarantines it.

Liveness gating

Forwarders consult the connection monitor's state. While a connection is known to be down, the forwarder idles instead of hammering it with retries. An unknown connection is treated as up — the sink's own error is the authority, not a stale assumption.

This is why connection health matters operationally: it is not only an indicator, it throttles retry traffic.

Historian specifics

The historian's sink adapts its QuestDB writer to the same forwarder, so history inherits ordered, ack-after-flush, no-drop semantics. Two extra behaviours are historian-only:

• A live tail buffer holds the most recent numeric samples in memory and is merged into live chart queries, closing the sub-second lag between ingestion and QuestDB's flush. Charts look live even though storage is batched.
• A write-ahead-log supervisor watches for QuestDB tables suspended by a failed WAL apply, and distinguishes recoverable stalls from poison segments — resuming the former, alarming on the latter.

→ Historian

What this means operationally

• A short outage is invisible. Data lands late, in order, complete.
• A long outage is visible early — via the buffer-pressure alarm — while there is still time to act.
• A bad payload is isolated, in quarantine, instead of blocking every good write behind it.
• Nothing is silently lost. Every drop is either an explicit operator action or a buffer that exceeded its configured limit after alarming.

Next

→ Connections · Historian