Skip to main content
Version: 1.0.6

Historian Configuration

1. Create a QuestDB connection​

The historian needs a QuestDB instance with both ports reachable:

PortProtocolUsed for
8812Postgres wireQueries
9009Line protocol (ILP)Ingestion

→ QuestDB connection

2. Designate it as the historian​

The designation lives in Settings → Historian. Reading it needs View connections; changing it needs Edit connections.

Exactly one connection can hold the designation. Changing it opens a new epoch rather than truncating existing history.

Before accepting a store, QUBIQ pre-flights it: it looks inside for an existing historian table and checks its shape, rather than issuing a CREATE TABLE IF NOT EXISTS that silently no-ops against a table it did not create.

3. Tune ingestion​

SettingEffectGuidance
Worker countParallel ILP senders1–4; more only helps at high tag counts
Auto-flush rowsFlush after N rowsHigher = more throughput, later visibility
Auto-flush intervalFlush after N msBounds visibility latency on slow plants
DeduplicationQuestDB DEDUP UPSERT KEYSLeave on — makes replays idempotent
PartitionBase table partition unitDAY suits most plants
Rollup partitionRollup partition unitCoarser than the base table
Rollup refresh everyRe-aggregation cadenceBalance freshness against load

Deduplication matters more than it looks. Store-and-forward can replay a batch that was flushed but not acknowledged — a crash in exactly that window. With dedup on, the replay overwrites identical rows; with it off, it duplicates them.

4. Enable history on tags​

Per tag: Enable history, plus a deadband and optionally a store interval.

Be deliberate. For each tag ask:

  • Will anyone trend this, or is the live value enough?
  • What deadband makes the trend honest without storing noise?
  • Does a frozen signal need a store interval to prove it was alive?

A 0.5–1% of span deadband typically removes 90%+ of the rows and changes no chart a human can read.

5. Set retention​

The sweep runs daily, dropping whole partitions past the configured age. Because partition drops are metadata-only in QuestDB, retention costs essentially nothing.

The first sweep waits for a warm-up period after boot, so it does not race schema creation.

Sizing:

rows ≈ tags × changes-per-second × 86,400 × retention-days

Apply QuestDB's compression to that, then add headroom. If the number is uncomfortable, revisit deadbands before revisiting retention — a value you never stored costs nothing to keep.

Repairing a store​

Schema drift​

QuestDB auto-creates a table for an ILP line naming a table that does not exist — with a schema QUBIQ did not choose: the designated timestamp named timestamp instead of ts, no dedup keys, no index, and val_* columns appearing lazily as each type first arrives. Every read path expects ts, so the historian goes silent with Invalid column: ts.

QUBIQ detects this and logs it as an error in System Logs; it is not alarmed. Missing columns, deduplication and the index are repaired in place automatically.

A wrong designated timestamp cannot be: QuestDB refuses to rename a designated timestamp column, so the data must be copied into a correctly-shaped table. There is no button for it; the historian logs that a rebuild is required — contact support.

Stale rollup​

Press Rebuild rollup (historian_1m) in Settings → Historian (needs Edit connections). It re-aggregates the 1-minute rollup. The automatic rebuild declines above a size threshold — this is the deliberate counterpart, run when you can accept the load.

Suspended tables​

If QuestDB's WAL apply job fails unrecoverably, the table is suspended: rows land in the WAL, nothing is applied, and charts go silent while ingestion appears healthy.

The WAL supervisor distinguishes a recoverable stall (resumed automatically) from a poison or missing segment after a disk event or unclean restart (alarmed, because blindly resuming would loop). A suspended-table alarm means: look at QuestDB's own logs.

Ownership conflict​

An ownership banner means another installation claims this store. Confirm the previous owner is retired, then:

Press Take ownership of this store in Settings → Historian (needs Edit connections). This is audited. Do not do it to silence a banner on a store another live system is writing to.

Migrating the historian​

  1. Create and test the new QuestDB connection.
  2. Change the designation in Settings → Historian.
  3. QUBIQ closes the old epoch and opens a new one.
  4. Charts spanning the changeover query both stores and stitch the timeline.
  5. Retire the old store only when its epoch is no longer within any window you need to read.

No export/import step, and no truncated trends.

Monitoring​

WhatWhere
Ingestion buffer statusThe Status panel (needs View connections)
Buffer pressureThe buffer-pressure alarm
Series cache statisticsThe Status panel's historian view
Schema driftAn error in System Logs
Suspension, ownershipHistorian alarms

Troubleshooting​

SymptomCauseAction
No data for a tagHistory not enabled on the tagEnable it
No data at allNo designated historian, or ILP unreachableCheck designation and port 9009
Invalid column: tsSchema driftThe table must be rebuilt; contact support
Charts stop, ingestion looks fineTable suspendedCheck QuestDB logs; the supervisor alarms
Ownership bannerAnother installation claims the storeVerify, then take ownership
Data appears seconds lateAuto-flush interval too highLower it, or rely on the live-tail merge
Disk filling faster than expectedDeadbands missing, or history on chattering tagsSet deadbands
Long-range charts slowRollup staleRebuild the rollup

Next​

→ Querying history