QuestDB
QuestDB is QUBIQ's time-series store. A QuestDB connection can serve two purposes:
- The system historian: the one connection designated to receive tag history.
- A data source, queried like any SQL connection without being the historian.
Only one connection may be the historian at a time.
From QUBIQ 1.0.8 the historian depends on QuestDB 9.4 features, and refuses to designate an older store. A store that is already the historian keeps ingesting, but every start logs a critical error and its charts are not guaranteed to work. Upgrade QuestDB before you upgrade QUBIQ. A QuestDB used only as a data source has no minimum. → Historian configuration
Configuration
| Field | Description |
|---|---|
| Connection Name | Connection identifier |
| Enable Connection | Off stops the connection. On the historian this stops recording tag history, and the form warns you first. |
| Host | QuestDB host |
| SQL Port (PG Wire) | Postgres-wire query port, 8812 by default |
| ILP Port (Ingress) | Line-protocol ingestion port: 9000 for HTTP, 9009 for TCP |
| ILP Transport 1.0.8 | HTTP (recommended) or TCP (legacy) |
| Database Name | qdb by default |
| Use TLS/SSL | Encrypt the connections |
| Username / Password | Query credentials. When editing, turn on Edit Credentials to change them. |
| AI Username / AI Password | Optional read-only credential for the AI assistant |
| Enable ILP Authentication | Token authentication for ingestion (QuestDB Cloud or Enterprise) |
| ILP Token / ILP Key ID | ILP credentials. The token is never shown back; leave it blank to keep the saved one. |
Two ports, two protocols: ILP for writes (fast, append-only) and pgwire for reads (SQL). Both must be reachable.
ILP transport Changed in 1.0.8
| Transport | Port | A row QuestDB rejects |
|---|---|---|
| HTTP (recommended) | 9000 | Is reported. The historian quarantines that one row and stores the rest of the batch. Transport errors are retried. |
| TCP (legacy) | 9009 | QuestDB just closes the connection, so the row can be lost after it has left the store-and-forward buffer |
New connections default to HTTP on port 9000. A connection saved earlier with no transport, on
port 9009, keeps TCP so that nothing breaks on upgrade, and logs a warning on every connect. Edit
it, choose HTTP (recommended), set ILP Port (Ingress) to 9000 (or your QuestDB's HTTP
port), and save. Use TCP only for a QuestDB that does not accept HTTP.
1.0.8 The form now opens on the settings actually stored: the transport, partitions, deduplication, rollup settings, retention, TLS and ILP authentication. Up to 1.0.7 it opened on the defaults, and saving wrote those defaults over the stored values.
Historian settings
These apply when the connection is the designated historian. They live under Ingestion Performance; leave a field blank to use its default.
| Field | Default | Meaning |
|---|---|---|
| Deduplication | On | UPSERT KEYS(ts, tag_id): replayed writes overwrite instead of duplicating. Leave it on. |
| Auto Flush Rows | 5000 | Rows buffered before a commit |
| Auto Flush Interval (ms) | 3000 | Longest wait before a commit. Raise it to about 5000 on a low-ingest historian; QuestDB flags commits under 100 rows as small. |
| Parallel Workers | 1 | Ingestion threads, 1–16 |
| Partition By | Day | Raw table partition unit: Hour, Day, Week, Month or Year. Hour lowers write amplification on a high-ingest historian. |
| Data Retention (days) | 0 (forever) | Raw history older than this is removed, whole partitions at a time. Minimum 1 day. |
Under Rollup (historian_1m materialized view):
| Field | Default | Meaning |
|---|---|---|
| Rollup Partition | Hour | Partition unit of the 1-minute rollup |
| Rollup Refresh Every | 5m | How often the 1-minute rollup re-aggregates |
| 1-hour Rollup Retention (days) 1.0.8 | 0 (forever) | History kept by the hourly tier, in whole months |
| 1-day Rollup Retention (days) 1.0.8 | 0 (forever) | History kept by the daily tier, in whole years |
Changing Rollup Partition rebuilds the 1-minute rollup at the next restart, since QuestDB cannot alter it in place, but only when that rebuild loses no history.
Between ingestion and QuestDB's flush there is a sub-second window where a value is stored but not yet queryable. The historian keeps a small in-memory tail of recent samples and merges it into live chart queries, so charts look live despite batched writes.
Deduplication matters
Store-and-forward can replay a batch that was written but not acknowledged, after a crash between flush and acknowledgement. With deduplication on, the replay overwrites identical rows instead of duplicating them. Turning it off trades correctness for a small write gain. Don't.
What QUBIQ creates
On the designated historian QUBIQ creates and maintains these tables and views:
| Object | Holds | Partitioned by |
|---|---|---|
historian | Every stored sample, with its quality, status, ts_origin and ingest time | Partition By |
historian_1m | 1-minute rollup | Rollup Partition |
historian_1h 1.0.8 | 1-hour rollup, built from raw | Month |
historian_1d 1.0.8 | 1-day rollup, built from raw | Year |
historian_annotation 1.0.8 | Annotations, as append-only revisions | Year |
Every rollup leaves Bad rows out of its values, and the hourly and daily tiers also count each bucket's Bad and Uncertain rows, for quality shading. → Rollup tiers
Retention Changed in 1.0.8
Retention is QuestDB's own table TTL. Each raw and rollup retention is applied as the TTL of its table or view, rounded up to a whole number of partitions and counted from the newest sample. The rollups keep their history whatever the raw retention is. Lowering a retention on the historian asks you to confirm first.
Retention ran a daily DROP PARTITION on the raw table. QuestDB invalidates every materialized view
on a table whose partitions are dropped by hand, and the recovery rebuilt the 1-minute rollup from
the raw rows that were left. The rollup history older than the raw window was lost each day. 1.0.8
uses the TTL, which leaves the views valid. History already lost does not come back.
The Data Retention (days) field was also never saved before 1.0.8. Check the value on your historian connection after the upgrade.
Designating the historian
Exactly one QuestDB connection stores tag history. It is named by a system setting, not by a per-connection flag, so a two-historian state cannot be represented at all. The form shows a System historian badge on the connection that holds the role.
The designation lives in Settings → Historian. Reading it needs View Connections; changing
it needs Edit Connection. Designating a store pre-flights it, checking the QuestDB version and
any existing historian table.
Only the designated connection, plus any still holding historised tags, gets a historian table, rollups, WAL settings, an ingestion worker and retention. A QuestDB connection added purely as a data source stays untouched.
Ownership
Two QUBIQ installations writing to one QuestDB historian table is silent corruption: interleaved
tag IDs, doubled counts, unexplainable gaps. QUBIQ therefore records physical ownership inside
the store itself, not in its own database. An application-level "only one historian" check cannot
see another machine.
If a store is owned by a different installation (a cloned VM, a restored backup, a decommissioned instance), Settings → Historian shows an ownership banner. History is still written, but QUBIQ makes no schema changes, does no rollup maintenance and applies no retention, so the other installation's history cannot be deleted. Claiming the store is deliberate and audited:
Press Take ownership of this store in Settings → Historian (needs Edit Connection). Confirm the previous owner is truly retired before doing this.
Schema drift and repair
QuestDB auto-creates a table for an ILP line that names a table that does not exist, with a
schema QUBIQ did not choose. QUBIQ repairs what it can in place: missing columns, deduplication
keys, the tag_id index. What it cannot repair is a designated timestamp named timestamp instead
of ts. QuestDB cannot rename one, every read fails with Invalid column: ts, and the table has to
be rebuilt into a correctly shaped one.
| Repair | When | Where |
|---|---|---|
| Rebuild rollup (historian_1m) | The 1-minute rollup, or an invalid hourly or daily tier, needs re-aggregating | Settings → Historian (needs Edit Connection) |
| Table rebuild | The base table has the wrong designated timestamp | No button in this release; the historian logs that it is needed |
A rollup rebuild re-aggregates from the raw rows, so once retention has expired raw rows that a rollup still holds, it deletes that older rollup history. The automatic repair therefore runs only when it loses nothing; otherwise the Historian rollup needs a rebuild alarm leaves the decision to you. → Repairing a store
Write-ahead-log supervision Changed in 1.0.8
QuestDB applies writes through a WAL. If the apply job fails, QuestDB suspends the table: rows keep landing in the WAL, nothing is applied, and charts stop.
- Resource problems (disk full, memory, file limits) resume once the resource is freed.
- Anything else gets one plain resume. If that fails, a Historian table suspended alarm (Critical) names the table, the stuck transaction and QuestDB's error.
The historian never skips a WAL transaction by itself, because a skipped transaction's history is gone. Settings → Historian shows each suspended table, and Skip the stuck transactions… does it on your confirmation, audited as a deletion. → Suspended tables
Using QuestDB as a plain data source
A QuestDB connection that is not the historian behaves like any SQL connection: named queries,
query bindings, pipeline SQL nodes, system.db.runQuery. It is useful for time-series you ingest by
other means.
The QuestDB pipeline node has two modes: ingress (high-speed ILP writes) and SQL query. In
ingress mode each row is stored at its event time, taken from a timestamp field in the payload,
else the message's time, before QuestDB's own arrival time is used. A store-and-forward replay
lands at those same times and with the same column types.
1.0.8 → Pipeline nodes
Sizing
| Factor | Effect |
|---|---|
| Tag count with history enabled | Linear on write rate |
| Change rate and storage rules | The biggest lever: a 1 %-of-range deadband often removes 90 % of rows |
| Retention window | Linear on disk |
| Partition size | Affects how finely retention removes data, not total size |
Estimate tags × stored rows per second × 86,400 × retention-days rows, then apply QuestDB's compression. Set storage rules before you set retention; it is far cheaper not to store a row. For the measured ingest rates, see Throughput.
Troubleshooting
| Symptom | Check |
|---|---|
| Designation refused, below the minimum supported version | QuestDB is older than 9.4.0. Upgrade it. |
| Reads work, writes do not | ILP port blocked, the wrong port for the transport (9000 HTTP, 9009 TCP), or ILP authentication on without a valid token |
| A warning about TCP in the log on every connect | A pre-1.0.8 connection still on TCP. Switch it to HTTP. |
Invalid column: ts | Schema drift. The table must be rebuilt. |
| Charts stop but ingestion looks fine | Table suspended. Look for the Historian table suspended alarm. |
| Ownership banner | Another installation claims this store |
| Data appears seconds late | Auto-flush interval too high. Lower it, or rely on the live-tail merge. |
| Disk filling faster than expected | Storage rules not configured, or history on chattering tags |