Quality and Time in History New in 1.0.8

A trend is only as honest as the rows behind it. In 1.0.8 every historian row says how good its value was and whose clock stamped it. Rows also mark the stretches when nothing was being recorded.

What a row holds

Column	Holds
ts	The event time: when the value was true, not when it reached the database
quality	Good, Uncertain or Bad, folded from the driver's own label
status	The device's reason, on non-Good rows only (Unknown: Connection down, Tag offline (unsubscribed))
ts_origin	Whose clock ts came from (see below)
ingest_time	When the historian received it

Rows written before 1.0.8 have no quality and read as Good. An existing historian table gains the four columns automatically on upgrade; nothing has to be rebuilt.

Quality in history

A reading that is…	Is stored as	And on a chart
Good	Good	Drawn
Uncertain (including Stale)	Uncertain, with its reason	Drawn, shaded in the warning colour
Bad	Bad, with its reason	Not drawn: the line breaks and the stretch is shaded in the danger colour
Unknown	Bad, with Unknown: … kept as the reason	As Bad

Unknown is Bad in history. A lost connection stamps every tag with its default quality, Unknown. Up to 1.0.7 that was stored as Uncertain, so a frozen last value went on counting in every average and rollup for the whole outage. On this platform Unknown always means there is no trustworthy value, so history treats it as Bad. Stale stays Uncertain: that value was real, just not confirmed recently.

Bad rows are left out of every value. The line, the M4 envelope, the averages, the time-weighted statistics and all three rollup tiers ignore Bad rows. A Bad row carries the last known value, not a measurement, so including it would average a dead sensor into your trend. Uncertain rows are kept.

A quality change is always stored. The deadband compares values within one quality. A device republishing its last value as Bad is a change, however close the number is, so an outage can never vanish inside the band. A tag that goes Bad before it ever had a value gets a quality-only row rather than nothing.

A boolean tag's quality-only row

QuestDB cannot store "no value" in a boolean column; an unset one reads false. From 1.0.8 a boolean is also written as 0/1 to a nullable column, so a quality-only row reads as no value and is not counted as false. Boolean rows written before 1.0.8 are read only when they were Good.

Event time

A row is stored at the time the value was true. The historian takes, in order:

1. The device's own timestamp, when it sent a usable one;
2. otherwise the server's receive time;
3. otherwise its own clock, as a last resort.

A candidate is skipped when it is unreadable (OPC UA and Sparkplug send - for "no device time") or implausible: before 1 January 2000, or more than 10 minutes ahead of now. A device with a bad clock therefore falls back to the server's time instead of planting history in the future. The future limit was one hour up to 1.0.7.

Because the event time travels with the value, a store-and-forward backlog lands at the times the values were read, not the time it drained. QuestDB's deduplication on (ts, tag) then makes a replay idempotent.

Device timestamps on MQTT

A plain MQTT payload carries no standard timestamp or quality. A binding can name where the device puts them: a Timestamp Path (with its format and the time zone for text with no offset) and a Quality Path (with optional code mappings). → MQTT

A broker replays a topic's retained message on every subscribe, and it may be hours old. When the binding reads no device time from it, that replay is shown live as Uncertain (retained message; original time unknown) and is not historised. Otherwise it would be stored as a fresh reading at the subscribe time.

Whose clock: ts_origin

ts_origin	Meaning
device	The device stamped it: OPC UA and Sparkplug with a source time, MQTT with a Timestamp Path, an internal tag whose writer supplied the time
server	QUBIQ stamped it on receipt: Modbus, Siemens S7, EtherNet/IP, SNMP, TCP/UDP, MQTT without a Timestamp Path, expression tags
ingest	Neither time was usable; the historian's own clock
heartbeat	A heartbeat row
shutdown · collector	An outage marker

Up to 1.0.7 the label said device for every row that had a source time. Most drivers fill that with their own poll time, so the label was wrong for most protocols. The stored time was right.

Device clock skew

A device whose clock drifts stores its history at the wrong time, and a time a few minutes off still looks plausible. The historian measures each source connection's device clock: the median of its last 16 samples against the time the server received them. A store-and-forward replay ages both alike, so a backlog does not trigger it.

Past 2 minutes, it raises a Device clock skew system alarm (priority Medium) naming the connection and whether its clock runs ahead or behind, or is not set at all (a date before 2000). The alarm clears once the skew falls below half the limit. Connections whose times QUBIQ stamps itself (ts_origin server) are not measured.

The limit is HISTORIAN_CLOCK_SKEW_ALARM (a duration; 0 turns the alarm off).

Heartbeat rows

A report-by-exception source sends nothing while its value holds still, and a steady plant then looks like one with no data. A tag with a Store interval gets at least one row per interval: the latest value received (even one the deadband suppressed), stamped at the time it is stored, with ts_origin heartbeat.

A heartbeat must stop the moment its value is no longer known to be true. Otherwise a dead device would go on writing its last value forever. So a heartbeat row is written only while all of these hold:

• the tag's last quality is Good;
• the service producing the tag (QBQ-Bridge, or QBQ-Expression for computed tags) is running;
• a real reading arrived within the last 24 hours (QUBIQ_HISTORIAN_HEARTBEAT_MAX_SILENCE).

Heartbeats go through the store-and-forward buffer like the telemetry. During a QuestDB outage they are buffered and land at their correct times, and alarms, the live view and broker egress never see a repeat as a new reading.

Outage markers

A tag stored on change has nothing new to say while it holds steady, so charts carry its last value forward across empty buckets, as PI and AVEVA do. That is correct while the plant runs. Across a period nobody recorded, it draws a confident flat line over a gap. The historian therefore writes one quality-only Bad row per affected tag at the start of an outage. Carrying stops at Bad, and the chart shades the gap with the reason.

Marker	When	status
Shutdown	The whole platform was down. Written at the next start, just after the newest row stored before it stopped.	Shutdown: QUBIQ was not running, no data was collected
Collector down	QBQ-Bridge or QBQ-Expression stopped beating (three missed heartbeats while other services still beat), or came back as a new process	Collector down: <service> was not running, no data was collected

Some things worth knowing:

• Only a real gap is marked. When just the historian or QuestDB was down, the collectors kept publishing into the store-and-forward buffer, which replays the real values. Marking that period would cut the carry of tags that really did hold still.
• Internal tags are not marked. QUBIQ holds their value itself.
• A hung collector that resumes in the same process gets its tags' current Good values stored again 10 seconds later. Otherwise a tag that held still through the hang would stay Bad until it next moved.
• The marker becomes the deadband baseline. The first reading after the outage is always stored, even when it equals the value before it.
• A Shutdown marker waits for the stored data to settle (up to 3 minutes) before it is placed, so the last second before the stop, still draining from the buffer, lands before the marker and not after it.

How far a value is carried

A chart carries an on-change tag's value across empty buckets only within limits:

• it starts from the last non-Bad row before the window;
• it stops at a Bad stretch and at now;
• it stops after 8 hours with no newer row (HISTORIAN_CARRY_MAX_HOLD). Past that, silence cannot be told from a dead collector.

A tag with a store interval is not carried. It is heartbeated, so a gap in its history is a real gap.

→ Querying history