A wrong value stands out in the trend immediately. A wrong timestamp does not. That an alarm was recorded two seconds before the pressure rise that triggered it is only noticed during root cause analysis, after a wrong conclusion has been reached. Below, we describe where production data timestamps and quality information go wrong along the data collection chain, how this is detected in the field and which settings prevent it; the target readers are teams working on IIoT, historian and MES projects.
The difference between source and server timestamps
A value passes through several layers before it reaches the historian from the sensor: PLC, gateway or OPC UA server, collector, database. Each layer can add its own timestamp to the value. OPC UA models this distinction explicitly: a DataValue structure carries, alongside the value, a SourceTimestamp (the moment the value was produced or changed at the source), a ServerTimestamp (the moment the server received or updated the value) and a StatusCode.
The problem starts when a layer in the chain discards the source timestamp and writes its own time. Common examples:
- When writing the value from OPC UA to the database, the collector uses its own “now” instead of the
SourceTimestamp. - When sending buffered data after a connection outage, the gateway writes the send time on every record; an hour of data is squeezed into a few seconds.
- There is no timestamp in the MQTT payload; the service reading from the broker uses the message’s arrival time. In this case, the value is recorded with the receipt time and is not labelled as source time.
- The PLC value is read periodically and the read time is recorded; where within the read interval the change happened is unknown.
The time of the event should be stamped at the layer closest to the event and carried unchanged along the chain. Keep three times in separate columns in the record: source time (SourceTimestamp, the moment the value was stamped at the data source), server time (ServerTimestamp, the moment the server received the value) and record time (the moment the value was written to the database). According to the OPC UA specification, if there is no source applying a timestamp, the source time is left empty (null); in that case, do not write the receipt time into the source time column. The difference between the columns shows latency and buffering.
Clock drift and time synchronisation: NTP or PTP
A source timestamp is only useful if the source’s clock is correct. The clocks of unsynchronised PLCs, HMI panels, gateways and servers can drift by seconds within days and by minutes within months. A PLC whose clock is set by hand can end up at a completely wrong time after a daylight saving change or a battery replacement.
| Criterion | NTP | PTP (IEEE 1588) |
|---|---|---|
| Typical accuracy (LAN) | Millisecond level | Sub-microsecond with hardware timestamping |
| Infrastructure requirement | Standard network, NTP server | PTP-capable switches (boundary/transparent clocks), grandmaster |
| Device support | Common on PLCs, HMIs, servers and gateways | More limited, hardware-dependent |
| Suitable use | Historian, OEE, MES, event logging | Event ordering, power systems, motion control |
For most production data applications, NTP is enough. What matters is that all devices are connected to the same time source: in the OT network, a local NTP server is set up, preferably fed from GPS or an upstream NTP source, and PLCs, HMIs, gateways, the historian and MES servers are pointed to it. NTP’s behaviour is defined in RFC 5905. Situations requiring precise time synchronisation at network level are covered in the article TSN and time synchronisation.
To verify the synchronisation status:
# Linux (chrony): selected source and offset
chronyc tracking
chronyc sources -v
# Windows: synchronisation status and source
w32tm /query /status
w32tm /stripchart /computer:ntp.ot.local /samples:5 /dataonly
On the PLC side, check in the project settings that the CPU’s NTP client is enabled and the server address has been entered correctly. A common mistake is that the NTP server address has been entered but UDP 123 has not been opened to the PLC cell on the OT firewall. A simple check that reads the PLC clock as a tag and compares it with the server clock catches drift early.
Quality codes and good/bad/uncertain data
In OPC UA, every value has a StatusCode. The top two bits of the code indicate severity: Good (0x0…), Uncertain (0x4…) and Bad (0x8…). The sub-fields carry the reason; for example, Bad_NoCommunication indicates the source cannot be reached, Bad_WaitingForInitialData that no value has arrived yet, and Uncertain_LastUsableValue that the last usable value is being carried. The complete list of codes is in the OPC UA specification (OPC Foundation online reference).
To make sure quality information is not lost in analysis:
- The historian schema should have a quality field alongside the value; at least the good/uncertain/bad distinction should be stored.
- Bad-quality records should not be discarded but flagged. A discarded record also erases the reason for the data gap.
- In summary and calculation queries, bad-quality values should be excluded by default, and how uncertain values are handled should be defined.
- Quality must be preserved in protocol conversion. For a value from a source that carries no quality information, such as Modbus, the gateway must not keep showing the value as “good” when there is a communication error.
A typical example of the last item: the cable has broken, and the gateway keeps publishing the last value read with “good” quality. The green connection indicator on screen does not show that the data is current and correct.
The effect of deadband and sampling on analysis
OPC UA subscriptions have two separate parameters: the sampling interval (how often the server samples the source) and the publishing interval (how often notifications are sent to the client). In addition, an absolute or percentage deadband can be defined with a data change filter. Their effects on analysis:
- A peak lasting less than the sampling interval is never seen. A 200 ms spike in a current signal sampled once per second may not be recorded.
- On a tag with a deadband, a flat line between two consecutive records does not mean “the value was constant”; it means “the value stayed within the deadband”. Whether the trend tool draws a step or a straight line between two points changes this interpretation.
- If averages in production monitoring dashboards are calculated with a simple arithmetic mean on unevenly spaced data, periods with frequent changes gain too much weight. A time-weighted average should be used.
- No deadband is used on counter and status signals; every change is recorded.
Data gaps and outage detection
In a system using a deadband, “no data is coming” can mean two things: the value is not changing, or the connection is down. To tell them apart, at least one of the following methods should be set up:
- The collector publishing the connection state for each source as a separate tag.
- Reading a heartbeat counter in the PLC that increments every second; if the counter is not increasing, there is a problem with the source or the connection.
- Using a protocol that reports state with birth and death messages, such as Sparkplug B.
Window functions are used to find gaps in historical data. The PostgreSQL/TimescaleDB query below lists gaps longer than 5 minutes per tag in the last day:
SELECT tag_id, previous, time AS next_time, time - previous AS gap
FROM (
SELECT tag_id, time,
lag(time) OVER (PARTITION BY tag_id ORDER BY time) AS previous
FROM tag_data
WHERE time > now() - INTERVAL '1 day'
) t
WHERE time - previous > INTERVAL '5 minutes'
ORDER BY gap DESC;
The threshold is set per tag: a 10-second gap on a heartbeat tag that changes every second is an outage, while 5 minutes may be normal on a temperature tag with a deadband. If the collector’s buffering (store and forward) feature preserves data during an outage, this query verifies that retrospectively written records arrive with their source timestamps. Verify this in a test environment or in a planned window approved by the plant: the collector’s network connection is cut for a few minutes, then you check whether the gap has been filled and whether duplicate records have been created. For details of buffering and gateway selection, see the article choosing an edge gateway.
Table of example failure scenarios
| Symptom | Likely cause | Fix |
|---|---|---|
| The alarm appears before the event that triggers it | Clocks on different devices are not synchronised | Connect all devices to the same NTP source, monitor offset |
| Many records in the same second after an outage | The source timestamp is lost when the buffer is sent | Preserve source time in the collector, store receipt time separately |
| A one-hour overlap on the same date every year | Storing in local time, daylight saving transition | Use UTC in all layers |
| The trend continues flat even though the connection is down | The gateway carries the last value with good quality | Pass on the quality code, add a connection state tag |
| Short peaks are missing from the trend | Sampling interval or compression tolerance is too wide | Narrow the interval on critical tags, or capture the peak value in the PLC |
| OEE inconsistent at the shift boundary | PLC and MES clocks differ | Clock synchronisation and UTC; calculate the shift boundary in one place |
Data quality checklist
- There is a single time source on the OT network; PLCs, HMIs, gateways and servers are synchronised to it.
- Clock offset is measured periodically and a warning is raised when a threshold is exceeded.
- The source timestamp is preserved along the chain; receipt time is stored in a separate field.
- All timestamps are UTC; local time is used for display only.
- Quality information is stored with every record; bad-quality records are not discarded.
- There is no deadband on counter and status signals; deadbands on analogue signals are chosen according to measurement uncertainty.
- There is a connection state or heartbeat tag for every source.
- Data gaps are detected with a regular query or report.
The reliability of OEE and root cause analysis depends directly on the items on this list. If the timestamp is wrong, the analysis is wrong too. The effects on the OEE side are described in the article the right data sources for OEE calculation. For designing and verifying the data collection chain, see our IIoT and production data collection service.
Frequently asked questions
Which timestamp should be stored in the historian: source or server?
Analysis is based on the source timestamp, because it represents the moment the event occurred. If possible, the server or receipt time is also stored in a separate field; the difference between the two helps diagnose latency and buffering problems. If the source timestamp is unreliable, for example because the device clock is not synchronised, this must be flagged explicitly.
Is NTP enough for PLCs, or is PTP needed?
For data collection, OEE and event ordering, the millisecond-level accuracy NTP provides is enough in most cases. If events on different devices need to be ordered at microsecond level, for example for event recording in power systems or motion control analysis, hardware-assisted PTP is needed.
Should values with bad quality be written to the historian?
They should, but together with the quality information. Discarding a bad-quality record makes it impossible to understand later why there was no data in that interval. On the reporting and calculation side, how bad and uncertain-quality values are handled must be defined explicitly.
Does no data arriving on a tag with a deadband mean the connection has dropped?
Not on its own; as long as the value stays within the deadband, no new record is created. To tell them apart, the collector must publish its connection state as a separate tag, a periodic heartbeat signal must be read, or a protocol with state notification such as Sparkplug B must be used.