Plant Data Historian: What It Is, What It Stores and When You Need One

A plant data historian is a database specialised in process time series: it records millions of timestamped values (temperatures, flows, counters, states) and keeps them for years while queries remain fast. It is the plant's memory. In this guide we look at how it differs from a conventional database and from a data lake, what exactly it stores, where it fits within an industrial data platform and how to estimate its volume with a numerical example.

What a plant data historian is

Every plant generates a continuous stream of values: the PLC updates counters each cycle, sensors publish temperatures and pressures, the SCADA logs alarms and state changes. That stream is ephemeral by nature; if nobody persists it, the value from an hour ago no longer exists. The historian (also called a plant historian or data historian) is the component that turns that stream into history: it writes each value with its timestamp and quality flag, compresses it and keeps it available for query over years.

Its reason for existing is a technical asymmetry: a plant's write pattern (an enormous number of small, time-ordered values that are never modified afterwards) looks nothing like the pattern transactional databases were designed for. From that asymmetry come both the classic vendor historians and the open-source time-series databases, such as InfluxDB, which play the same role with a different licensing model. In this guide we use "historian" for the whole category: what matters is the function, not the brand.

Three questions summarise what a historian answers well:

What value did this signal have at this instant?Reconstructing the state of the line at the moment of a failure, minute by minute.
How has this signal evolved over this period? A month of furnace temperature trends or of a line's energy consumption.
What is the aggregate per interval? Hourly averages, maxima per shift, counters per day: the raw material of OEE, reports and models.

Historian versus conventional database

The usual objection is reasonable: if the company already has a relational database, why not store the signals there? You can, and many plants do start that way, until volume takes over. A relational database treats each value as a generic row: indexes designed for updates, transactions that add nothing here and a much higher storage cost per value. With a few dozen signals it works; with thousands, long-range queries degrade and maintaining indexes and partitions becomes a job in itself.

The historian inverts the priorities. It assumes data arrives ordered by time and is never modified, and in exchange offers sustained bulk writes, compression specific to time series and query functions a relational database does not ship with: resampling, windowed aggregates, interpolation, last known value per signal. It does not replace the business database (orders, master data and users still live in the relational world); it complements it on the ground where the relational model performs worst.

What it stores exactly: tags, quality and compression

The historian's unit of work is the tag: a signal identified by a name, with a series of values over time. Each sample stores at least three things: the timestamp (ideally the source timestamp, from the device that measured, not the time of arrival at the server), the value and a quality flag stating whether the reading was reliable at that instant. That third field is what separates a serious historical record from a table of numbers: when the gateway loses its connection to the PLC, the gap is marked as such, and whoever queries it in two years will know there was no data there, not that the value was zero.

Compression deserves understanding because it affects what you will see later. There are two families: lossless compression, which exploits the fact that consecutive values are similar to encode them in few bits, and deadband filtering, which simply discards samples when the value has not changed by more than a threshold. The second saves far more space, but it is an engineering decision: a poorly chosen deadband erases precisely the micro-variation a later analysis needed. The practical rule is conservative: narrow or no deadband on critical process signals, more aggressive on slow, stable ones.

Historian vs industrial data lake

The comparison that dominates architecture conversations today is no longer historian versus relational database, but historian versus industrial data lake: the analytical store, usually in the cloud, where plant data joins business data for large-scale analysis. They are sometimes presented as alternatives, and they are not: they answer different questions about the same data.

Criterion	Historian	Industrial data lake
Question it answers	What happened in the process, signal by signal, second by second	What patterns emerge when crossing process, quality, energy and business
Data model	Time series per tag, fixed and compact schema	Files and tables with flexible schema, raw and refined data
Query latency	Seconds for operational ranges: it is a plant tool	Designed for analytical jobs, not for daily operation
Resolution	Maximum: each sample with its source timestamp	Usually aggregates or extracts; fine detail is optional
Typical location	On or near the plant; useful even without an outside connection	Cloud or corporate data centre
Typical user	Operations, maintenance, process engineering	Analytics, data science, management
Dominant cost	Licence or local infrastructure, bounded and predictable	Cheap storage, but querying and data movement are billed
Risk if missing	The plant loses its operational memory	The business loses cross-cutting analytical capability

A healthy architecture does not choose: it chains them. The historian captures and serves the detail at the plant; the data lake receives, from the platform, the aggregates and extracts that analytics needs, published as datasets with an agreed schema and frequency. That contract between the two worlds is exactly the subject of the guide to industrial data contracts. The expensive mistake is the reverse shortcut: dumping raw plant data into the lake "to have everything" and discovering later that without context or governance nobody can use it.

Its place in the data platform

In a layered industrial data platform, the historian lives in the data layer and behaves like one more consumer: it subscribes to the broker and persists everything that flows through the namespace. That position has two practical consequences worth underlining.

First: the historian must not be the integration source. In many plants the historical store ended up acting as a distributor: every new application queried the historian because it was the only place where everything lived. That turns it into a bottleneck and couples every application to its schema. In the platform, real-time distribution is done by the broker; the historian stores and serves history, which is what it does well.

Second: the completeness of the historical record is earned at the edge, not at the server. If the gateway has no persistent local storage, every network outage opens a gap that no historian can fill in afterwards. The store-and-forward mechanism that prevents it is developed in the guide to industrial edge with buffering and QoS, and the distribution pattern the historian subscribes to, in the guide to unified namespace and MQTT Sparkplug B.

Sizing: a numerical example

Historian numbers are less frightening once you do the arithmetic. Take a generic mid-sized plant: 4 lines, 500 signals in total, with an average sampling rate of one reading every 5 seconds per signal.

Writes per second: 500 signals / 5 s = 100 values per second. Any modern time-series database sustains this on a modest server; the problematic order of magnitude starts several zeros higher.
Samples per year: 100 per second is 8.64 million per day, around 3,150 million per year.
Volume on disk: at a raw cost of 16 bytes per sample that would be about 50 GB per year; with typical time-series compression (consecutive values are similar) the real figure usually ends up at a fraction of that. Even uncompressed, five years of history fit on an ordinary disk.
With deadband: if half the signals are slow and deadband filtering discards 80 per cent of their samples, the annual volume drops roughly by half. It is the most powerful saving lever and also the most delicate, as we saw above.

The moral: for a mid-sized plant, the historian is not a storage-cost problem; it is a design problem. What determines its value is not the gigabytes but the prior decisions: which signals go in, under which names, at which resolution and with which timestamp.

Common mistakes with the historian

Historising without a naming convention. A historical record where the same magnitude has three names depending on who configured the tag is a record only its author understands. The naming hierarchy is agreed before creating the second tag, just as with the platform's namespace.

Storing the arrival timestamp. If the value is stamped when it enters the server rather than at the source, any network delay reorders history. Root-cause analyses, which depend on knowing what happened before and what after, are compromised beyond repair.

Aggressive deadband by default. Configuring the same deadband for every signal saves disk today and destroys analyses tomorrow. Filtering is decided signal by signal, with process judgement.

Using it as a universal integrator. Ten applications polling the historian in a loop to find out "the latest" is the classic symptom of a missing distribution layer. The latest travels through the broker; the historian is for history.

Never testing restoration. Years of history are only worth what their backup is worth, and a backup that has never been restored is a hypothesis. The recovery test is scheduled like any other maintenance.

Frequently asked questions about plant data historians

Do I need a historian if my SCADA already keeps history?

The SCADA's history is usually sized for operation: days or weeks of retention, limited resolution and queries designed for the SCADA itself. If you only need to review recent operating trends, it may be enough. As soon as you need years of retention, access from outside the SCADA or feeds for calculations and models, a dedicated historian within the platform is the path with the least friction.

Can a data lake replace the historian?

Not in practice. The data lake answers analytical questions by crossing sources, but it is not designed to serve operations the detail of a signal with seconds of latency, nor to keep working when the plant loses its outside connection. The usual approach is to chain them: the historian stores and serves the detail at the plant and the data lake receives aggregates and extracts under a defined data contract.

Classic historian or open-source time-series database?

Functionally they play the same role: bulk writing of timestamped values and efficient range queries. Classic vendor historians bring native integration with their ecosystem and mature client tools; open-source time-series databases, such as InfluxDB, bring deployment flexibility and no per-tag licensing cost. The decision depends on the existing ecosystem, the volume and the team that will maintain it.

How much resolution should be stored?

As a rule, the maximum the source offers for critical process signals and a relaxed resolution for slow, stable ones. Storage is cheap compared with the cost of discovering that the micro-variation an analysis needed was filtered out two years ago. Aggregates (averages, maxima per shift) are calculated afterwards from the detail; the reverse path does not exist.

If you are weighing up how to give your plant a reliable process history, at Captia Connect we deploy the full chain: multi-protocol capture at the edge, local buffering so network outages do not open gaps and time-series persistence within an industrial data platform that serves the data to operations and to the business. The first step is an inventory of signals and protocols.