Industrial Data Contracts: Schemas, Data Quality and Lineage

An industrial data contract is an explicit, verifiable agreement on what each plant data source publishes: which fields it carries, in which units, at what frequency and who answers when something changes. It turns plant data into a product with guarantees, rather than a stream every consumer interprets in their own way. Throughout this guide we unpack the four parts of the contract (schema, quality, lineage and versioning) and the question that holds them all together: who owns each piece of data.

What an industrial data contract is

In corporate data engineering, the term data contract designates a formal agreement between whoever produces a dataset and whoever consumes it: structure, semantics, quality and evolution rules are written down and are machine-verifiable. The idea comes from the world of data warehouses and data mesh architectures, but it fits naturally on the shop floor, where the problem is even more acute: the producers are PLCs, sensors and MES systems that nobody designed with a thought for the analysts who would consume their signals ten years later.

Anyone who has ever integrated factory data will recognise the usual contract-free situation. A systems integrator configures a gateway that publishes an oven temperature on an MQTT topic. Two years later, a continuous improvement team builds a dashboard that reads that topic. Nobody documented whether the temperature is in degrees Celsius or Fahrenheit, whether the value is instantaneous or a moving average, or what it means when the field arrives empty. The dashboard works until someone recalibrates the sensor, renames the field or adjusts the sampling interval. Then the panel shows wrong figures without warning, and trust in the whole system suffers.

A data contract attacks that problem in writing. For every flow that matters, it defines four things:

• Schema: which fields exist, what type they are and what they mean.
• Quality: what guarantees the producer gives on completeness, frequency, valid ranges and units.
• Lineage: where each value comes from and which transformations it has been through before reaching the consumer.
• Evolution: how the schema is versioned and what process a compatibility-breaking change must follow.

Throughout the article we will use a single example: the temperature sensor on the curing oven of a paint line. It is a deliberately small case, because the central thesis of data contracts is that rigour is applied flow by flow, not through a documentation big bang. That sensor publishes every five seconds on a topic of an MQTT broker, within a naming hierarchy organised as a unified namespace. Each section will add one layer of contract on top of it.

The schema: what every topic and table promises

The schema is the part of the contract that describes the shape of the data. In an industrial environment there are two typical surfaces where that shape lives: messaging topics (MQTT, Kafka) and the tables or views where the data ends up persisted (historian, time-series database, analytical store). Both need a schema, and both need the schema to say more than the types.

For the curing oven, a useful topic schema does not stop at saying that value is a number. It says, at a minimum:

topic: planta-valencia/pintura/horno-curado/temperatura
payload:
  value:      float      # temperature in °C, 0.1 resolution
  timestamp:  ISO 8601   # sampling time at the sensor, UTC
  quality:    enum       # GOOD | UNCERTAIN | BAD (OPC UA semantics)
  source_id:  string     # source PLC tag: TT-4012

Every line is a promise. The producer promises that the unit is the degree Celsius and will not change without notice. It promises that the timestamp is the sampling time at the source, not the arrival time at the broker, a nuance that decides whether process analytics can be trusted. It promises a per-message quality flag, following the status-code semantics popularised by OPC UA, so the consumer can tell a good value from a suspect one without guessing.

The specific format matters less than the existence of the agreement. In industrial MQTT ecosystems, the Eclipse Foundation's Sparkplug B specification solves part of the problem out of the box: it defines the payload structure, metric typing and the birth certificates in which each node declares which metrics it publishes and of what type. That birth certificate is, in effect, a self-declared schema. In Kafka pipelines or REST APIs, the role is played by Avro, Protobuf or JSON Schema alongside a schema registry. And at the table end, data tests (in the style popularised by dbt) verify that every column honours the agreement on every load.

What no format solves on its own is semantics. Calling a field temperatura does not say whether it is the oven air temperature or the part surface temperature. That layer comes from the contract's textual description and, where one exists, the domain information model: the OPC UA companion specifications for specific sectors, or the hierarchical equipment model of ISA-95 that usually gives structure to the unified namespace itself. The practical rule: if two engineers can read a field name and understand different things, the schema is not finished yet.

Industrial data quality: completeness, frequency and units

The schema says what shape the data has; the quality section of the contract says what operational guarantees it offers. In plant data, three dimensions concentrate most of the real-world problems.

Completeness

What percentage of the expected samples actually arrives? A sensor publishing every five seconds should generate 17,280 samples per day; if 15,000 arrive, there is a 13 per cent gap that any daily average will carry along silently. The contract must set a completeness threshold and, more importantly, define what a gap means: absence of a message, a message with a null value, or the last value retained? All three conventions exist on the shop floor and they mix badly. Protocols with edge buffering (store and forward) reduce losses from network outages, but they introduce late arrivals the consumer must know how to handle: that is why the origin timestamp is mandatory in the schema.

Frequency and timeliness

Sampling frequency (how often the sensor measures) is not the same as publication frequency (how often the value is sent), nor as end-to-end latency (how long it takes to be available to the consumer). The contract must distinguish them. For the curing oven: sampling every 5 seconds, publication on change of value with a 0.5 °C deadband, maximum latency of 30 seconds to the historian. A supervision dashboard lives comfortably with that latency; a safety interlock does not, which is why interlocks are never built on this layer. Making the intended use explicit prevents anyone from building on the flow something the flow cannot sustain.

Units and ranges

Units are the cheapest error to prevent and the most expensive to discover late. The contract fixes the unit of every field and the physically plausible range: the curing oven operates between 120 and 200 °C, so a value of 950 is almost certainly a sensor failure or an upstream scaling change, and must be flagged as quality BAD instead of entering the averages. These validations run where the data enters the system, typically in the gateway or in the OT/IT integration layer, which is the natural place to enforce contracts because it is where the data crosses from one organisational domain to another.

One table summarises the three dimensions applied to the example:

Lineage: from sensor to dashboard without leaps of faith

Lineage answers the question that always surfaces in the worst possible meeting: this number on the dashboard, where exactly does it come from? In a typical industrial pipeline, the oven temperature passes through five or six stages before reaching a panel: the physical sensor, the PLC analogue input with its scaling, the gateway that reads the tag and publishes it, the broker, the ingestion process that writes it into the time-series database and the aggregations that compute averages per batch or per shift. Every stage can transform the value, and every undocumented transformation is a leap of faith.

Documenting lineage does not require exotic tooling. The minimal version is an explicit chain in the contract itself: tag TT-4012 on the oven PLC, 4-20 mA scaled to 0-300 °C on the input card, published untransformed by the gateway, 1-minute moving average computed at ingestion, per-batch average computed in the analytical layer. With that chain written down, when the panel shows a batch at 168 °C anyone can walk the path backwards and pinpoint which stage to inspect. Without it, the investigation starts with configuration archaeology.

Two practices keep lineage sustainable. First: keep the raw data. Aggregations can be recomputed; the sensor's original values cannot. If the moving average turns out to be badly parameterised, having the raw data lets you correct the entire history. Second: propagate the origin identifier (source_id in our schema) through every stage, so that any aggregated record can be traced back to the tags that fed it. In generalist data ecosystems, open standards such as OpenLineage formalise this trace at the level of jobs and datasets; on the shop floor, the discipline of stable identifiers within a hierarchical namespace fulfils much of the same function.

Schema versioning: change without breaking

Schemas change, and a contract that does not anticipate change breaks with the first improvement project. The central distinction comes from API engineering: there are backward-compatible changes and breaking changes. Adding an optional field (for instance, the oven setpoint next to the measured temperature) is compatible: existing consumers ignore it. Renaming value, switching the unit to Fahrenheit or making an optional field mandatory breaks everyone who reads the flow.

The contract sets rules for both cases:

1. Compatible changes are deployed without ceremony, but they are recorded: the schema has a version number and a change history, just like code.
2. Breaking changes require a new version and a period of coexistence: the producer publishes version 2 in parallel with version 1 for an agreed period, consumers migrate at their own pace and the old version is retired on an announced date. In a unified namespace this materialises naturally by including the version in the topic hierarchy or in the payload metadata.
3. Never change the meaning while keeping the name. It is the most important rule and the most frequently violated: reusing an existing field for a different quantity produces silent errors that no type validator can detect.

Schema registries automate the policing of these rules in Kafka and similar ecosystems: they reject at source any publication whose schema breaks the declared compatibility. In MQTT environments with Sparkplug B, the birth certificate plays a similar role by declaring each node's metrics, although the discipline of version coexistence remains organisational. Technology polices the shape; the agreed process protects the meaning.

Who owns each piece of data

Everything above rests on an organisational question: who answers for each flow? A contract without an owner is a document; with an owner, it is a commitment. In the usual division of labour in a factory there are three candidates and all three have a point. Maintenance and automation know the sensor and the PLC. The IT or data department operates the broker, the ingestion and the store. Production is who understands what the data means and who suffers its errors.

The criterion that works best is what the data mesh approach calls domain ownership: the data owner is whoever can answer for its meaning and its correctness at source, not whoever operates the pipes. For the curing oven temperature, the natural owner is the person responsible for the paint process, supported by automation for the physical part of the chain. IT owns the platform (broker, historian, infrastructure availability), which has its own service contract, distinct from the data contract.

Separating the two planes resolves most disputes:

In practice, every contract carries the name of a person or a role, not of a generic department. And the test of whether ownership is real is simple: when a quality validation fails, the alert reaches someone who can fix the cause, not a shared mailbox.

How to introduce data contracts without stopping the plant

The classic mistake is trying to put the whole plant under contract at once: an inventory of thousands of tags, an exhaustive template and a documentation project that dies of exhaustion. The path that works is incremental and driven by value, not by exhaustiveness.

1. Start with the flows that already hurt. Those feeding decisions (production reports, KPIs, models) that have failed at least once are the first candidates. One contract per flow, one or two pages long.
2. Write the contract of the current state, not the ideal one. Document what the flow does today, defects included; improvements (adding per-message quality, correcting timestamps) become versioned changes on top of that baseline.
3. Automate verification as early as possible. A contract that lives only in a document goes stale within months. Completeness, range and frequency guarantees must run as continuous validations in the ingestion layer, with alerts to the data owner.
4. Anchor the contracts to the naming structure. If the plant has a unified namespace, each node of the hierarchy is the natural place to attach its contract. The naming structure and the contracts reinforce each other: one provides stable addresses, the others provide guarantees about what lives at each address.
5. Treat the contract as part of every project deliverable. Each new integration (a machine, a system, a migration) delivers its contract alongside the technical work, just as it delivers drawings or electrical documentation.

The cumulative result is what is usually called data as a product: every relevant flow in the plant has a known shape, measured guarantees, a traceable history and a named owner. On that foundation, dashboards, reports and models stop being fragile constructions built on sand and become consumers of a product with terms of service.

Frequently asked questions

Data contracts are the trust layer on top of connectivity: first the data arrives, then the data makes promises. If you are building that foundation in your plant, from OT/IT integration to the naming structure and per-flow guarantees, at Captia Connect we cover exactly that stretch: from the sensor to data you can trust. Contracts are one more piece of a complete industrial data platform, where they sit alongside acquisition, storage and data consumption.