I spent the first part of my data career in a world of batches. Data sat still in databases, and at night a job woke up, processed everything, and went back to sleep. That model is so ingrained in how we think about data that it's almost invisible — it just is how data works. And then I started building for buildings, and discovered that for a whole class of problems, the batch model isn't just suboptimal. It's a fundamental mismatch with the shape of reality. This is a post about that mismatch, and the shift in thinking that resolves it.

The core realisation is simple to state and surprisingly deep once you sit with it: a building does not produce data. A building produces events.

Batch thinking versus event thinking

Let me draw the contrast sharply, because the whole argument lives in it.

Batch thinking treats data as a stock — a pile of records that exists, sits still, and gets processed periodically. You accumulate a day's worth, then you process the day. The fundamental unit is the dataset, and the fundamental rhythm is the schedule. This model was shaped by a world where data lived in databases and compute was something you ran overnight because that's when the machine was free.

Event thinking treats data as a flow — a continuous stream of things that happen, each meaningful the moment it occurs. A sensor reading isn't a record waiting to be batched; it's an event, a small announcement that something just changed, right now. The fundamental unit is the event, and the fundamental rhythm is as it happens.

A building lives entirely in the second world. A room becomes occupied — that's an event, and it's most valuable at the instant it happens, not at 2am when a batch job finally notices. Temperature crosses a threshold — event. A door opens, a machine faults, energy draws spike — events, all of them, each carrying its full meaning in the moment and beginning to decay the instant it's delayed.

Why forcing events into batches goes wrong

You can force a building's events into a batch model. Collect the readings, store them, process them nightly, report in the morning. It works, in the narrow sense that data comes out the other end. But you've thrown away the thing that made the data valuable, and you've done it structurally, so no amount of tuning gets it back.

  • You've destroyed timeliness. The whole point of "the room is occupied" is that you can do something now — adjust the climate, note the space is in use, react. Learn it eight hours later in a report and you've converted a live signal into a historical footnote. The value was in the immediacy, and batching is a machine for discarding immediacy.
  • You've inverted the natural model. You're taking something that is natively a stream and freezing it into stock so you can process it the way you're used to, then wondering why everything feels sluggish and after-the-fact. You're fighting the grain of the data.
  • You've made "react to what's happening" architecturally impossible. Not hard — impossible. A batch architecture has no place to put a real-time reaction, because its entire structure assumes nothing needs to happen until the next scheduled run. The capability you most want isn't slow; it simply has nowhere to live.

What event-driven architecture actually means here

So the shift is to build an architecture whose fundamental unit is the event — where things flow through and get reacted to as they arrive, rather than piling up to be processed on a timer. Concretely, that reorients the design around a few ideas:

  • Events are first-class and move through a pipeline. Instead of landing in a table to wait, each event flows through a processing path the moment it arrives — evaluated, routed, reacted to, and also archived, but the archiving is the side-effect, not the main event. The live reaction is the point.
  • Reacting-now becomes a place in the architecture. Because events flow continuously, there's a natural home for "when this happens, do that" — a stream processor watching the flow, firing when a condition is met. The capability that was impossible in batch becomes the centre of gravity in events.
  • History is derived from the stream, not the other way round. You still get your reports and your trends — but they're accumulated from the event flow, rather than the event flow being an inconvenience you batch away. The stream is the source of truth; the stored history is its sediment.
Batch asks "what happened?" and answers it on a schedule. Events ask "what is happening?" and answer it now. For a building — which is always doing something — only the second question is the interesting one.

A worked example: one occupancy event's journey

Let me trace a single event through, because it makes the abstraction concrete. A presence sensor on the third floor fires: the room just became occupied. In a batch world, that fact sits in a table until tonight's job notices it. In an event world, watch what happens to it at once, in parallel:

The event flows into the stream, and several things react to it simultaneously. A live dashboard updates — someone watching the building sees the third floor light up, now. A rule evaluates — if this room becoming occupied should trigger something, climate or lighting or a notification, it fires immediately. And the same event flows onward to storage, joining the accumulating history from which, months later, you'll derive "third-floor occupancy patterns this quarter." One event, arriving once, serving the live reaction and the historical record at the same time — because the architecture treats it as a thing that happened, not a row to be processed later.

History as a side-effect of the stream

That example surfaces the idea that most reorganised my thinking: in an event-driven design, your historical reports are derived from the stream, not stored instead of it. The stream of events is the source of truth. The tidy summary tables your dashboards read are essentially materialised views over that stream — accumulations you build up as events flow past.

This feels backwards at first if you come from databases, where the stored table is the truth and a stream is just a means of updating it. Invert it: the events are the truth, and every table is merely one particular, pre-computed way of looking at them. The practical payoff is real. When someone asks a new historical question you never anticipated, you're not stuck — because you kept the raw events, you can compute a fresh view of the past, rather than discovering you only ever stored the one summary someone happened to think of at the time. Batch architectures throw away the questions you didn't know to ask. Event architectures keep them open.

The wider lesson

What's stayed with me from making this shift is that it's not really about IoT or buildings. It's about matching your architecture to the actual shape of your data, rather than to the model you happen to be used to. Batch isn't wrong — for genuinely stock-shaped data, data that really does sit still and only needs periodic processing, it's exactly right. The error is applying it reflexively to flow-shaped data because batch is the water we all swim in.

Buildings just make the mismatch impossible to ignore, because a building is so obviously, continuously alive — it never stops producing events, so the cost of pretending otherwise is right in your face. But once you've felt the difference, you start seeing flow-shaped data everywhere it was being awkwardly forced into batches, quietly stripped of its timeliness, in domains that have nothing to do with sensors. The building taught me to ask, of any data, a question I now ask constantly: is this really a stock I should process on a schedule — or a flow I should have been reacting to all along?