Trend Log Interval, Storage, and Controller Buffer Overrun
How much data a trend system makes, and the place it is silently losing some.
Example
You enter
- Points logged site-wide 5000
- Sample interval (min) 5
- Retention (years) 2
- Bytes per sample 16
- Points on one controller (0 to skip) 200
- Controller buffer depth (samples) 1000
- Archive poll interval (min) 60
- Change-of-value changes per point per day (0 to skip) 100
- Faster interval to compare (min, 0 to skip) 1
You get
- Samples per point per year 105120
- Total samples 1051200000
- Storage gb 16.8192
- At the faster interval 84.1 GB
- Controller samples per hour 2400
- Buffer fill (min) 25
- Poll loss (%) 58.3333
- Cov total samples 365000000
- Cov storage gb 5.84
- Cov reduction (%) 65.2778
Details, formula, and sources
STORAGE IS RARELY WHAT STOPS A TREND SYSTEM: a 5,000 point site logged every five minutes and kept two years is under 20 GB, which is nothing on a modern server. Trend systems fail by collecting too MUCH rather than too little -- a site that logs every point at a fast interval forever accumulates a dataset nobody queries, on a server nobody sized, and the volume itself becomes the reason the data is eventually purged. Choosing the interval from the question being asked is what keeps a trend system useful: roughly fifteen minutes for energy analysis, five for fault detection, one or faster for control diagnosis, and two years of retention for energy work against weeks for commissioning. THE CONTROLLER BUFFER IS THE FAILURE THAT ACTUALLY CAUSES TROUBLE, AND IT LEAVES NO TRACE. A field controller holds a fixed number of samples and WRAPS when it is full, so if the supervisory poll is slower than the fill rate, the oldest samples are discarded before they are collected. The archive receives a continuous-looking series with unmarked gaps in it, the trend plots as a line because the graph joins the points it has, and any analysis built on it is quietly wrong. A controller with two hundred points at a five-minute interval fills a thousand-sample buffer in twenty-five minutes; an hourly poll -- a completely ordinary setting -- loses more than half of every hour and reports no error. The maximum poll interval is therefore set by the buffer depth and the point count on EACH controller, which is not a preference and not the same number for every controller on a site. CHANGE-OF-VALUE IS THE RIGHT TOOL FOR THE RIGHT POINT AND NOT A GENERAL ANSWER. A binary status that changes a few times a day compresses by orders of magnitude; a noisy analogue input crossing a small deadband can generate MORE records than periodic sampling would, so change-of-value applied without a sensible deadband per point can increase the volume it was adopted to reduce. The comparison against the periodic rate is computed here rather than assumed. This is a sizing estimate. Bytes per sample is entered and varies widely with the historian -- a compressed columnar time-series store can be a small fraction of the naive figure while a relational table with indexes can be several times it -- so the volume is an order-of-magnitude planning number rather than a disk requirement. It does not address database indexing, retention policies, roll-up or downsampling of older data, backup volume, or the query performance that actually determines whether a historian is usable. Controller buffer depths, whether the buffer wraps or stops, and whether a buffer-full condition is signalled at all are product-specific. It does not address field bus bandwidth, the effect of trend polling on controller processor load, or time synchronisation across controllers, which is what makes trends from different devices comparable at all. The controller and historian manufacturers' documentation and the controls engineer govern.
samples per point per year = 60 / interval x 24 x 365; storage = points x samples x years x bytes per sample; the controller's own rate is its points x 60 / interval, and its buffer fills in depth / that rate -- which is the maximum archive poll interval.
Trend volume and buffer fill relations. Bytes per sample, buffer depth and the poll interval are ENTERED from the controller and historian documentation.
Two rate calculations.
Estimate. AHJ and licensed professional govern.
Field names used by the API: point_count, interval_min, retention_years, bytes_per_sample, controller_points, controller_buffer_samples, poll_interval_min, cov_changes_per_point_day, alt_interval_min, samples_per_point_year, total_samples, storage_gb, alt_storage_gb, controller_samples_per_hour, buffer_fill_min, poll_loss_pct, cov_total_samples, cov_storage_gb, cov_reduction_pct
- Bytes per sample is entered a compressed columnar store is a fraction of the naive figure; an indexed relational table several times itthe historian's documentation
- The buffer wraps some controllers stop or signal instead; it is product-specificthe controller's documentation
- Uniform interval across points a real site mixes intervals by point purposethe trend design