Time-Series Forecasting

Time-series forecasting estimates future quantities using information available at a defined forecast origin. Start by specifying when the prediction is issued, which horizon matters, and what decision it supports. A model can fit historical data beautifully while failing that operational task.

Use temporal backtests and simple baselines before adding complexity. Keep point accuracy and uncertainty calibration separate, and test the same information constraints the future system will face. A feature that arrives after the forecast is issued is not available merely because it exists in today's historical table.

TL;DR

Quick Example

This fictional series repeats the same four-value season exactly. The one-step rolling-origin check compares the most recent value with the previous matching seasonal position.

The seasonal baseline has zero error because repetition is built into the data. That is a sanity check for the code, not evidence that seasonal forecasting is perfect on real observations. An additional check altered future values and verified that the earlier baseline calculation was unchanged.

Core Concepts

Forecasting: Principles and Practice describes rolling-origin evaluation in which training observations precede the test observation. It also distinguishes one-step and multi-step evaluation, seasonal naive baselines, and prediction intervals. Match those concepts to the actual decision horizon. Forecasting toolbox.

Separate fitted residuals from future forecast errors. A model's fit to data used during estimation is not an independent test of future behavior. Temporal evaluation must preserve the information boundary at each origin.

Define the target's calendar and revision policy. Daily totals, business days, weeks, and irregular events need explicit alignment. A later-corrected historical value may not be the value the production forecaster knew at the time.

Design the backtest around the real decision

This editorial backtest matrix is a planning aid. It does not validate a particular model or forecast service.

Choose evaluation origins that span representative seasons, operating conditions, and changes. A short convenient period can reward a model that fails during the most important event. Preserve counts and conditions rather than reporting one unqualified average.

Prevent temporal leakage beyond the row split

Fit preprocessing inside each training window. Imputation, scaling, feature selection, seasonal decomposition, and tuning can leak future information if computed across the complete series first. A correct train/test index does not repair a globally fitted transformation.

For external regressors, establish what was known at the origin. Future weather, finalized economic values, or later-updated inventory may need their own forecasts or historical versions. A backtest with perfect future regressors answers a different question from the production system.

Review target construction and reporting delay. If yesterday's total is finalized only after the forecast deadline, it may not be a valid lag at that origin. Represent the actual availability time rather than assuming the event date is the arrival time.

Keep model selection separate from final evaluation. Repeatedly choosing models from one holdout can make it part of the tuning process. Use a design that preserves an appropriate later assessment and disclose the selection history.

Use baselines to expose the value of complexity

Compare last-value, seasonal naive, or other justified simple forecasts under the same origins, horizons, and metrics. A baseline should represent a plausible operational alternative, not an intentionally weak opponent.

Inspect when the candidate improves or worsens. A small average gain can conceal poor behavior for sparse series, regime changes, or a critical horizon. Report the slices that matter to the use and preserve uncertainty around small differences.

Account for operating cost: data dependencies, refresh time, failure handling, interpretability, and maintenance. A model that marginally improves a chosen metric can still be a poor fit if it cannot issue forecasts before the decision deadline.

In the toy example, exact seasonality makes the seasonal baseline an obvious winner. If the season changes or the series trends, that conclusion can reverse. Test those mechanisms rather than promoting the example into a universal rule.

Evaluate forecast uncertainty as an outcome

A prediction interval describes uncertainty in future observations under the model and assumptions. It is not a guarantee that each particular future value will lie inside it. Check observed coverage and width across relevant horizons and cohorts.

Distinguish nominal coverage from measured coverage. A label such as “90 percent interval” states the intended probability or procedure level; evaluation asks whether the method behaves appropriately under the actual forecasting task. A very wide interval can cover often while being operationally unhelpful.

Account for model misspecification, changing regimes, and input uncertainty where they matter. A narrow interval from a stable fitted model can underestimate uncertainty after the operating environment changes. Define monitoring and fallback rather than assuming the original backtest remains permanent evidence.

Do not sum arbitrary marginal interval endpoints to create a joint forecast interval without a justified dependence model. Aggregated uncertainty depends on relationships among future quantities. Preserve that question for the appropriate method and reviewer.

Best Practices

Save the forecast origin, model/input versions, target definition, and predicted horizon with each evaluation record. That makes it possible to reproduce what information was used and compare a later actual value under the correct revision policy.

Test missing data, delayed inputs, sparse series, and failed model updates. A forecasting service needs an understandable outcome when the preferred model cannot run. Define which baseline or explicit unavailable state is appropriate.

Use realistic chronological evaluation before claiming quality. The synthetic example here validates indexing and a baseline calculation only; it does not fit a production model or establish interval coverage.

Common Mistakes

Bad: Randomly shuffle dependent observations and call the result a temporal backtest.

Correct: Preserve the forecasting information boundary and evaluate the actual horizon and availability rules.

Bad: Compare a trained model with a baseline on different dates.

Correct: Use aligned origins, targets, horizons, and metrics so the comparison answers one question.

Bad: Report a nominal interval level as verified coverage.

Correct: Evaluate coverage and usefulness under representative future-like conditions and retain the model assumptions.

FAQ

Is a seasonal baseline always necessary?

Use it when the task has a meaningful seasonal structure and enough prior observations. Other tasks need different baselines; the goal is a credible simple comparator.

Does zero error in the example establish forecast skill?

No. Exact repetition was deliberately constructed. It tests the calculation rather than a claim about real uncertainty or predictive value.

What would change the model choice?

A changed horizon, input delay, regime, error consequence, or maintenance constraint can reverse the choice. Reevaluate the task and evidence rather than retaining the winner of an outdated backtest.

Related Topics

References