Trend and Seasonality
Most series are a level, a trend, one or more seasonal cycles and calendar effects on top of noise. Model each explicitly or difference it away — and know that a model fitted to one regime is a bet that the trend continues.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
What are the components of a series, how does a model account for each, and what happens to a fitted model when the trend breaks?
A SaaS finance team forecasts monthly recurring revenue for the board. The model trained on five years of growth. Three months ago a pricing change and a large customer's exit flattened growth, and the model is still forecasting the old curve upward, which the board has now planned against.
Fit a model to the whole history. Five years is a lot of signal; the model will learn the growth and the seasonality and extrapolate them.
The model learned a trend. A trend is a bet that the future continues the past; the pricing change and the customer exit ended the regime the trend was fitted in, and the model has no way to know (Concept Drift).
- The model learned a trend. A trend is a bet that the future continues the past; the pricing change and the customer exit ended the regime the trend was fitted in, and the model has no way to know (Concept Drift).
- The seasonality was learned as "December is low". This year December contains a fiscal-year-end push that moved from January; the calendar effect is attached to an event, not a month, and the month-indexed seasonality is wrong.
- Sixty rows and a flexible model: the model fit the noise of five years and reports a confident twelve-month curve with an interval far narrower than the last three months' surprise.
- Nobody plotted the residuals after the break. Three consecutive months of the model over-forecasting is a structural signal, and it was read as three unlucky months.
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- Predict monthly recurring revenue twelve months ahead. The label is the recognised MRR from the billing system at month end.
- The decision is hiring and spend commitments, so an over-forecast costs real money and an under-forecast costs growth; the two are not symmetric.
- One example is one month: MRR, new bookings, churned MRR, expansion, contraction, plus calendar features and a marker for pricing changes. Sixty rows.
- A weekly signups series at higher resolution, with a strong day-of-week pattern, an annual pattern around the fiscal year, and holiday dips that move with the calendar.
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- Decomposition treats a series as a level (where it is now), a trend (the direction it is moving), one or more seasonal components (repeating cycles — weekly, annual — with fixed period), calendar effects (holidays, month length, paydays, events that move with a calendar rather than a period), and a remainder. Classical models estimate these explicitly and update them as new data arrives; a tree or network model has to be given them as features or learn them from lags.
- Differencing replaces
y[t]withy[t] - y[t-1](ory[t] - y[t-m]for seasonal differencing). It removes a trend or a seasonal cycle so that what remains is stationary — its statistical properties do not depend on time — which is what most models assume. The forecast is made on the differenced series and integrated back. - A structural break is a point where the components change: a new level, a new trend slope, a different seasonal shape. A model fitted across a break averages two regimes and forecasts neither; a model fitted before the break extrapolates the old one. Both look fine offline if the evaluation window is inside a single regime, and both fail when the regime changes (Performance Decay).
What a series is made of
Plot MRR and you see a curve going up. Decompose it and you see four things: a level that steps when a large customer arrives or leaves, a trend that was steep for four years and flattened in month fifty-seven, an annual cycle with a fiscal-year-end bump, and a remainder that is small until the break and large after it.
Each component is a different modelling problem. Level and trend are what exponential smoothing tracks and what differencing removes. Seasonality is a fixed-period cycle a Fourier term or a seasonal lag can carry. Calendar effects are features of the date. The remainder is what is left, and its size after the break is the honest interval.
Differencing, and what it assumes
Subtract yesterday from today and the trend is gone; subtract last week's same day and the weekly cycle is gone too. What remains is closer to stationary, and a model that assumes stationarity — most classical ones — can be fitted to it. The forecast is made on differences and summed back onto the last observed level.
The assumption is that the differences are stationary. Across a structural break they are not: the mean difference changes sign. Differencing does not remove a break; it turns a step in the level into a spike in the differences, and a change in slope into a step. The model still has to be told.
1import numpy as np2 3def difference(y, lag=1):4 return y[lag:] - y[:-lag] # removes trend (lag=1) or season (lag=m)5 6def integrate(last_level, d_forecast):7 return last_level + np.cumsum(d_forecast) # sum differences back onto the level8 9y = mrr.values # 60 months10d1 = difference(y, 1) # month-on-month change11# a change in trend slope is a *step* in d1 — plot it and the break is obvious:12# months 1..56 have a mean change near +40k; months 57..60 near 013before, after = d1[:56].mean(), d1[56:].mean()14 15# fitting the difference model on all 59 changes forecasts (before*56 + after*3)/5916# per month — a slope that has never been true in either regime.17d_fc = np.full(12, after) # the current regime, honestly18fc = integrate(y[-1], d_fc)The step in the differenced series is a diagnostic, not a fix. The decision — fit the current regime, weight it, or model the break explicitly — is yours; the differenced plot only makes the decision unavoidable.
A trend is a bet
The fitted slope encodes four years of a pricing structure, a market and a sales team. Extrapolating it twelve months is a claim that all of those continue. When the pricing changes, the claim is false and the model does not know, because nothing in its inputs changed — the series is the only input, and the series has only just begun to disagree.
This is concept drift in its purest form: the relationship between past and future changed while the inputs looked normal. The monitor for it is not a feature distribution; it is the residual, and specifically its sign over consecutive periods.
The conditions that produced the fitted trend — pricing, market, sales capacity, product — persist over the forecast horizon, so extrapolating the trend is a forecast rather than a description of the past.
holds when No known change to those conditions is scheduled; the residuals of the last several periods are small and of mixed sign; the seasonal shape re-estimated this year matches last year's.
breaks when A pricing change, a large customer event, a market shift or a sales reorganisation happens; the residuals run same-signed for several periods; a known event is scheduled inside the horizon.
respond Refit on the current regime with a widened interval, add the known change as a covariate or a level shift, and re-present the forecast as a new bet with its assumption stated. Do not retrain on all history and call it fixed.
How to build it
Most important first.
- Plot the decomposition before modelling. Level, trend, seasonal and remainder on separate axes tells you which components exist, whether the seasonal shape is stable, and where the breaks are. It is the cheapest and most informative step and it is usually skipped.
- Attach calendar effects to events, not to periods. A holiday is a feature of the date (
days_to_holiday,is_fiscal_year_end), not a property of "December". Known future events — a planned price change, a product launch — are covariates. - Weight recent data more, or fit on the current regime only, when a break is known. A model that knows the break happened in month fifty-seven is better than one that pretends sixty months are exchangeable (Retraining Strategies).
- Forecast with an interval, and widen it at the horizon and after a break. A point forecast twelve months out from sixty rows is a number the board will plan against, and its honest interval is wide.
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- Residuals over time, plotted. A run of same-signed residuals is the signature of a break or a missed component, and it is visible months before any summary metric moves.
- Rolling-origin error on the most recent regime, separately from error over the full history. A model that is good on average and bad since the break is bad.
- Do not measure fit on the full history as evidence of forecast quality. A model that fits five years including the break has fitted the break; it has not learned to forecast past one.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- The trend the model extrapolates is driven by conditions that will continue over the horizon — pricing, market, product — and a known change to any of them invalidates the extrapolation until the model is refitted or told.
- The seasonal shape is stable from year to year at the period the model uses, and calendar events fall where the features say they do this year.
- The most recent observations are complete and not revised; a billing correction that restates last month's MRR moves the level the forecast starts from.
- Offline: rolling-origin evaluation with the origin advancing across the known break. Error before and after should be reported separately; a model that only wins before the break has not been evaluated on the problem.
- Online: monitor the residual sign over consecutive months; three same-signed residuals of growing size triggers a review of the regime, not an automatic retrain.
- Over time: re-estimate the decomposition each quarter and compare the seasonal shape and trend slope to the previous estimate. A changed shape is a changed business, and the model's assumption should be re-stated.
What can go wrong
- Seasonal differencing at period twelve is applied to a series with a four-week payroll cycle; the seasonal period was assumed from the calendar rather than measured from the data, and the residual keeps a cycle.
- A holiday that moves — a lunar-calendar festival, a floating public holiday — is encoded as a fixed week and the calendar effect lands in the wrong week each year.
- The team retrains on the current regime only, which is three months; the new model has no seasonality at all and forecasts a flat line into the fiscal-year-end push.
- Explicit decomposition and calendar features are more work than handing the raw series to a flexible model, and they make the model's assumptions visible and arguable, which is the point.
- Fitting on the current regime uses less data and produces a worse-looking in-sample fit; it is also the only model that describes the business as it now is.
- Honest intervals are wide and the board does not like them; a narrow interval from an over-fitted model is what they planned against last time.
- "Five years of data is more signal than three months." Five years of a different business is not signal about this one. The question is which rows describe the regime you are forecasting into.
- "The model has a seasonal component, so it handles December." It handles the average December of the training years. It does not handle a fiscal-year-end push that moved, or a holiday that fell on a different week.
- "Drift means retrain." A structural break is not drift in the inputs — it is a change in the process. Retraining on all history rebuilds the average of two regimes; the response is to decide which regime to fit, and to tell the model about the break.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThat a trend is a bet on the continuation of its causes, and that a model fitted across a structural break describes neither regime, holds for every series and model family.
- SIMPLIFIEDDecomposition is described as additive level + trend + seasonal + remainder; many series are multiplicative — the seasonal swing grows with the level — and are decomposed on the log scale, and the choice between the two is itself a modelling decision this lesson leaves out.
- DOMAIN-SPECIFICRevenue and demand series have calendar effects tied to fiscal years, paydays and holidays that move; sensor and infrastructure series have seasonality tied to daily and weekly load and almost no calendar effects, so the feature set that matters is different.
Where the depth lives
This domain teaches the model and hands the rest off by name.