Time Series Forecasting — AI Tutorials

Time Series Fundamentals

A time series is a sequence of observations indexed in time order: $y_1, y_2, \ldots, y_T$ .

Decomposition

Any time series can be decomposed into:

Component	Description	Example
Trend	Long-term direction	Yearly revenue growth
Seasonality	Periodic, calendar-driven pattern	Holiday sales spike
Cyclicality	Multi-year irregular cycles	Business cycles
Residual / Noise	Random unexplained variation	Day-to-day fluctuations

Additive model: $y_t = T_t + S_t + R_t$ (magnitudes don't scale with level)

Multiplicative model: $y_t = T_t \times S_t \times R_t$ (variance grows with level — log-transform to make additive)

Stationarity

A stationary series has constant mean, variance, and autocovariance over time. Most classical models require stationarity — achieve it with:

Differencing: $y'_t = y_t - y_{t-1}$ (removes trend)
Seasonal differencing: $y'_t = y_t - y_{t-m}$ (removes seasonality)
Log transform: stabilizes exponentially growing variance

Augmented Dickey-Fuller (ADF) test: null hypothesis = unit root (non-stationary). Low p-value → reject → stationary.

ARIMA / SARIMA

ARIMA(p, d, q) combines three components:

AR(p) — Autoregressive: $y_t$ depends on its own $p$ past values
I(d) — Integrated: apply $d$ rounds of differencing for stationarity
MA(q) — Moving Average: $y_t$ depends on $q$ past forecast errors

$y_t = c + \phi_1 y_{t-1} + \cdots + \phi_p y_{t-p} + \theta_1 \varepsilon_{t-1} + \cdots + \theta_q \varepsilon_{t-q} + \varepsilon_t$

Selecting p, d, q

Tool	What it reveals
ADF test	Whether to difference (d)
ACF plot	MA order q (sharp cut-off after lag q)
PACF plot	AR order p (sharp cut-off after lag p)
AIC / BIC	Compare candidate models (lower = better)

SARIMA(p,d,q)(P,D,Q)_m

Extends ARIMA with seasonal terms at period $m$ (e.g., $m=12$ monthly, $m=7$ daily). (P, D, Q) are the seasonal AR, differencing, and MA orders.

Auto-ARIMA

pmdarima.auto_arima() or statsforecast.AutoARIMA grid-search over (p,d,q) and select by AIC — avoids manual order selection.

ARIMA with statsmodels + Auto-ARIMApython

import pandas as pd
import numpy as np
from statsmodels.tsa.statespace.sarimax import SARIMAX
from statsmodels.tsa.stattools import adfuller
import pmdarima as pm

# Synthetic monthly airline-style data
np.random.seed(42)
dates = pd.date_range('2018-01', periods=96, freq='MS')
trend = np.linspace(100, 300, 96)
seasonal = 40 * np.sin(2 * np.pi * np.arange(96) / 12)
noise = np.random.normal(0, 10, 96)
y = pd.Series(trend + seasonal + noise, index=dates, name='passengers')

# --- Stationarity check ---
result = adfuller(y)
print(f"ADF p-value: {result[1]:.4f}")  # likely > 0.05 → non-stationary

# --- Auto-ARIMA (finds best p,d,q,P,D,Q automatically) ---
model = pm.auto_arima(
    y,
    seasonal=True,
    m=12,                   # monthly seasonality
    stepwise=True,
    information_criterion='aic',
    trace=True,
)
print(model.summary())

# --- Forecast next 12 months ---
forecast, conf_int = model.predict(n_periods=12, return_conf_int=True)
future_dates = pd.date_range(dates[-1], periods=13, freq='MS')[1:]
forecast_series = pd.Series(forecast, index=future_dates)

print("\nNext 12-month forecast:")
for date, val, (lo, hi) in zip(future_dates, forecast, conf_int):
    print(f"  {date.strftime('%Y-%m')}: {val:.1f}  [{lo:.1f}, {hi:.1f}]")

# --- Manual SARIMAX for reference ---
manual_model = SARIMAX(y, order=(1,1,1), seasonal_order=(1,1,1,12))
fit = manual_model.fit(disp=False)
print(f"\nManual SARIMA(1,1,1)(1,1,1)12 AIC: {fit.aic:.2f}")

Facebook Prophet

Prophet is a decomposable forecasting model designed for business time series:

$y(t) = g(t) + s(t) + h(t) + \varepsilon_t$

Term	Meaning
$g(t)$	Trend: piecewise linear or logistic growth with automatic changepoints
$s(t)$	Seasonality: Fourier series for yearly, weekly, daily patterns
$h(t)$	Holiday effects: user-provided date list
$\varepsilon_t$	Gaussian noise

Why Prophet Works Well in Practice

Automatic changepoint detection — detects trend shifts without manual intervention
Multiple seasonalities — yearly + weekly + daily simultaneously
Robust to missing data and outliers — no stationarity required
Human-friendly tuning — parameters correspond to business intuitions
Uncertainty intervals — via Laplace approximation or MCMC

Limitations

Assumes trend + seasonality decomposition — struggles with complex cross-variable dynamics
Not designed for high-frequency (minute/second) data
Weaker than TFT for multivariate / covariate-rich scenarios

Prophet: Multi-seasonality Forecasting with Holidayspython

from prophet import Prophet
import pandas as pd
import numpy as np

# Prophet requires columns 'ds' (date) and 'y' (value)
np.random.seed(0)
dates = pd.date_range('2020-01-01', periods=730, freq='D')
y = (
    50
    + 0.05 * np.arange(730)                            # trend
    + 10 * np.sin(2 * np.pi * np.arange(730) / 365)   # yearly seasonality
    + 5  * np.sin(2 * np.pi * np.arange(730) / 7)     # weekly seasonality
    + np.random.normal(0, 3, 730)                      # noise
)
df = pd.DataFrame({'ds': dates, 'y': y})

# Define holidays (US public holidays as an example)
holidays = pd.DataFrame({
    'holiday': 'us_holiday',
    'ds': pd.to_datetime(['2020-12-25', '2021-12-25', '2021-07-04', '2022-07-04']),
    'lower_window': -1,
    'upper_window': 1,
})

# Build and fit model
model = Prophet(
    changepoint_prior_scale=0.05,    # controls trend flexibility (0.001–0.5)
    seasonality_prior_scale=10.0,    # controls seasonality strength
    holidays=holidays,
    yearly_seasonality=True,
    weekly_seasonality=True,
    daily_seasonality=False,
    uncertainty_samples=500,
)
model.fit(df)

# Forecast 90 days ahead
future = model.make_future_dataframe(periods=90)
forecast = model.predict(future)

tail = forecast[['ds', 'yhat', 'yhat_lower', 'yhat_upper']].tail(5)
print(tail.to_string(index=False))

# Inspect components
# model.plot_components(forecast)  # shows trend, holidays, and each seasonality

Temporal Fusion Transformer (TFT)

The TFT (Lim et al., 2021) is a deep learning architecture purpose-built for multi-horizon time series forecasting with covariates.

Key Architecture Components

Component	Role
Variable Selection Networks	Learns which input features matter most at each time step
Gated Residual Networks (GRN)	Adaptive nonlinear processing with skip connections
LSTM Encoder-Decoder	Captures local temporal patterns
Multi-head Attention	Identifies long-range seasonal dependencies
Quantile outputs	Produces prediction intervals (P10, P50, P90)

Why TFT Outperforms Classical Methods

Handles static covariates (product category, region), known future inputs (promotions, calendar), and past observables jointly
Interpretable attention weights show which past time steps the model attends to
Quantile regression gives calibrated uncertainty without simulation
Beats ARIMA/Prophet on M5 competition and industrial demand forecasting benchmarks

When to Use Each Model

Model	Best For
ARIMA/SARIMA	Single univariate series, small data, interpretability required
Prophet	Business metrics, multiple seasonalities, stakeholder-friendly
LSTM/RNN	Multivariate, sequential dependencies, moderate data
TFT	Large-scale, rich covariates, multi-horizon, production forecasting

TFT with PyTorch Forecastingpython

import pandas as pd
import numpy as np
from pytorch_forecasting import TemporalFusionTransformer, TimeSeriesDataSet
from pytorch_forecasting.metrics import QuantileLoss
import lightning.pytorch as pl

# --- Synthetic retail dataset ---
np.random.seed(42)
n_groups, n_steps = 5, 200
records = []
for g in range(n_groups):
    for t in range(n_steps):
        records.append({
            'group_id': str(g),
            'time_idx': t,
            'value': 50 + 5*g + 0.1*t + 10*np.sin(2*np.pi*t/52) + np.random.randn()*3,
            'price': np.random.uniform(8, 12),          # known future covariate
            'is_promo': int(t % 13 == 0),               # known future covariate
        })
data = pd.DataFrame(records)

max_encoder_length = 52   # look-back window (1 year)
max_prediction_length = 8  # forecast horizon (2 months)
training_cutoff = n_steps - max_prediction_length

# --- Dataset ---
training = TimeSeriesDataSet(
    data[data.time_idx <= training_cutoff],
    time_idx='time_idx',
    target='value',
    group_ids=['group_id'],
    max_encoder_length=max_encoder_length,
    max_prediction_length=max_prediction_length,
    static_categoricals=['group_id'],
    time_varying_known_reals=['time_idx', 'price', 'is_promo'],
    time_varying_unknown_reals=['value'],
    target_normalizer=None,
)
validation = TimeSeriesDataSet.from_dataset(
    training, data, predict=True, stop_randomization=True
)
train_loader = training.to_dataloader(train=True, batch_size=64)
val_loader = validation.to_dataloader(train=False, batch_size=64)

# --- Model ---
tft = TemporalFusionTransformer.from_dataset(
    training,
    learning_rate=3e-3,
    hidden_size=32,
    attention_head_size=2,
    dropout=0.1,
    hidden_continuous_size=16,
    loss=QuantileLoss(),
    log_interval=10,
)
print(f"Parameters: {tft.size() / 1e3:.1f}k")

trainer = pl.Trainer(max_epochs=30, enable_progress_bar=True)
trainer.fit(tft, train_dataloaders=train_loader, val_dataloaders=val_loader)

# --- Predict + interpret ---
predictions = tft.predict(val_loader, return_y=True, trainer_kwargs={"accelerator": "cpu"})
print(f"MAE: {(predictions.output.prediction[:,:,3] - predictions.y[0]).abs().mean():.3f}")

Evaluation Metrics for Forecasting

Metric	Formula	Notes
MAE	$\frac{1}{T}\sum	y_t - \hat{y}_t
RMSE	$\sqrt{\frac{1}{T}\sum(y_t-\hat{y}_t)^2}$	Penalizes large errors more
MAPE	$\frac{100}{T}\sum\left	\frac{y_t - \hat{y}_t}{y_t}\right
sMAPE	$\frac{200}{T}\sum\frac{	y_t-\hat{y}_t
MASE	$\frac{MAE}{MAE_{naive}}$	Scale-free; < 1 means beating naive
Coverage	% of actuals inside PI	Evaluates interval calibration

Train/Test Split for Time Series

Never use random splits — this causes data leakage.

|-------- training --------|-- val --|-- test --|
         chronological order

Use expanding window or sliding window cross-validation for robust estimates.

Knowledge check

A retail analyst has monthly sales data with a strong Christmas spike and an overall upward trend. The dataset spans 5 years (60 observations) with no external covariates. Which model is most appropriate?

Summary

Time series have trend, seasonality, cyclicality, and noise — decompose before modeling
Stationarity is required for ARIMA — test with ADF, fix with differencing or log-transform
ARIMA/SARIMA: classical, interpretable, great for single univariate series
Prophet: robust, business-friendly, handles multiple seasonalities and holidays out of the box
TFT: state-of-the-art deep learning for multivariate, covariate-rich, multi-horizon forecasting
Always evaluate with MASE or sMAPE and use chronological splits — never random

Next: Graph Neural Networks — extending deep learning to graph-structured data.

← ML Fundamentals