CPI Transformer Risk Dashboard

Overview

This project forecasts year-over-year CPI inflation from a panel of macroeconomic indicators using a Transformer encoder, and benchmarks it against a classical ARIMA model. A gradient-based SHAP layer attributes each forecast to its macro drivers, and everything is surfaced through an interactive Streamlit dashboard.

The pipeline runs as five composable stages:

1. Data — load or synthesize a monthly macro panel.
2. Features — compute YoY targets and window the series into supervised sequences.
3. Model — train a Transformer encoder with early stopping.
4. Baseline — fit a rolling SARIMAX and compare RMSE.
5. Explain — compute SHAP driver rankings and visualize.

1Data

The model consumes a monthly, date-indexed table (data/raw/macro_cpi.csv) with a headline CPI level plus six macro indicators:

• UNRATE — unemployment rate (labor slack).
• FEDFUNDS — policy rate (monetary stance).
• OIL_PRICE — energy costs (supply-side pressure).
• M2 — broad money supply (demand-side pressure).
• INDPRO — industrial production (real activity).
• HOUSING — housing index (shelter dynamics).

The loader validates the path and reads the CSV into a pandas frame; the pipeline then parses date, sets it as a sorted index, and proceeds to feature engineering.

def load_csv(filename: str) -> pd.DataFrame:
    path = os.path.join(RAW_DIR, filename)
    if not os.path.exists(path):
        raise FileNotFoundError(f"Raw data file not found: {path}")
    return pd.read_csv(path)

2Synthetic Generator

To develop and stress-test the pipeline without a data licence, a synthetic generator builds 650 months (~54 years) of macro series with smooth cycles plus noise, then constructs CPI YoY as a function of lagged macro features — so the relationships are real, learnable, and non-trivial.

Each driver is a normalized transform of lagged inputs, for example an oil-momentum term and a labor-gap term:

$$ \text{oil\_mom}_t = \frac{P^{oil}_{t-1} - P^{oil}_{t-6}}{45}, \qquad \text{labor\_gap}_t = \frac{u_{t-1} - 5.5}{12}. $$

These combine into a base signal (with saturating $\tanh$ nonlinearities so extreme moves don't blow up), plus financial-stress and global-supply terms, lagged echoes of the macro signal, and Gaussian noise:

base_signal = (
    0.012 * np.tanh(3 * oil_momentum)
    - 0.015 * labor_gap
    + 0.01  * yield_curve
    + 0.013 * credit_growth
    + 0.012 * production_cycle
    + 0.008 * np.tanh(4 * housing_cycle)
    + 0.004 * np.sin(t / 24)
    + 0.0035 * np.cos(t / 10)
)
macro_signal = base_signal + 0.0065 * np.tanh(3.5 * financial_stress) + 0.0075 * global_supply
yoy = (0.02 + macro_signal) + 0.35 * lag(macro_signal, 3) - 0.18 * lag(macro_signal, 6)
yoy += np.random.normal(scale=0.0035, size=n)
yoy = np.clip(yoy, -0.01, 0.075)

Finally the CPI level is reconstructed from YoY so the feature-engineering step can re-derive the target exactly as it would from real data: $\text{CPI}_t = \text{CPI}_{t-12}\,(1 + \text{yoy}_t)$.

3Feature Engineering

The target is the 12-month percentage change of CPI (YoY), computed with pandas and expressed in percentage points:

$$ \text{CPI\_yoy}_t = \left(\frac{\text{CPI}_t}{\text{CPI}_{t-12}} - 1\right)\times 100. $$

def add_yoy(df, col, new_col):
    out = df.copy()
    out[new_col] = out[col].pct_change(12) * 100.0
    return out

Features are every column except the raw CPI level. A small helper de-duplicates the requested columns and drops rows with missing values (the first 12 months, which have no YoY), aligning features and target on a clean common index.

4Sequence Windows

Forecasting is framed as sequence-to-one: given a lookback window of $L = 24$ months of features, predict the next $H = 1$ month of CPI YoY. A sliding window produces tensors $X \in \mathbb{R}^{N\times L\times F}$ and targets $y \in \mathbb{R}^{N\times H}$, plus the target timestamps for plotting.

for t in range(lookback, len(df) - horizon + 1):
    X_list.append(values[t - lookback:t, :])   # (L, F) window
    y_list.append(target[t:t + horizon])       # (H,)  future target
    ts_list.append(df.index[t + horizon - 1])  # timestamp of the prediction

Because windows only ever look backward, there is no temporal leakage from the future into any training sample.

5Split & Scaling

The sequences are split chronologically — 70% train, 15% validation, 15% test — so the model is always evaluated on the most recent, unseen period.

Two independent StandardScalers are fit on the training set only and reused everywhere else:

• Feature scaler — fit on the flattened $(N\cdot L,\,F)$ matrix, then reshaped back to windows, so every feature is zero-mean/unit-variance.
• Target scaler — standardizes CPI YoY so the MSE loss is well-conditioned; predictions are inverse-transformed back to percentage points before RMSE is computed.

Both scalers are persisted with joblib so inference and SHAP use exactly the same transforms.

6Architecture

The model is a compact Transformer encoder, TimeSeriesTransformer. Its dimensions in the pipeline are $d_{model}=96$, 4 attention heads, 3 encoder layers, feed-forward width 192, and dropout 0.1.

class TimeSeriesTransformer(nn.Module):
    def __init__(self, n_features, d_model=64, nhead=4, num_layers=2,
                 dim_feedforward=128, dropout=0.1, horizon=1):
        super().__init__()
        self.input_proj = nn.Linear(n_features, d_model)
        self.pos_encoder = PositionalEncoding(d_model, dropout)
        encoder_layer = nn.TransformerEncoderLayer(
            d_model=d_model, nhead=nhead, dim_feedforward=dim_feedforward,
            dropout=dropout, batch_first=True)
        self.transformer_encoder = nn.TransformerEncoder(encoder_layer, num_layers)
        self.head = nn.Sequential(
            nn.LayerNorm(2 * d_model), nn.GELU(), nn.Linear(2 * d_model, horizon))

The forward pass has four stages:

1. Input projection — a linear map lifts the $F$ raw features into the $d_{model}$ embedding space.
2. Positional encoding — adds order information (§6.1).
3. Encoder stack — multi-head self-attention mixes information across the 24 months (§6.2).
4. Pooling head — combines a summary and the final step, then projects to the forecast (§6.3).

6.1Positional Encoding

Self-attention is permutation-invariant, so fixed sinusoidal positional encodings are added to inject the notion of time order:

$$ PE_{(pos,2i)} = \sin\!\left(\frac{pos}{10000^{2i/d}}\right),\quad PE_{(pos,2i+1)} = \cos\!\left(\frac{pos}{10000^{2i/d}}\right). $$

pe[:, 0::2] = torch.sin(position * div_term)
pe[:, 1::2] = torch.cos(position * div_term)
x = x + pe[:, :x.size(1), :]   # broadcast-add to the sequence

6.2Self-Attention

Each encoder layer computes scaled dot-product attention over the sequence, letting month $i$ weigh information from every other month $j$:

$$ \mathrm{Attn}(Q,K,V) = \operatorname{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V. $$

Multiple heads learn complementary temporal patterns (e.g. a 12-month seasonal echo vs. a short oil-shock response), and a position-wise feed-forward network adds nonlinear capacity.

6.3Pooling Head

Rather than a single CLS token, the head concatenates a mean-pooled representation (a smooth summary of the whole window) with the last time-step representation (the most recent state), giving a $2\,d_{model}$ vector that is normalized, passed through GELU, and projected to the forecast:

x = self.transformer_encoder(self.pos_encoder(self.input_proj(src)))
last_repr   = x[:, -1, :]         # most recent month
pooled_repr = x.mean(dim=1)       # whole-window summary
combined    = torch.cat([pooled_repr, last_repr], dim=-1)
return self.head(combined)        # (batch, horizon)

7Training

Training minimizes mean-squared error on the scaled target with Adam ($lr = 3\times10^{-4}$), a ReduceLROnPlateau scheduler, gradient clipping, and early stopping on validation loss (up to 120 epochs, patience 15).

$$ \mathcal{L} = \frac{1}{N}\sum_{i=1}^{N}\big(\hat{y}_i - y_i\big)^2. $$

optimizer = torch.optim.Adam(model.parameters(), lr=lr)
scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, factor=0.5, patience=3)
criterion = nn.MSELoss()

for ep in range(epochs):
    model.train()
    for Xb, yb in train_loader:
        optimizer.zero_grad()
        loss = criterion(model(Xb), yb.view(yb.size(0), -1))
        loss.backward()
        torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
        optimizer.step()
    # ... validate, scheduler.step(val_loss), keep best_state, early-stop ...

Three ingredients stabilize a Transformer on a modest dataset:

• Gradient clipping ($\lVert g\rVert \le 1$) prevents exploding updates.
• LR scheduling halves the rate when validation loss stalls.
• Early stopping restores the best-validation weights, guarding against overfitting.

Evaluation inverse-transforms predictions to percentage points and reports RMSE: $\text{RMSE}=\sqrt{\tfrac1N\sum_i(y_i-\hat y_i)^2}$.

8ARIMA Baseline

The baseline is a rolling SARIMAX(1,1,1) fit only on the CPI YoY series — a strong, interpretable univariate benchmark. An ARIMA($p,d,q$) model differences the series $d$ times and regresses on $p$ lags plus $q$ past errors:

$$ \Delta y_t = c + \sum_{i=1}^{p}\phi_i\,\Delta y_{t-i} + \sum_{j=1}^{q}\theta_j\,\varepsilon_{t-j} + \varepsilon_t. $$

history = train.copy()
for t in range(len(test) - horizon + 1):
    res = SARIMAX(history, order=(1,1,1),
                  enforce_stationarity=False, enforce_invertibility=False).fit(disp=False)
    preds.append(res.forecast(steps=horizon).values[-1])
    truth.append(test.iloc[t + horizon - 1])
    history = pd.concat([history, pd.Series([test.iloc[t]], index=[test.index[t]])])

The rolling refit means the baseline always uses all data up to the forecast date — an honest, non-cheating comparison. The headline improvement metric is $(\text{RMSE}_{ARIMA}-\text{RMSE}_{Transformer})/\text{RMSE}_{ARIMA}$.

9SHAP Attribution

Interpretability uses SHAP's GradientExplainer, which is efficient for differentiable models. A background set from training data anchors the expectation; SHAP values on a test subset are then aggregated by mean absolute value across samples and time to rank each feature's global influence:

explainer = shap.GradientExplainer(model, background)   # background: train subset
shap_values = explainer.shap_values(eval_data)          # (N, L, F)
agg = np.abs(sv_array).mean(axis=(0, 1))                # mean |SHAP| over samples & time
idx = np.argsort(agg)[::-1][:top_k]                     # top-k drivers

SHAP values are grounded in cooperative game theory: the Shapley value fairly distributes a prediction across features by averaging each feature's marginal contribution over all orderings,

$$ \phi_i = \sum_{S\subseteq F\setminus\{i\}} \frac{|S|!\,(|F|-|S|-1)!}{|F|!}\, \big[f(S\cup\{i\}) - f(S)\big]. $$

The result is written to artifacts/top_drivers.json for the dashboard.

10Dashboard

The Streamlit app reads pre-computed artifacts, so it loads instantly. It renders three things:

• Scorecard — Transformer RMSE, ARIMA RMSE, and the % improvement as metric cards.
• Timeline — forecast vs. actual CPI YoY on the held-out test set.
• Drivers — a bar chart of the top-5 SHAP macro features.

c1, c2, c3 = st.columns(3)
c1.metric('Transformer RMSE', f"{metrics['transformer_rmse']:.4f}")
c2.metric('ARIMA RMSE',       f"{metrics['arima_rmse']:.4f}")
c3.metric('RMSE Improvement', f"{metrics['rmse_improvement_pct']:.1f}%")

st.line_chart(df_tl.set_index('timestamp')[['y_true', 'y_pred']])
st.bar_chart(pd.DataFrame({'importance': drivers['importances']}, index=drivers['features']))

Each panel degrades gracefully: if an artifact is missing, the app shows an instruction instead of erroring.

11Results

The pipeline writes a machine-readable scorecard to artifacts/metrics.json containing both models' RMSE, the percentage improvement, and the best validation loss. The timeline.csv stores aligned (timestamp, y_true, y_pred) rows for the test window.

12Limitations

• Data scale. ~600 monthly points is small for a Transformer; deep attention models are data-hungry and can overfit.
• Strong baseline. Highly autoregressive targets favor ARIMA; the deep model's edge is easier to show on multivariate, regime-switching, or higher-frequency data.
• Synthetic dynamics. Conclusions depend on the generator; real CPI data is the true test.
• Regime shifts. Out-of-sample accuracy can degrade around structural breaks and policy changes.

AAPI Reference

# Model
class TimeSeriesTransformer(nn.Module):
    def __init__(self, n_features, d_model=96, nhead=4, num_layers=3,
                 dim_feedforward=192, dropout=0.1, horizon=1): ...
    def forward(self, src: torch.Tensor) -> torch.Tensor: ...

# Sequence windowing
def make_supervised_sequences(df, target_col, feature_cols, lookback, horizon)
    -> tuple[np.ndarray, np.ndarray, list]: ...

# Training / evaluation
def train_transformer(model, train_loader, val_loader, device='cpu',
                      epochs=50, lr=1e-3, patience=7) -> dict: ...
def evaluate_model(model, loader, device='cpu', target_scaler=None)
    -> tuple[float, np.ndarray, np.ndarray]: ...

# Baseline
def rolling_forecast_rmse(series, order=(1,1,1), seasonal_order=(0,0,0,0),
                          horizon=1, split_index=None)
    -> tuple[float, np.ndarray, np.ndarray]: ...

# Explainability
def shap_top_features(model, X_background, X_eval, feature_names, top_k=5)
    -> tuple[list[str], np.ndarray]: ...

BRepository Layout

src/
  data/loader.py            # CSV IO + directory helpers
  features/engineering.py   # YoY target, align + dropna
  features/sequence.py      # sliding-window supervised sequences
  models/transformer.py     # TimeSeriesTransformer + PositionalEncoding
  models/train_utils.py     # Dataset, split, scaling, train, evaluate
  models/arima_baseline.py  # rolling SARIMAX RMSE
  models/shap_utils.py      # GradientExplainer driver ranking
  utils/generate_synthetic.py  # feature-driven synthetic macro panel
  train_pipeline.py         # orchestrates the full run -> artifacts/
  explain_shap.py           # writes top_drivers.json
dashboard/app.py            # Streamlit UI
artifacts/                  # metrics.json, timeline.csv, top_drivers.json, weights
data/raw/ , data/processed/ # inputs, scalers, cached arrays

CGlossary

• CPI YoY — 12-month % change in the Consumer Price Index; the forecast target.
• Transformer encoder — attention-based sequence model that weighs all timesteps in parallel.
• Self-attention — mechanism computing pairwise interactions between timesteps.
• Positional encoding — signal added to inputs to represent order.
• ARIMA / SARIMAX — classical model combining autoregression, differencing, and moving-average terms.
• Autoregressive — a series predicted from its own past values.
• RMSE — root mean squared error, in the same units as the target.
• SHAP — Shapley-value feature attributions explaining individual predictions.
• StandardScaler — zero-mean/unit-variance normalization fit on training data.

DInterview Q&A

Why a Transformer over an LSTM? Parallel training and direct long-range attention — month $t$ can attend to a 12-month-old shock without information decaying through recurrent steps.

Why keep an ARIMA baseline? It is a strong, interpretable, low-data benchmark. Beating it is meaningful; failing to is informative and reported honestly.

How do you prevent leakage? Windows only look backward, the split is chronological, and scalers are fit on training data only.

Why scale the target? It conditions the MSE loss for stable optimization; predictions are inverse-transformed before RMSE.

Why mean-pool and last-step? The mean is a smooth window summary; the last step is the most recent state. Concatenating both gave a stronger head than either alone or a CLS token.

Why SHAP instead of raw attention weights? Attention is not a faithful importance measure; SHAP gives consistent, additive attributions with theoretical guarantees.

What would you change with more data? Real FRED series, multi-horizon and quantile outputs, and a more sample-efficient architecture (patch/temporal-fusion Transformer).