nirs4all.pipeline.analysis.model_diagnostics module

Model diagnostics over cross-validation predictions.

Pure numerical analyses used by result-inspection UIs:

• bias_variance_decomposition() — per-sample bias²/variance over repeated CV predictions of the same samples.

• robustness_axes() — normalized multi-axis robustness profile (CV stability, train/val gap, absolute score, fold coverage) across a set of chains.

• learning_curve_points() — train/val score aggregation by training-set size, and estimate_train_size() for the fold-based approximation when the exact size is unavailable.

These previously lived in the nirs4all-studio HTTP layer (flagged as a boundary violation in its 2026-06-05 tech-debt closeout, INS-04). They take plain numbers/sequences — callers assemble the data from the workspace store.

class nirs4all.pipeline.analysis.model_diagnostics.BiasVarianceDecomposition(bias_squared: float, variance: float, total_error: float, n_samples: int)

Bases: object

Aggregated bias²/variance over samples with repeated predictions.

Variables:

• bias_squared (float) – Mean of per-sample (mean_pred - y_true)².

• variance (float) – Mean of per-sample Var(y_pred).

• total_error (float) – bias_squared + variance.

• n_samples (int) – Number of samples with >= 2 predictions that contributed.

bias_squared: float

n_samples: int

total_error: float

variance: float

nirs4all.pipeline.analysis.model_diagnostics.bias_variance_decomposition(sample_predictions: Mapping[Hashable, Sequence[tuple[float, float]]]) → BiasVarianceDecomposition | None

Decompose repeated-prediction error into bias² and variance.

For every sample observed in two or more folds/chains: bias² = (mean_pred - y_true)² and variance = Var(y_pred); the result aggregates both as means over contributing samples.

Parameters:

sample_predictions – Mapping of sample key to its (y_true, y_pred) pairs collected across folds/chains. Non-finite pairs are ignored.

Returns:

The aggregated decomposition, or None when no sample has at least two finite predictions.

nirs4all.pipeline.analysis.model_diagnostics.estimate_train_size(val_size: int, fold_count: int) → int

Approximate the training-set size from a validation-fold size.

Assumes balanced K-fold splitting: total ≈ val_size * K and train ≈ val_size * (K - 1). Used when the exact per-fold training size is not recorded alongside predictions.

Parameters:

• val_size – Number of validation samples in one fold.

• fold_count – Number of CV folds (values < 2 are treated as 5).

Returns:

The estimated training-set size (0 when val_size is not positive).

nirs4all.pipeline.analysis.model_diagnostics.learning_curve_points(size_scores: Mapping[int, Sequence[Mapping[str, float | None]]]) → list[dict[str, Any]]

Aggregate train/val scores per training-set size.

Parameters:

size_scores – Mapping of training size to entries with optional train and val scores.

Returns:

Points sorted by training size, each with train_size, train_mean/train_std, val_mean/val_std (None when insufficient data; std requires >= 2 values) and count.

nirs4all.pipeline.analysis.model_diagnostics.robustness_axes(chains: Sequence[Mapping[str, Any]], *, lower_better: bool = False) → list[dict[str, dict[str, float]]]

Compute the normalized robustness axes for a set of chains.

Each input mapping provides the raw ingredients per chain: fold_scores (per-fold scores for the inspected partition), train_score, val_score, score (the headline score) and fold_count. Axes are normalized 0-1 across the input set (higher = more robust):

• cv_stability: 1 - std(fold_scores)/max_std

• train_test_gap: 1 - |train - val|/max_gap

• score_absolute: min-max normalized headline score (flipped when lower_better)

• fold_count_ratio: fold_count / max_fold_count

Returns:

One dict per input chain (same order), mapping axis name to {"value": normalized, "raw": raw_value}.