nirs4all.api.robustness module

Public audit-only robustness/generalization reports.

class nirs4all.api.robustness.RobustnessMetricSet(n_samples: int, rmse: float, mae: float, bias: float, max_abs_error: float, version: int = 1)

Bases: object

Point-prediction diagnostics for one robustness report cell.

bias: float

property fingerprint: str

TCV1 fingerprint of the metric payload.

classmethod from_dict(payload: Mapping[str, Any]) → RobustnessMetricSet

Parse and verify a point-metric payload.

mae: float

max_abs_error: float

n_samples: int

rmse: float

to_dict() → dict[str, Any]

Return a JSON-compatible metric payload.

version: int = 1

class nirs4all.api.robustness.RobustnessReport(mode: ~typing.Literal['clean_frozen', 'matched_recalibration', 'structural_refit'], scenarios: tuple[~nirs4all.api.robustness.RobustnessScenarioResult, ...], slice_by: tuple[str, ...] = (), metadata: ~collections.abc.Mapping[str, ~typing.Any] = <factory>, version: int = 1)

Bases: object

Audit-only robustness/generalization report.

degradation_rows(*, reference: int = 0) → tuple[dict[str, Any], ...]

Return deterministic per-scenario degradation rows versus a reference.

property fingerprint: str

TCV1 fingerprint of the report.

classmethod from_dict(payload: Mapping[str, Any]) → RobustnessReport

Parse a serialized report and verify its fingerprint.

classmethod from_json(payload: str) → RobustnessReport

Parse a JSON report and verify its fingerprint.

classmethod load_artifacts(path: str | Path) → RobustnessReport

Load and verify a deterministic artifact directory export.

classmethod load_json(path: str | Path) → RobustnessReport

Load a persisted report and verify its fingerprint.

classmethod load_parquet(path: str | Path) → RobustnessReport

Load a Parquet-directory export and verify the embedded report.

metadata: Mapping[str, Any]

mode: Literal['clean_frozen', 'matched_recalibration', 'structural_refit']

save_artifacts(path: str | Path, *, formats: Sequence[str] = ('json', 'summary', 'markdown', 'html', 'parquet')) → dict[str, Path]

Persist a complete deterministic artifact directory with manifest.

save_html(path: str | Path) → Path

Persist the report as a deterministic standalone HTML summary.

save_json(path: str | Path) → Path

Persist the report as deterministic verified JSON.

save_markdown(path: str | Path) → Path

Persist the report as a deterministic Markdown summary.

save_parquet(path: str | Path) → Path

Persist flat report tables as a portable Parquet directory.

save_summary(path: str | Path) → Path

Persist the lightweight deterministic summary artifact JSON.

scenarios: tuple[RobustnessScenarioResult, ...]

slice_by: tuple[str, ...] = ()

summary_artifact() → dict[str, Any]

Return the lightweight deterministic summary artifact payload.

summary_rows(*, reference: int = 0, worst_slice_metric: str = 'rmse') → tuple[dict[str, Any], ...]

Return compact per-scenario rows for CI dashboards and Studio.

tabular_records() → dict[str, tuple[dict[str, Any], ...]]

Return deterministic flat records used by tabular exports.

to_dict() → dict[str, Any]

Return a deterministic JSON-compatible report payload.

to_html() → str

Return a deterministic standalone HTML summary of the report.

to_json(*, indent: int | None = 2) → str

Return deterministic JSON with a trailing newline.

to_markdown() → str

Return a deterministic Markdown summary of the report.

to_summary_json(*, indent: int | None = 2) → str

Return deterministic summary JSON with a trailing newline.

version: int = 1

worst_slices(*, metric: str = 'rmse', top_k: int = 1) → tuple[dict[str, Any], ...]

Return the worst diagnostic slices across all scenarios.

class nirs4all.api.robustness.RobustnessScenarioResult(scenario: ~collections.abc.Mapping[str, ~typing.Any], severity: float, metrics: ~nirs4all.api.robustness.RobustnessMetricSet, conformal_metrics: ~collections.abc.Mapping[float, ~nirs4all.pipeline.dagml.conformal_contracts.ConformalMetricSet] = <factory>, slices: tuple[~nirs4all.api.robustness.RobustnessSliceResult, ...] = (), version: int = 1)

Bases: object

Audit-only result for one scenario/severity cell.

conformal_metrics: Mapping[float, ConformalMetricSet]

property fingerprint: str

TCV1 fingerprint of the scenario payload.

classmethod from_dict(payload: Mapping[str, Any]) → RobustnessScenarioResult

Parse and verify one scenario payload.

metrics: RobustnessMetricSet

scenario: Mapping[str, Any]

severity: float

slices: tuple[RobustnessSliceResult, ...] = ()

to_dict() → dict[str, Any]

Return a JSON-compatible scenario payload.

version: int = 1

class nirs4all.api.robustness.RobustnessScenarioSpec(kind: Literal['observed', 'prediction_bias', 'prediction_noise', 'spectral_noise', 'spectral_offset', 'spectral_scale', 'spectral_slope', 'spectral_shift'] | str, severity: float = 0.0, distribution: str | None = None, extra: Mapping[str, ~typing.Any]=<factory>)

Bases: object

Typed public scenario for nirs4all.robustness() audit reports.

distribution: str | None = None

extra: Mapping[str, Any]

kind: Literal['observed', 'prediction_bias', 'prediction_noise', 'spectral_noise', 'spectral_offset', 'spectral_scale', 'spectral_slope', 'spectral_shift'] | str

severity: float = 0.0

to_dict() → dict[str, Any]

Return the public mapping consumed by nirs4all.robustness().

class nirs4all.api.robustness.RobustnessSliceResult(slice_key: ~collections.abc.Mapping[str, ~typing.Any], metrics: ~nirs4all.api.robustness.RobustnessMetricSet, conformal_metrics: ~collections.abc.Mapping[float, ~nirs4all.pipeline.dagml.conformal_contracts.ConformalMetricSet] = <factory>, version: int = 1)

Bases: object

Diagnostics for one metadata slice.

conformal_metrics: Mapping[float, ConformalMetricSet]

property fingerprint: str

TCV1 fingerprint of the slice payload.

classmethod from_dict(payload: Mapping[str, Any]) → RobustnessSliceResult

Parse and verify one slice payload.

metrics: RobustnessMetricSet

slice_key: Mapping[str, Any]

to_dict() → dict[str, Any]

Return a JSON-compatible slice payload.

version: int = 1

nirs4all.api.robustness.get_robustness_summary_schema() → dict[str, Any]

Return the JSON Schema for robustness summary artifacts.

nirs4all.api.robustness.load_workspace_robustness_report(workspace_path: str | Path, robustness_id: str) → RobustnessReport

Load a verified robustness/generalization report from a nirs4all workspace.

nirs4all.api.robustness.robustness(result: PredictResult | CalibratedRunResult, *, y_true: Any, X: Any | None = None, predictor: Any | None = None, predictor_bundle: str | Path | None = None, mode: Literal['clean_frozen', 'matched_recalibration', 'structural_refit'] = 'clean_frozen', scenarios: Sequence[RobustnessScenarioSpec | Mapping[str, Any]] | None = None, slice_by: Sequence[str] | None = None, metadata: Mapping[str, Any] | Sequence[Mapping[str, Any]] | None = None, seed: int | None = None, workspace_path: str | Path | None = None, workspace_name: str = '', workspace_robustness_id: str | None = None, workspace_metadata: Mapping[str, Any] | None = None) → RobustnessReport

Compute audit-only robustness/generalization diagnostics.

This public audit gate evaluates already replayed predictions. It does not refit or recalibrate. Post-prediction stress scenarios operate on already materialized predictions and intervals; spectral stress scenarios require explicit X together with either predictor or predictor_bundle and replay the frozen predictor without refit or recalibration.

nirs4all.api.robustness.robustness_summary_schema_json(*, indent: int | None = 2) → str

Serialize the robustness summary JSON Schema deterministically.

nirs4all.api.robustness.save_workspace_robustness_report(workspace_path: str | Path, report: RobustnessReport, *, name: str = '', robustness_id: str | None = None, metadata: Mapping[str, Any] | None = None, run_id: str | None = None, pipeline_id: str | None = None, chain_id: str | None = None, conformal_id: str | None = None, prediction_id: str | None = None) → str

Persist a robustness/generalization report in a nirs4all workspace.