Module-Level API

The module-level API provides a simplified, ergonomic interface for nirs4all. These functions serve as the primary entry points for most users.

Overview

import nirs4all

# Training
result = nirs4all.run(pipeline, dataset)

# Prediction
predictions = nirs4all.predict(model, new_data)

# Explanation
explanations = nirs4all.explain(model, data)

# Retraining
new_result = nirs4all.retrain(model, new_data)

# Session for multiple runs
with nirs4all.session(verbose=1) as s:
    result1 = nirs4all.run(pipeline1, data, session=s)
    result2 = nirs4all.run(pipeline2, data, session=s)

Functions

nirs4all.fit_native_pipeline()

Run the deliberately narrow, fully-native raw-array training lane. This is the portable alternative when the pipeline is one splitter plus one supported model and every training and prediction row has a stable identity.

from sklearn.cross_decomposition import PLSRegression
from sklearn.model_selection import KFold

methods_library = "/opt/nirs4all/lib/libn4m.so"

estimator = nirs4all.fit_native_pipeline(
    [KFold(n_splits=3), {"model": PLSRegression(n_components=2)}],
    X_train,
    y_train,
    sample_ids=["train-0", "train-1", "train-2"],
    methods_library_path=methods_library,
)
estimator.export_native_archive("model.n4a", archive_id="archive:demo.pls")
prediction = estimator.predict_with_identity(
    X_predict,
    sample_ids=["predict-0", "predict-1"],
)

The fit result retains the actual DAG-ML TrainingResult, TrainingOutcome, and Package V2. Export writes Archive V2 from those exact native objects; it does not invoke the compatibility PipelineRunner or refit. Unsupported pipeline shapes, implicit identities, non-finite arrays, absent native capabilities, and non-native probability decoding fail explicitly. methods_library_path is an explicit deployment override. With nirs4all[native], the compatible library from the installed nirs4all-methods wheel is discovered automatically; the route never substitutes a Python backend.

Native full refit and Archive V3

An in-memory NativeMethodsRunResult can produce one fresh full-refit child on a new, explicitly identified training cohort. The parent Package V2 remains the authority for the selected plan; the child Package V3 records only fresh REFIT evidence and its artifact. It never re-enters CV or SELECT and never invents a score for the new cohort.

child = nirs4all.retrain(
    result,
    {"X": X_target, "y": y_target, "sample_ids": target_ids},
    mode="full",
    engine="native",
)
child.export("refit.n4a")  # Core Archive V3

with nirs4all.load_session("refit.n4a", engine="native") as loaded:
    prediction = nirs4all.predict(
        data={"X": X_predict, "sample_ids": prediction_ids},
        engine="native",
        session=loaded,
    )

This first retrain capability is intentionally narrow: it needs the in-memory, attested parent and mode="full". Archive-parent retrain, transfer, fine-tuning, host-sidecars and implicit sample identities fail before native data execution. Archive V3 supports PREDICT only and carries no conformal presentation state.

Generic n4m role recipes and trained envelopes (version 8)

Every n4m.roles estimator or procedure serializes as the language-neutral token "n4m:<catalog method id>" (with params when they differ from the native defaults), for example {"class": "n4m:models.pls.cppls", "params": {"n_components": 3}}. Pipeline JSON/YAML files may use these tokens as steps or inside exclude, tag, model and sample_augmentation; the R, JS/WASM and Rust n4m bindings resolve the same token from the native manifest.

PortableN4MRolePipeline (the nirs4all_core.N4mRolePipeline wrapper over the native Methods role pipeline) fits such a recipe (sample filters, transformers and selectors, then one regressor or classifier) and exports every fitted step as its native N4ME state. Any n4m binding rebuilds the pipeline from those bytes and predicts identically:

from nirs4all.pipeline.portable_n4m_roles import PortableN4MRolePipeline

recipe = {"pipeline": [
    {"class": "n4m:filters.y_outlier", "params": {"threshold": 2.0}},
    "n4m:preprocessing.scatter.snv",
    {"class": "n4m:models.pls.cppls", "params": {"n_components": 3}},
]}
fitted = PortableN4MRolePipeline.fit_recipe(recipe, X_train, y_train)
fitted.to_json("trained-v8.json")
replayed = PortableN4MRolePipeline.from_json("trained-v8.json")
predictions = replayed.predict(X_validation)

The envelope (nirs4all.n4m.trained_pipeline.v8) holds the recipe, the input width, feature_names when the fit had column names and, per fitted step, method_id, the base64 N4ME bytes, their SHA-256 and contains_training_rows (plus class_names for classifiers trained on label names). Envelopes without feature_names / contains_training_rows still load. Filters act on training rows only and carry no state. The native pipeline checks the recipe (one final regressor or classifier), passes every target column to the steps that need y, refuses a DataFrame whose columns are renamed or reordered and, on import, states that contradict the recipe. A state that embeds training rows (kernel PLS, LW-PLS, …) is written only with to_json(..., allow_training_rows=True).

R/Python trained n4m recipe envelope

For qualified n4m-only recipes, the R package nirs4all and the full Python package can exchange a bounded JSON envelope containing the recipe and hash-checked fitted native state. Earlier envelopes cover PLS regression, sparse PLS-DA, native selectors and qualified affine regressions. Version 6 adds an explicit fitted N4MP preprocessing payload alongside N4MM model bytes:

from nirs4all.pipeline.portable_n4m_trained import PortableN4MTrainedPipeline

with PortableN4MTrainedPipeline.from_json("trained-in-r.json") as fitted:
    predictions = fitted.predict(X_validation)
    fresh = fitted.retrain(X_new_train, y_new_train)
    fresh_predictions = fresh.predict(X_validation)

with PortableN4MTrainedPipeline.fit_recipe(recipe, X_train, y_train) as fitted:
    fitted.to_json("trained-in-python.json")  # importable by nirs4all R

# Opt in to version 6 for a qualified linear native preprocessing chain.
with PortableN4MTrainedPipeline.fit_recipe(
    recipe, X_train, y_train, preprocessing="native_n4mp"
) as fitted:
    fitted.to_json("trained-v6-in-python.json")

Version 6 accepts only the qualified default SNV/MSC, bounded polynomial detrend and supported Savitzky–Golay settings before PLS or a qualified affine regressor. It verifies the native operator plan, input width, ordered feature names, payload hashes and model descriptor at import; prediction executes the fitted C++ state without learning from validation rows. R-produced version 6 artifacts predict and retrain in Python; Python-produced RidgePLS and GroupSparsePLS artifacts predict in R. EMSC, branches, selectors and other unqualified preprocessing are rejected in version 6 because their semantics have not been established as equivalent to the generic native pipeline.

Both bindings use Methods for preprocessing and native model prediction. Cross-language tests compare held-out predictions in both directions for plain, stateless, MSC/EMSC, embedded SNV/Savitzky-Golay and feature-merge branch profiles. The version-2 sparse PLS-DA envelope carries ordered text class labels and native N4MM affine class scores. predict() returns labels, while predict_scores() returns the raw score matrix and predict_proba() applies an uncalibrated softmax. Plain, SNV and MSC classifier profiles are tested R↔Python, including fresh retraining. An affine N4MM descriptor verifies dimensions and prediction semantics, but cannot independently attest the training algorithm, hyperparameters or class-name mapping declared by the manifest. Its SHA-256 checks are not digital signatures; load artifacts only from a trusted source. With the current Methods source binding, MSC/EMSC references are exported and restored by the native ABI. The published nirs4all-methods 1.0.21 wheel does not yet expose those accessors: this API reconstructs only the documented training-column mean as portable metadata and restores it through a one-row native fit. Both paths pass the same held-out numerical tests; neither fits state on validation rows. retrain performs a fresh Methods fit; it does not reuse the imported model or training references. This envelope is not a DAG-ML Archive V2/V3 package: it has no DAG selection, OOF, sample-identity or refit provenance, and it does not cover non-n4m controllers or arbitrary n4m methods. Those require a later native artifact contract rather than silently placing host state in this wire.

nirs4all.run()

Execute a training pipeline on a dataset.

result = nirs4all.run(
    pipeline,           # Pipeline definition (list, dict, path, or list of pipelines)
    dataset,            # Dataset (path, arrays, config, or list of datasets)
    *,
    name="",            # Pipeline name for logging
    session=None,       # Optional Session for resource sharing
    verbose=1,          # Verbosity level (0-3)
    save_artifacts=True,  # Save model artifacts
    save_charts=True,   # Save visualization charts
    plots_visible=False,  # Show plots interactively
    random_state=None,  # Random seed for reproducibility
    **runner_kwargs     # Additional PipelineRunner options
)

Parameters:

Parameter	Type	Description
`pipeline`	`PipelineSpec`	Pipeline definition as list of steps, dict config, YAML path, `PipelineConfigs`, or list of pipelines
`dataset`	`DatasetSpec`	Dataset as path, numpy arrays, tuple, dict, `DatasetConfigs`, or list of datasets
`name`	`str`	Optional pipeline name for identification
`session`	`Session`	Optional session for resource reuse
`verbose`	`int`	Verbosity: 0=quiet, 1=info, 2=debug, 3=trace
`save_artifacts`	`bool`	Whether to save model artifacts
`save_charts`	`bool`	Whether to save charts
`plots_visible`	`bool`	Whether to display plots interactively
`random_state`	`int`	Random seed for reproducibility

Returns: RunResult containing predictions and convenience accessors.

Verified native Methods subset

engine="native" is currently a deliberately narrow, fail-closed route. It accepts a list containing one CV splitter and one supported Methods model, and a mapping with finite X/y arrays plus explicit stable sample_ids. It returns NativeMethodsRunResult without creating a legacy PipelineRunner. Native sessions own the exact pipeline object and can be used with both run(..., session=session) and predict(..., session=session).

from sklearn.cross_decomposition import PLSRegression
from sklearn.model_selection import KFold
import nirs4all

pipeline = [KFold(n_splits=2), {"model": PLSRegression(n_components=1)}]
dataset = {"X": X_train, "y": y_train, "sample_ids": ["train-0", "train-1"]}

with nirs4all.session(pipeline, engine="native") as native:
    result = nirs4all.run(pipeline, dataset, engine="native", session=native, save_charts=False)
    prediction = nirs4all.predict(
        data={"X": X_predict, "sample_ids": ["predict-0", "predict-1"]},
        session=native,
        engine="native",
    )
    result.export("model.n4a")

Live Methods execution observation

Before a strict native result is closed, result.native_execution_claim gives an immutable, audit-only description of the local callback-free dag_ml.execute_methods_training call and result.native_execution_is_live reports whether its exact DAG-ML facade is still attached. This is deliberately process-local rather than a portable receipt: result.close(), result.detach(), or closing the owning native session invalidates the claim. The signed outcome and portable package are the durable evidence, so result.export("model.n4a") remains valid after that live observation has been released.

The native tuning request is {"engine": "methods-hpo", "trials": N}. It may additionally select sampler="random"|"tpe" and pruner="none"|"median". It uses the DAG-ML Methods controller with the attested V1 PLS n_components search space; no Python objective, callback, intermediate score or optimizer-specific setting can be injected. A session can bind that same request once:

with nirs4all.session(
    pipeline,
    engine="native",
    tuning={"engine": "methods-hpo", "trials": 8, "sampler": "tpe", "pruner": "median"},
) as native:
    result = native.run(dataset)

An in-memory native session can also full-refit its already attested selected Methods variant on a new explicitly identified cohort:

with nirs4all.session(pipeline, engine="native") as native:
    native.run(dataset)
    refitted = native.retrain(
        {"X": X_next, "y": y_next, "sample_ids": ["next-0", "next-1"]}
    )

This is not transfer learning or finetuning: it preserves only the selected portable PLS recipe and its attested patch. Each supported full retrain writes the source outcome, request, plan, selected variant and seed into the signed outcome diagnostics of the new result; exporting that result preserves this parent-child lineage in the new native archive. Archive-based retraining, transfer/finetuning, calibration, generic branching/stacking and explanation remain explicit capability refusals; none is redirected to the legacy backend.

Example - Single pipeline:

import nirs4all
from sklearn.preprocessing import MinMaxScaler
from sklearn.cross_decomposition import PLSRegression

result = nirs4all.run(
    pipeline=[MinMaxScaler(), PLSRegression(10)],
    dataset="sample_data/regression",
    name="QuickTest",
    verbose=1
)

print(f"Best RMSE: {result.best_rmse:.4f}")
print(f"Best R²: {result.best_r2:.4f}")

Example - Multiple pipelines (batch execution):

# Define different strategies
pipeline_pls = [MinMaxScaler(), PLSRegression(10)]
pipeline_rf = [StandardScaler(), RandomForestRegressor()]

# Run both independently
result = nirs4all.run(
    pipeline=[pipeline_pls, pipeline_rf],  # List of pipelines
    dataset="sample_data/regression",
    verbose=1
)
print(f"Total configurations: {result.num_predictions}")

Example - Cartesian product (pipelines × datasets):

# 2 pipelines × 2 datasets = 4 runs
result = nirs4all.run(
    pipeline=[pipeline_pls, pipeline_rf],
    dataset=["data/wheat", "data/corn"],
    verbose=1
)
# Runs: PLS×wheat, PLS×corn, RF×wheat, RF×corn

nirs4all.predict()

Make predictions with a trained model.

result = nirs4all.predict(
    source,             # Trained model (prediction dict, bundle path, etc.)
    dataset,            # New data for prediction
    *,
    verbose=0,          # Verbosity level
    all_predictions=False,  # Return all non-conformal prediction entries
    coverage=None,      # Select materialized conformal interval coverage(s)
    save_to_workspace=False,  # Publish prediction/evidence row to workspace
    **kwargs            # Additional options
)

Parameters:

Parameter	Type	Description
`source`	`PredictionSource`	Trained model as prediction dict, `RunResult`, or bundle path
`dataset`	`DatasetSpec`	New data for prediction
`verbose`	`int`	Verbosity level
`all_predictions`	`bool`	If True, return predictions from all CV folds/entries for non-conformal predictions. With a conformal sidecar and `coverage=...`, only `all_predictions=False` is currently supported.
`coverage`	`float \| list[float] \| None`	Select already-materialized conformal interval coverages for calibrated replayed-array results or model `.n4a` bundles carrying a conformal sidecar. It never recalibrates or creates a new guarantee; invalid sidecars fail validation instead of falling back to uncalibrated prediction, including sidecars whose non-empty calibrated prediction cohort is missing canonical physical `sample_ids`.
`save_to_workspace`	`bool`	If True, publish the returned `PredictResult` through the workspace prediction store and add `workspace_prediction_id` to `result.metadata`. This writes prediction/evidence rows only; conformal artifacts remain owned by `save_workspace_calibrated_result(...)`.

Returns: PredictResult containing prediction values.

Example:

import nirs4all

# From training result
result = nirs4all.run(pipeline, train_data)
predictions = nirs4all.predict(result.best, test_data)
print(predictions.values)

# From exported bundle
predictions = nirs4all.predict("exports/model.n4a", new_data)
df = predictions.to_dataframe()

# Publish prediction evidence directly into a workspace
published = nirs4all.predict(
    "exports/model.n4a",
    {"X": new_X},
    workspace_path="workspace/",
    save_to_workspace=True,
    workspace_result_metadata={"robustness_evidence": {"predictor_bundle": "exports/model.n4a"}},
)
prediction_id = published.metadata["workspace_prediction_id"]

nirs4all.calibrate()

Fit or apply the current public split-conformal calibration surface.

calibrated = nirs4all.calibrate(
    calibration_data=(y_cal, y_cal_pred, calibration_sample_ids),
    y_pred=y_pred,
    prediction_sample_ids=prediction_sample_ids,
    coverage=[0.9, 0.95],
)

Current scope: replayed-array calibration evidence, explicit physical sample ids, selected dataset-backed calibration cohorts, verified filesystem/workspace persistence, and .n4a conformal result archives. Unsupported implicit splits and broad automatic predictor discovery fail closed.

Returns: CalibratedRunResult by default, or PredictResult when as_predict_result=True.

Related helpers:

nirs4all.predict_calibrated(...) applies an existing conformal calibrator to new already-computed point predictions.
nirs4all.conformal_metrics(...) computes empirical observed coverage, interval width and interval score diagnostics.
nirs4all.load_workspace_calibrated_predict_result(...) loads a workspace conformal result directly as PredictResult, preserving intervals, calibrated_result_fingerprint, calibration_replay_source, and tuning_calibration_source for bindings and notebooks.
nirs4all.load_workspace_predict_result(...) loads one workspace prediction row directly as PredictResult, preserving arrays, sample ids, model metadata and replay provenance when the workspace stores them.
nirs4all.load_workspace_predict_results(...) performs the same conversion for all matching workspace prediction rows, optionally filtered by dataset.
nirs4all.save_workspace_predict_result(...) persists a PredictResult back into the workspace prediction store, including row-aligned executable X/spectra evidence and result_metadata for later robustness replay. nirs4all.predict(..., save_to_workspace=True, workspace_result_metadata=...) is the direct prediction-time shortcut for the same publisher.
nirs4all.robustness_from_workspace_prediction(...) loads one workspace prediction as PredictResult, lets robustness() consume any executable X/spectra + predictor_bundle evidence already stored on the row, and can persist the report back to the workspace linked to that prediction id.
nirs4all.load_calibrated_result(...), nirs4all.export_calibrated_result(...), and nirs4all.attach_calibrated_result_to_bundle(...) handle local conformal artifacts.

nirs4all.robustness()

Create an audit-only robustness/generalization report for already replayed prediction cohorts.

report = nirs4all.robustness(
    calibrated,
    y_true=y_observed,
    scenarios=[nirs4all.RobustnessScenarioSpec(kind="observed")],
    slice_by=["instrument"],
    metadata={"instrument": instrument_ids},
    workspace_path="workspace/",
    workspace_robustness_id="external-audit",
    workspace_name="External robustness audit",
)
report.save_json("robustness-report.json")
report.save_html("robustness-report.html")
report.save_summary("robustness-summary.json")
report.save_artifacts("robustness-artifacts")
summary_schema = nirs4all.get_robustness_summary_schema()
summary_schema_json = nirs4all.robustness_summary_schema_json()
verified_bundle = nirs4all.RobustnessReport.load_artifacts("robustness-artifacts")

# Native tuning summaries use the same lightweight publication pattern:
tuning_summary = result.tuning_result.summary_artifact()
tuning_schema = nirs4all.get_tuning_summary_schema()
tuning_schema_json = nirs4all.tuning_summary_schema_json()
tuning_space_schema = nirs4all.get_tuning_space_schema()
tuning_space_schema_json = nirs4all.tuning_space_schema_json()
result.tuning_result.save_summary("tuning-summary.json")

# Equivalent convenience syntax on public result containers:
same_report_from_result = calibrated.robustness(y_true=y_observed)
same_report_from_prediction = prediction.robustness(y_true=y_observed)
same_report_from_workspace_prediction = nirs4all.robustness_from_workspace_prediction(
    "workspace/",
    "pred-001",
    y_true=y_observed,
    scenarios=[{"kind": "spectral_offset", "severity": 0.01}],
    save_to_workspace=True,
    workspace_robustness_id="pred-001-spectral-audit",
)

robustness_id = nirs4all.save_workspace_robustness_report(
    "workspace/",
    report,
    robustness_id="external-audit-copy",
    name="External robustness audit",
)
same_report = nirs4all.load_workspace_robustness_report("workspace/", "external-audit")

The same verified JSON artifact can be republished from the CLI without re-running the audit:

nirs4all robustness-report robustness-report.json --format markdown --output robustness-report.md
nirs4all robustness-report robustness-report.json --format summary --output robustness-summary.json
nirs4all robustness-report robustness-report.json --format html --output robustness-report.html
nirs4all robustness-report robustness-report.json --format parquet --output robustness-report.parquet
nirs4all robustness-report robustness-report.json --format artifacts --output robustness-artifacts/
nirs4all robustness-report robustness-artifacts/ --format markdown --output reviewed-report.md
nirs4all workspace robustness list --workspace workspace --json
nirs4all workspace robustness show external-audit --workspace workspace --json
nirs4all workspace robustness from-prediction --workspace workspace --prediction-id pred-001 --y-true "1.0,2.0,3.0" --scenarios-json '[{"kind":"spectral_offset","severity":0.01}]' --save-to-workspace --workspace-robustness-id pred-001-spectral-audit --format summary --output robustness-summary.json
nirs4all workspace robustness export external-audit --workspace workspace --format summary --output robustness-summary.json
nirs4all workspace robustness export external-audit --workspace workspace --format artifacts --output robustness-artifacts/
nirs4all workspace robustness export external-audit --workspace workspace --format html --output robustness-report.html

workspace robustness from-prediction is the CLI equivalent of nirs4all.robustness_from_workspace_prediction(...): it requires observed targets through --y-true or --y-true-json, accepts scenarios through --scenarios-json, can slice with --metadata-json + --slice-by, and can persist the generated report with --save-to-workspace.

Current scope: frozen prediction audits, deterministic post-prediction stress cells, optional spectral-noise replay with an explicit predictor, point metrics, conformal diagnostics, slices, degradation rows, worst slices, and deterministic JSON/Markdown/HTML/Parquet exports. It does not refit or renew a conformal guarantee.

nirs4all.ROBUSTNESS_MODES lists the reserved mode vocabulary: clean_frozen, matched_recalibration, and structural_refit. nirs4all.ROBUSTNESS_EXECUTABLE_MODES currently contains only clean_frozen; the other modes are reserved and fail closed until their matched recalibration/refit protocols are implemented.

scenarios accepts RobustnessScenarioSpec helpers or equivalent mappings. The supported scenario kind values are observed, prediction_bias, prediction_noise, spectral_noise, spectral_offset, spectral_scale, spectral_slope, and spectral_shift; distribution is currently accepted only for prediction_noise and spectral_noise, and is fail-closed to "normal" when provided. Spectral replay scenarios require explicit X= and exactly one frozen replay source: an in-memory predictor= or a saved predictor_bundle= path. The bundle path is replayed through public nirs4all.predict(model=predictor_bundle, data={"X": X, "sample_ids": ...}, all_predictions=False) without refit or recalibration. Spectral reports record metadata["spectral_replay"] with the replay source, bundle path for saved-bundle replay, route, and sample-id forwarding status. RobustnessReport.summary_artifact() carries the same spectral_replay block when present so UI/CI consumers can audit replay provenance without loading the full report. nirs4all.ROBUSTNESS_SCENARIO_KINDS exposes the same ordered vocabulary for Python tooling, bindings and Studio forms, and matches the keyword registry enum and report metadata. nirs4all.ROBUSTNESS_STOCHASTIC_SCENARIO_KINDS lists the scenario kinds that accept distribution, and nirs4all.ROBUSTNESS_SCENARIO_DISTRIBUTIONS lists the accepted distribution tokens.

Returns: RobustnessReport.

nirs4all.get_keyword_registry()

Return the machine-readable keyword/effect registry used by generated docs, forms, Studio and bindings.

registry = nirs4all.get_keyword_registry()
payload = nirs4all.keyword_registry_json(indent=2)
schema = nirs4all.get_keyword_registry_schema()
schema_payload = nirs4all.keyword_registry_schema_json(indent=2)

scenario_entry = next(
    entry for entry in registry["entries"] if entry["id"] == "robustness.scenarios"
)
print(scenario_entry["value_schema"])

The same registry can be exported from the CLI for CI, docs generation or Studio build steps:

nirs4all keyword-registry --output keyword-registry.json
nirs4all keyword-registry --schema --output keyword-registry.schema.json
nirs4all keyword-registry --compact
nirs4all robustness-summary-schema --output robustness-summary.schema.json

Published HTML documentation also includes the same document as _static/keyword-registry.json and the JSON Schema as _static/keyword-registry.schema.json, so Studio/Web builds can consume and validate the registry without importing Python at runtime. It also publishes _static/robustness-summary.schema.json for robustness summary cards.

The registry records keyword paths, value schemas, read/change effects, calibration invalidation behavior, engine support and UI hints. It is descriptive only; execution still happens through the public runtime functions.

Native tuning helpers

The native DAG-ML tuning subset can be driven through run(tuning=...) or the lower-level tune_single_estimator(...) helper. Typed helpers are exported from nirs4all for forms, bindings and scripts:

NativeTuning
TUNING_ENGINES
TUNING_DIRECTIONS
TUNING_CONTRACT_KEYS
TUNING_RUNTIME_KEYS
TUNING_OPTIMIZER_PERSISTENCE_KEYS
TUNING_SPACE_SCHEMA_ID
inspect_tuning_space
get_tuning_space_schema() and tuning_space_schema_json()
OrderedSearchSpaceSpec, SearchSpaceParameter, and ParameterPatch
CONFORMAL_TUNING_SCORE_METRICS
FINETUNE_ENGINES
FINETUNE_APPROACHES
FINETUNE_EVAL_MODES
FINETUNE_OPTUNA_SAMPLERS
FINETUNE_OPTUNA_PRUNERS
FINETUNE_N4M_SAMPLERS
FINETUNE_N4M_PRUNERS
FINETUNE_DAGML_DETERMINISTIC_ENGINES
FINETUNE_DAGML_META_KEYS
FINETUNE_DAGML_SELECTION_METRICS
FINETUNE_DAGML_APPROACHES
FINETUNE_DAGML_EVAL_MODES
FINETUNE_ENGINE_ALIASES
FINETUNE_SAMPLER_KEY_ALIASES
FINETUNE_EVAL_MODE_ALIASES
CONFORMAL_CALIBRATION_METHODS
CONFORMAL_CALIBRATION_UNITS
CONFORMAL_EXECUTABLE_MULTI_TARGET_POLICIES
CONFORMAL_MULTI_TARGET_POLICIES
ConformalCalibrationData
ConformalMethod
TuningEngine
TuningDirection
FinetuneEngine
FinetuneSampler
FinetunePruner
FinetuneApproach
FinetuneEvalMode
TuningScoreData
TuningConformalScoreCalibration
TuningWinner
TuningCalibration
TunedSingleEstimatorConformalResult
TuningResult
TrialResult
ConformalMultiTarget
ConformalUnit
RobustnessScenarioSpec
ROBUSTNESS_EXECUTABLE_MODES
ROBUSTNESS_MODES
ROBUSTNESS_SCENARIO_DISTRIBUTIONS
ROBUSTNESS_SCENARIO_KINDS
ROBUSTNESS_STOCHASTIC_SCENARIO_KINDS
RobustnessMode
RobustnessScenarioDistribution
RobustnessScenarioKind

NativeTuning(..., force_params={...}) is part of the deterministic tuning contract. It enqueues an explicit first optimizer trial with public decoded parameter values whose keys must be present in space; changing it can change the selected predictor and therefore stale any previous calibration.

When calibration is requested through run(tuning=..., calibration=...) or tune_single_estimator(..., calibration=...), the return value is TunedSingleEstimatorConformalResult. It keeps the tuned RunResult in result.run and the calibrated PredictResult or CalibratedRunResult in result.calibrated, but routine access does not require reaching into those fields:

print(result.tuning_best_params)
interval_90 = result.interval(0.9)
metrics = result.metrics(y_true_prediction)
report = result.robustness(y_true=y_true_prediction)

Those accessors delegate to the existing tuned run and calibrated result. They do not refit, recalibrate, recompute intervals, or infer a new guarantee.

See Native tuning and conformal calibration for the supported syntax, keywords, guarantee metadata and current fail-closed boundaries.

nirs4all.explain()

Generate SHAP explanations for model predictions.

result = nirs4all.explain(
    source,             # Trained model
    dataset,            # Data to explain
    *,
    n_samples=200,      # Number of samples for SHAP
    explainer_type="auto",  # SHAP explainer type
    verbose=0,          # Verbosity level
    **kwargs            # Additional SHAP options
)

Parameters:

Parameter	Type	Description
`source`	`PredictionSource`	Trained model
`dataset`	`DatasetSpec`	Data to explain
`n_samples`	`int`	Number of background samples for SHAP
`explainer_type`	`str`	“auto”, “tree”, “kernel”, “deep”, or “linear”

Returns: ExplainResult containing SHAP values and feature importance.

Example:

import nirs4all

result = nirs4all.run(pipeline, data)
explanations = nirs4all.explain(result.best, data, n_samples=100)

print(explanations.top_features[:10])
importance = explanations.get_feature_importance(top_n=20)

nirs4all.retrain()

Retrain a pipeline on new data.

result = nirs4all.retrain(
    source,             # Original trained model
    dataset,            # New training data
    *,
    mode="full",        # Retrain mode
    verbose=1,          # Verbosity level
    **kwargs            # Additional options
)

Parameters:

Parameter	Type	Description
`source`	`PredictionSource`	Original trained model
`dataset`	`DatasetSpec`	New training data
`mode`	`str`	“full”, “transfer”, or “finetune”
`verbose`	`int`	Verbosity level

Returns: RunResult from retraining.

Example:

import nirs4all

# Original training
result = nirs4all.run(pipeline, data_2023)

# Retrain on new data
new_result = nirs4all.retrain(
    source=result.best,
    dataset=data_2024,
    mode="transfer"
)

nirs4all.session()

Create an execution session for resource reuse.

with nirs4all.session(
    verbose=1,
    save_artifacts=True,
    **runner_kwargs
) as s:
    # Multiple runs share configuration
    result1 = nirs4all.run(pipeline1, data, session=s)
    result2 = nirs4all.run(pipeline2, data, session=s)

Parameters:

All parameters are passed to the underlying PipelineRunner. Common options:

Parameter	Type	Description
`verbose`	`int`	Default verbosity for all runs
`save_artifacts`	`bool`	Default artifact saving
`workspace_path`	`Path`	Shared workspace directory

Returns: Session context manager.

Example:

import nirs4all
from sklearn.cross_decomposition import PLSRegression
from sklearn.ensemble import RandomForestRegressor

# Compare multiple models efficiently
with nirs4all.session(verbose=1, save_artifacts=True) as s:
    for n_components in [5, 10, 15, 20]:
        result = nirs4all.run(
            pipeline=[PLSRegression(n_components=n_components)],
            dataset="data",
            name=f"PLS_{n_components}",
            session=s
        )
        print(f"PLS({n_components}): RMSE = {result.best_rmse:.4f}")

Result Classes

RunResult

Result from nirs4all.run(). Wraps predictions with convenience accessors.

Properties:

Property	Type	Description
`best`	`dict`	Best prediction entry
`best_score`	`float`	Best model’s primary test score
`best_rmse`	`float`	Best RMSE (regression)
`best_r2`	`float`	Best R² (regression)
`best_accuracy`	`float`	Best accuracy (classification)
`num_predictions`	`int`	Total prediction count
`artifacts_path`	`Path`	Path to saved artifacts
`predictions`	`Predictions`	Raw predictions object
`per_dataset`	`dict`	Per-dataset details

Methods:

Method	Description
`top(n=5)`	Get top N predictions
`filter(**kwargs)`	Filter predictions by criteria
`get_models()`	List unique model names
`get_datasets()`	List unique dataset names
`export(path)`	Export best model to bundle
`summary()`	Get summary string

PredictResult

Result from nirs4all.predict().

Properties:

Property	Type	Description
`values`	`ndarray`	Prediction values (alias for y_pred)
`y_pred`	`ndarray`	Raw prediction array
`shape`	`tuple`	Shape of predictions
`is_multioutput`	`bool`	True if multiple outputs
`model_name`	`str`	Name of model used

Methods:

Method	Description
`to_numpy()`	Get as numpy array
`to_list()`	Get as Python list
`to_dataframe()`	Get as pandas DataFrame
`flatten()`	Get flattened 1D array

ExplainResult

Result from nirs4all.explain().

Properties:

Property	Type	Description
`values`	`ndarray`	Raw SHAP values
`shap_values`	`Any`	SHAP Explanation object or array
`shape`	`tuple`	Shape of SHAP values
`mean_abs_shap`	`ndarray`	Mean
`top_features`	`list`	Features sorted by importance
`feature_names`	`list`	Feature names
`base_value`	`float`	Baseline prediction

Methods:

Method	Description
`get_feature_importance(top_n=None)`	Get importance ranking
`get_sample_explanation(idx)`	Get explanation for single sample
`to_dataframe()`	Get as pandas DataFrame

Type Aliases

PipelineSpec

Pipeline definition accepts multiple formats:

# List of steps (most common)
[MinMaxScaler(), PLSRegression(10)]

# Dict configuration
{"steps": [...], "name": "my_pipeline"}

# Public wrapper alias
{"pipeline": [...], "name": "my_pipeline"}

# Path to YAML/JSON config
"configs/my_pipeline.yaml"

# PipelineConfigs object
PipelineConfigs(steps, name="...")

# List of pipelines (batch execution)
[pipeline1, pipeline2, pipeline3]  # Each runs independently

DatasetSpec

Dataset accepts multiple formats:

# Path to data folder
"sample_data/regression"

# Numpy arrays
(X, y)
X  # y inferred as None

# Dict with arrays
{"X": X, "y": y, "metadata": meta}

# SpectroDataset instance
SpectroDataset(...)

# DatasetConfigs object
DatasetConfigs("path")

# List of datasets (batch execution)
["data/wheat", "data/corn"]  # Each dataset tested
[dataset1, dataset2]  # List of SpectroDataset instances

Batch Execution

When you pass lists for both pipeline and dataset, nirs4all.run() executes the cartesian product:

# 3 pipelines × 2 datasets = 6 runs
result = nirs4all.run(
    pipeline=[pipeline_a, pipeline_b, pipeline_c],
    dataset=["data/wheat", "data/corn"]
)

All results are collected into a single RunResult for unified analysis.

Module-Level API

Overview

Functions

nirs4all.fit_native_pipeline()

Native full refit and Archive V3

Generic n4m role recipes and trained envelopes (version 8)

R/Python trained n4m recipe envelope

nirs4all.run()

Verified native Methods subset

Live Methods execution observation

nirs4all.predict()

nirs4all.calibrate()

nirs4all.robustness()

nirs4all.get_keyword_registry()

Native tuning helpers

nirs4all.explain()

nirs4all.retrain()

nirs4all.session()

Result Classes

RunResult

PredictResult

ExplainResult

Type Aliases

PipelineSpec

DatasetSpec

Batch Execution

See Also