Export and Deployment

This guide covers exporting trained pipelines as standalone bundles for deployment, sharing, or archival.

Overview

The bundle export feature allows you to package a trained pipeline into a self-contained file that can be:

Deployed without the full nirs4all workspace
Shared with colleagues or clients
Archived for reproducibility years later
Run on edge devices or in containers

Bundle Formats

nirs4all supports two bundle formats:

`.n4a` Format (Full Bundle)

A ZIP archive containing:

manifest.json: Bundle metadata (chain_id, model_class, preprocessings, etc.)
chain.json: Chain steps with fold and shared artifact references
artifacts/: Directory with serialized model and transformer binaries

Best for: Deployment with nirs4all, archival, full reproducibility

`.n4a.py` Format (Portable Script)

A single Python file with:

Embedded artifacts (base64 encoded)
Standalone predict() function
No nirs4all dependency (only numpy, scipy, scikit-learn, joblib required)

Best for: Lightweight deployment, edge devices, sharing with non-nirs4all users

Warning

Model Compatibility: The .n4a.py portable script only supports sklearn-compatible models (PLSRegression, RandomForest, SVM, XGBoost, etc.). Deep learning models (PyTorch, JAX, TensorFlow) require the full .n4a bundle format and the corresponding framework installed.

Model Compatibility

Not all models work with all export formats:

Model Type	`.n4a` Bundle	`.n4a.py` Script
sklearn (PLS, SVM, RF, etc.)	✅ Full support	✅ Full support
sklearn transformers (MinMaxScaler, SNV, etc.)	✅ Full support	✅ Full support
XGBoost / LightGBM / CatBoost	✅ Full support	⚠️ Requires library
PyTorch models	✅ Full support	❌ Not supported
JAX models	✅ Full support	❌ Not supported
TensorFlow / Keras	✅ Full support	❌ Not supported

Why Deep Learning Models Don’t Work with `.n4a.py`

State dict vs full model: PyTorch/JAX save state dicts, not executable models
Model class required: You need the Python class definition to reconstruct the model
Framework dependency: These models require their full framework at runtime
Device management: GPU/CPU placement logic is framework-specific

For deep learning models, always use the .n4a bundle format and ensure the target environment has nirs4all plus the required framework (torch, jax, tensorflow) installed.

Pipeline Feature Support

The .n4a bundle format supports all nirs4all pipeline features:

Feature	`.n4a` Bundle	`.n4a.py` Script
Linear pipelines	✅ Full support	✅ Full support
CV ensemble (fold averaging)	✅ Full support	✅ Full support
Weighted fold averaging	✅ Full support	✅ Full support
Branching pipelines	✅ Full support	❌ Not supported
Stacking (meta-models)	✅ Full support	❌ Not supported
Nested branches	✅ Full support	❌ Not supported

The .n4a bundle stores the complete execution trace, enabling accurate replay of complex pipelines including branches and stacking configurations.

Basic Export

Export to .n4a Bundle

import nirs4all

# Train your pipeline
result = nirs4all.run(
    pipeline=[MinMaxScaler(), PLSRegression(10)],
    dataset="sample_data/regression",
)

# Export best model to .n4a bundle
result.export("exports/wheat_model.n4a")

# Export a specific prediction's model
result.export("exports/specific_model.n4a", prediction_id="abc123")

Export to Portable Script

# Export to portable Python script
result.export("exports/wheat_predictor.n4a.py", format="n4a.py")

Export Options

The result.export() method accepts the following parameters:

result.export(
    path="exports/model.n4a",          # Output path
    prediction_id=None,                 # Specific prediction (default: best)
    format="n4a",                       # Format: "n4a" or "n4a.py"
)

Parameters:

path: Path for the output bundle file
prediction_id: Export a specific prediction’s chain (default: best model)
format: Bundle format, either "n4a" (default) or "n4a.py"

Store-Level Export

For advanced export operations, use WorkspaceStore directly:

from nirs4all.pipeline.storage import WorkspaceStore

store = WorkspaceStore(workspace_path)

# Export a chain by ID
store.export_chain(chain_id, Path("model.n4a"))

# Export pipeline config as JSON
store.export_pipeline_config(pipeline_id, Path("config.json"))

# Export run metadata as YAML
store.export_run(run_id, Path("run_summary.yaml"))

# Export filtered predictions as Parquet
store.export_predictions_parquet(Path("results.parquet"), dataset_name="wheat")

Lightweight Artifact Access

For cases where you need direct access to individual artifacts from a chain:

from nirs4all.pipeline.storage import WorkspaceStore

store = WorkspaceStore(workspace_path)

# Load a specific artifact by ID
artifact = store.load_artifact(artifact_id)

# Get the filesystem path of an artifact
path = store.get_artifact_path(artifact_id)

Using Exported Bundles

Predict from .n4a Bundle

import nirs4all

# Predict directly from bundle (no workspace needed)
preds = nirs4all.predict("exports/wheat_model.n4a", new_data)

Use Portable Script

The .n4a.py script can be run standalone:

# Command line usage
python wheat_predictor.n4a.py input_spectra.csv output_predictions.csv

Or imported in Python:

# Import the script
import wheat_predictor

import numpy as np
X_new = np.load("new_spectra.npy")

# Make predictions
y_pred = wheat_predictor.predict(X_new)
print(f"Predictions: {y_pred}")

Direct BundleLoader Usage

BundleLoader is the deprecated direct-Python replay API for historical legacy .n4a bundles. New applications should use nirs4all.predict(), which routes DAG-ML/Core archives through their native replay path; those backends do not execute BundleLoader. For compatibility tooling that explicitly needs the legacy lane:

from nirs4all.pipeline.bundle import BundleLoader

loader = BundleLoader("exports/model.n4a")

# Access metadata
print(f"Pipeline UID: {loader.metadata.pipeline_uid}")
print(f"Model step: {loader.metadata.model_step_index}")
print(f"Preprocessing: {loader.metadata.preprocessing_chain}")
print(f"Fold weights: {loader.fold_weights}")

# Make predictions
import numpy as np
X_new = np.random.randn(10, 100)
y_pred = loader.predict(X_new)

Export Sources

Export works from RunResult or directly from WorkspaceStore:

# From RunResult (most common)
result.export("model.n4a")

# From store chain ID
store.export_chain("chain_abc123", Path("model.n4a"))

# From bundle path (re-export)
preds = nirs4all.predict("old_model.n4a", data)

Bundle Inspection

Check Bundle Contents

import zipfile
import json

with zipfile.ZipFile("exports/model.n4a", 'r') as zf:
    # List files
    print("Bundle contents:")
    for name in zf.namelist():
        info = zf.getinfo(name)
        print(f"  {name}: {info.file_size / 1024:.1f} KB")

    # Read manifest
    manifest = json.loads(zf.read('manifest.json'))
    print(f"\nChain ID: {manifest['chain_id']}")
    print(f"Model: {manifest['model_class']}")
    print(f"Preprocessing: {manifest['preprocessings']}")
    print(f"Exported at: {manifest['exported_at']}")

Batch Export

Export multiple models from a training run:

# Get top 3 predictions
top_preds = result.top(3)

# Export each
for i, row in enumerate(top_preds.iter_rows(named=True), 1):
    pred_id = row["prediction_id"]
    model = row["model_class"].split(".")[-1]
    result.export(f"exports/rank_{i}_{model}.n4a", prediction_id=pred_id)

Deployment Patterns

Docker Container

FROM python:3.11-slim

# Install minimal dependencies
RUN pip install numpy joblib scikit-learn scipy

# Copy portable script
COPY wheat_predictor.n4a.py /app/predict.py

WORKDIR /app
ENTRYPOINT ["python", "predict.py"]

AWS Lambda

import json
import numpy as np
import wheat_predictor  # The .n4a.py script

def lambda_handler(event, context):
    # Parse input spectra
    X = np.array(event['spectra'])

    # Make prediction
    y_pred = wheat_predictor.predict(X)

    return {
        'statusCode': 200,
        'body': json.dumps({'predictions': y_pred.tolist()})
    }

FastAPI Service

from fastapi import FastAPI
import numpy as np
import wheat_predictor

app = FastAPI()

@app.post("/predict")
async def predict(spectra: list[list[float]]):
    X = np.array(spectra)
    y_pred = wheat_predictor.predict(X)
    return {"predictions": y_pred.tolist()}

Retrain from Bundle

Bundles can be used as a starting point for retraining on new data:

import nirs4all

# Retrain from exported bundle
result = nirs4all.retrain("exports/wheat_model.n4a", new_data, mode="transfer")

Retrain modes:

'full': Train everything from scratch (same pipeline structure)
'transfer': Use existing preprocessing artifacts, train new model
'finetune': Continue training existing model (deep learning only)

Best Practices

Version bundles: Include version info in filenames
Test before deployment: Verify predictions match training validation
Document dependencies: Note required Python version and packages
Store source metadata: Keep the original prediction ID for traceability
Use compression: Enable compression for smaller bundle sizes

Troubleshooting

Bundle Too Large

Use compression: compress=True
Use export_model() for lightweight model-only export
Consider .n4a.py for simpler models

Portable Script Errors

Ensure numpy, scipy, scikit-learn, and joblib are installed
Check Python version compatibility (3.11+)
Verify input data shape matches expected features
For deep learning models: Use .n4a format instead

Prediction Mismatch

Verify same nirs4all version for export and load
Check preprocessing is correctly replayed
Compare fold weights between training and bundle