Pipelines

This section covers advanced pipeline patterns and techniques.

Overview

NIRS4ALL pipelines are flexible and powerful. This section covers advanced patterns beyond basic sequential pipelines.

✍️ Writing Pipelines

Complete guide to pipeline syntax and patterns.

Start Here

Writing a Pipeline in nirs4all

🌿 Branching

Create parallel paths with different preprocessing strategies.

Parallel

Pipeline Branching

🔀 Merging

Combine branch outputs: features, predictions, or both for stacking.

Combine

Branch Merging

📚 Stacking

Build meta-models that combine predictions from multiple base models.

Ensembles

Meta-Model Stacking User Guide

🔀 Multi-Source Pipelines

Handle multiple data sources with source_branch and merge_sources.

Fusion

Multi-Source Pipelines

⚡ Cache Optimization

Speed up generator-heavy pipelines with step-level caching.

Performance

Cache Optimization

Pipeline Patterns

Sequential Pipeline

pipeline = [
    MinMaxScaler(),
    SNV(),
    KFold(n_splits=5),
    {"model": PLSRegression(10)}
]

Branching Pipeline

pipeline = [
    {"branch": [
        [SNV(), PLSRegression(10)],
        [MSC(), RandomForestRegressor()]
    ]},
    {"merge": "predictions"}  # Combine for stacking
]

Generator Pipeline

pipeline = [
    {"_or_": [SNV, MSC, Detrend]},  # Try each preprocessing
    {"_range_": [1, 20, 2], "model": PLSRegression}  # Sweep n_components
]