nirs4all.controllers.data.branch module

Unified Branch Controller for pipeline branching.

This controller enables two types of branching:

  1. Duplication branches (original behavior): - All samples processed by all branches - Each branch applies different preprocessing - Used for comparing preprocessing strategies

  2. Separation branches (new in v2): - Samples partitioned into non-overlapping subsets - Each sample processed by exactly one branch - Used for per-group models (by tag, metadata, source, or filter)

Syntax detection:

  • List syntax: {“branch”: [[A], [B]]} -> duplication branches

  • Dict with by_* keys: {“branch”: {“by_tag”: “outlier”}} -> separation branches

Examples

>>> # Duplication branch: compare SNV vs MSC
>>> pipeline = [
...     {"branch": [
...         [SNV(), PCA(n_components=10)],
...         [MSC(), FirstDerivative()],
...     ]},
...     PLSRegression(n_components=5),  # Runs on BOTH branches
... ]
>>>
>>> # Separation branch by tag: different models for outliers vs inliers
>>> pipeline = [
...     {"tag": YOutlierFilter()},  # Creates 'y_outlier_iqr' tag
...     {"branch": {
...         "by_tag": "y_outlier_iqr",
...         "values": {
...             "outliers": True,
...             "inliers": False,
...         },
...         "steps": [PLS(n_components=5)],
...     }},
... ]
>>>
>>> # Separation branch by metadata: per-site models
>>> pipeline = [
...     {"branch": {
...         "by_metadata": "site",
...         "steps": [SNV(), PLS(10)],
...     }},
...     {"merge": "concat"},  # Reassemble predictions
... ]
>>>
>>> # Separation branch by source: per-source preprocessing
>>> pipeline = [
...     {"branch": {
...         "by_source": True,
...         "steps": {
...             "NIR": [SNV(), SavitzkyGolay()],
...             "markers": [VarianceThreshold()],
...         },
...     }},
...     {"merge": {"sources": "concat"}},
... ]
class nirs4all.controllers.data.branch.BranchController[source]

Bases: OperatorController

Unified controller for pipeline branching.

Implements both duplication and separation branching mechanisms:

Duplication branches (all samples to all branches):

  • Creates independent context copies for each branch

  • Each branch sees all samples

  • Executes branch steps sequentially within each branch

  • Post-branch steps iterate over all branch contexts

Separation branches (samples partitioned):

  • Partitions samples into non-overlapping subsets

  • Each sample goes to exactly one branch

  • Supports by_tag, by_metadata, by_filter, by_source modes

  • Use {“merge”: “concat”} to reassemble

Variables:

priority (int) – Controller priority (5 = high, executes early).

execute(step_info: ParsedStep, dataset: SpectroDataset, context: ExecutionContext, runtime_context: RuntimeContext, source: int = -1, mode: str = 'train', loaded_binaries: list[tuple[str, Any]] | None = None, prediction_store: Any | None = None) → tuple[ExecutionContext, StepOutput][source]

Execute the branch step.

Detects branch mode (duplication vs separation) and delegates to the appropriate execution method.

Parameters:

  • step_info – Parsed step containing branch definitions

  • dataset – Dataset to operate on

  • context – Pipeline execution context

  • runtime_context – Runtime infrastructure context

  • source – Data source index

  • mode – Execution mode (“train” or “predict”)

  • loaded_binaries – Pre-loaded binary objects for prediction mode

  • prediction_store – External prediction store for model predictions

Returns:

Tuple of (updated_context, StepOutput with collected artifacts)

classmethod matches(step: Any, operator: Any, keyword: str) → bool[source]

Check if the step matches the branch controller.

Parameters:

  • step – Original step configuration

  • operator – Deserialized operator

  • keyword – Step keyword

Returns:

True if keyword is “branch”

priority: int = 5
classmethod supports_prediction_mode() → bool[source]

Branch controller executes in prediction mode to reconstruct branches.

classmethod use_multi_source() → bool[source]

Branch controller supports multi-source datasets.