nirs4all.pipeline.storage.artifacts.utils module
Utility functions for artifact identification and path handling (V3).
This module provides utility functions for the V3 artifact system, primarily for file path handling, content hashing, and artifact ID utilities.
For the core V3 artifact ID functions (compute_chain_hash, generate_artifact_id_v3, parse_artifact_id_v3, is_v3_artifact_id), use the operator_chain module directly.
- V3 Artifact ID Format:
“{pipeline_id}${chain_hash}:{fold_id}”
Examples
“0001_pls$a1b2c3d4e5f6:all” - Shared artifact
“0001_pls$7f8e9d0c1b2a:0” - Fold 0 artifact
“0001_pls$3c4d5e6f7a8b:1” - Fold 1 artifact
- class nirs4all.pipeline.storage.artifacts.utils.ExecutionPath(pipeline_id: str, chain_path: str = '', branch_path: list[int] | None = None, step_index: int = 0, source_index: int | None = None, fold_id: int | None = None, substep_index: int | None = None)[source]
Bases:
objectRepresents the execution context for an artifact (V3).
Captures all context needed to uniquely identify an artifact within a pipeline execution.
- Variables:
pipeline_id (str) – Pipeline identifier (e.g., “0001_pls_abc123”)
chain_path (str) – Full operator chain path string
branch_path (list[int] | None) – List of branch indices for nested branching
step_index (int) – Logical step number within current branch
source_index (int | None) – Multi-source index (None for single source)
fold_id (int | None) – CV fold identifier (None for shared artifacts)
substep_index (int | None) – Substep index (for [model1, model2])
- classmethod from_artifact_id_v3(artifact_id: str, chain_path: str = '') ExecutionPath[source]
Create ExecutionPath from V3 artifact ID string.
- Parameters:
artifact_id – V3 artifact ID to parse
chain_path – Full chain path (required for complete reconstruction)
- Returns:
ExecutionPath instance
- nirs4all.pipeline.storage.artifacts.utils.artifact_id_matches_context(artifact_id: str, pipeline_id: str | None = None, branch_path: list[int] | None = None, step_index: int | None = None, fold_id: int | None = None) bool[source]
Check if a V3 artifact ID matches a given context.
Partial matching is supported - only specified parameters are checked. Note: branch_path and step_index matching requires ArtifactRecord access.
- Parameters:
artifact_id – V3 artifact ID to check
pipeline_id – Expected pipeline ID (None = don’t check)
branch_path – Expected branch path (ignored for V3 - use ArtifactRecord)
step_index – Expected step index (ignored for V3 - use ArtifactRecord)
fold_id – Expected fold ID (None = don’t check)
- Returns:
True if artifact matches specified criteria, False otherwise
- nirs4all.pipeline.storage.artifacts.utils.extract_fold_id_from_artifact_id(artifact_id: str) int | None[source]
Extract fold ID from artifact ID (V2 or V3).
- Parameters:
artifact_id – Full artifact ID
- Returns:
Fold ID or None if “all”
- nirs4all.pipeline.storage.artifacts.utils.extract_pipeline_id_from_artifact_id(artifact_id: str) str[source]
Extract pipeline ID from artifact ID (V2 or V3).
- Parameters:
artifact_id – Full artifact ID
- Returns:
Pipeline ID component
- nirs4all.pipeline.storage.artifacts.utils.parse_artifact_id(artifact_id: str) tuple[str, list[int], int, int | None, int | None][source]
Parse an artifact ID into its components (V3 only).
V3 format: {pipeline_id}${chain_hash}:{fold_id}
- Parameters:
artifact_id – V3 artifact ID to parse
- Returns:
Tuple of (pipeline_id, branch_path, step_index, fold_id, sub_index) For V3: step_index will be 0, branch_path empty (use ArtifactRecord for full info)
- Raises:
ValueError – If artifact ID format is not V3
- nirs4all.pipeline.storage.artifacts.utils.parse_filename(filename: str) tuple[str, str, str] | None[source]
Parse artifact filename into components.
Handles new format: <type>_<class>_<short_hash>.<ext> Also handles legacy format: <class>_<short_hash>.<ext>
- Parameters:
filename – Filename to parse
- Returns:
Tuple of (artifact_type, class_name, short_hash) or None if invalid