nirs4all.api.dataset_inspection module

Scientific dataset previews for thin application hosts (no jobs or stores).

CSV/Parquet assembly uses the installed native IO owner. Other formats are selected before loading through the existing library loaders; this is an explicit format route, not retry/fallback after a native error. Presentation arrays always accompany textual/numeric summaries and keep original axes.

nirs4all.api.dataset_inspection.dataset_statistics(config: dict[str, Any], *, partition: str = 'train', **load_options: Any) → dict[str, Any]

Compute historical first-source global statistics with reader evidence.

nirs4all.api.dataset_inspection.inspect_format_file(path: str, *, params: dict[str, Any] | None = None, sample_rows: int = 10, max_input_bytes: int = 536870912) → dict[str, Any]

Inspect a non-native format through its existing registered file loader.

CSV/Parquet file inspection belongs to native IO in the Rust host. No native decompression/shape limits are claimed for this explicit format lane.

nirs4all.api.dataset_inspection.load_dataset_for_analysis(config: dict[str, Any], *, load_limits: dict[str, int] | None = None, max_input_bytes: int = 536870912) → tuple[Any, dict[str, Any]]

Load one authorized explicit config and return dataset/reader evidence.

Paths are normalized but must be authorized by the host before this call. Native CSV/Parquet errors propagate; other formats are selected before loading through the existing registered library loaders. The returned dataset retains all sources, partitions, targets and metadata.

nirs4all.api.dataset_inspection.load_prediction_file(path: str | Path, *, params: dict[str, Any] | None = None, load_limits: dict[str, int] | None = None, max_input_bytes: int = 536870912) → tuple[Any, dict[str, Any], list[str] | None]

Load an authorized prediction upload through its existing reader owner.

All numeric columns remain features in file order; the first string column supplies optional display labels, never scientific sample identifiers. CSV/Parquet use an explicit IO DatasetSpec, not target-column inference. Excel uses the registered Excel loader with typed cells preserved. No training, target synthesis, parser retry or implicit legacy execution occurs.

The host must authorize the path and provide confirmed CSV parsing options (for example from IO’s bounded text detector and a header checkbox). Without overrides IO’s defaults apply, including a header and semicolon delimiter. Native limits are rejected for Excel because its loader cannot guarantee them; its separate raw-file admission budget remains effective.

nirs4all.api.dataset_inspection.preview_dataset(config: dict[str, Any], *, max_samples: int = 100, load_limits: dict[str, int] | None = None, max_input_bytes: int = 536870912, max_response_bytes: int = 33554432) → dict[str, Any]

Return partition/source-aware spectral previews and exact dataset counts.

Spectral statistics retain the historical evenly spaced display-sample semantics, now explicitly labelled. Targets use the complete partition. No feature-axis truncation, fit, server, job or workspace is introduced.