nirs4all.utils.memory module

Memory estimation and RSS tracking utilities.

Provides fast, numpy-first memory estimation for datasets and cache entries, plus process RSS tracking via /proc/self/statm (Linux) or psutil (macOS/Windows).

Phase 1.4 – Cache Implementation Backlog

nirs4all.utils.memory.estimate_cache_entry_bytes(entry: Any) → int

Estimate memory size of a cache entry.

Uses numpy nbytes for arrays and SpectroDatasets (O(1)). Falls back to sys.getsizeof for other types.

Parameters:

entry – The object to measure. Can be a numpy array, a SpectroDataset, a tuple/list of arrays, or any other object.

Returns:

Estimated byte count.

nirs4all.utils.memory.estimate_dataset_bytes(dataset: SpectroDataset) → int

Estimate total memory footprint of a SpectroDataset.

Sums numpy nbytes across all feature sources and target processings. O(1) per array – reads shape and dtype only, no serialization.

Parameters:

dataset – The SpectroDataset to measure.

Returns:

Estimated byte count (features + targets).

nirs4all.utils.memory.format_bytes(n: int) → str

Format a byte count as a human-readable string.

Parameters:

n – Number of bytes.

Returns:

Formatted string, e.g. “152.6 MB”, “1.2 GB”, “800 B”.

nirs4all.utils.memory.get_process_rss_mb() → float

Get the current process Resident Set Size in megabytes.

Reads /proc/self/statm on Linux (no dependencies). Falls back to psutil, then to platform-native APIs on other platforms.

Returns:

RSS in MB, or 0.0 if measurement is unavailable.