nirs4all.operators.filters.base module

Base class for sample filtering operators.

Sample filters identify samples to be excluded from training based on various criteria. They follow the sklearn TransformerMixin pattern for consistency with the existing augmentation and transformation operators in nirs4all.

class nirs4all.operators.filters.base.CompositeFilter(filters: list[SampleFilter] | None = None, mode: str = 'any', reason: str | None = None, tag_name: str | None = None)[source]

Bases: SampleFilter

Combine multiple filters with AND/OR logic.

This filter aggregates the results of multiple sub-filters using either “any” or “all” mode: - “any” (default): Exclude if ANY filter flags the sample - “all”: Exclude only if ALL filters flag the sample

Variables:

  • filters (List[SampleFilter]) – List of filter instances to combine

  • mode (str) – Combination mode - “any” or “all”

Example

>>> from nirs4all.operators.filters import YOutlierFilter, CompositeFilter
>>>
>>> # Exclude if either filter flags
>>> combined = CompositeFilter(
...     filters=[
...         YOutlierFilter(method="iqr", threshold=1.5),
...         YOutlierFilter(method="zscore", threshold=3.0),
...     ],
...     mode="any"
... )
property exclusion_reason: str

Get combined exclusion reason from all filters.

fit(X: ndarray, y: ndarray | None = None) → CompositeFilter[source]

Fit all sub-filters to the training data.

Parameters:

  • X – Feature array of shape (n_samples, n_features).

  • y – Target array of shape (n_samples,) or (n_samples, n_targets).

Returns:

The fitted composite filter.

Return type:

self

get_filter_stats(X: ndarray, y: ndarray | None = None) → dict[str, Any][source]

Get statistics including per-filter breakdown.

Parameters:

  • X – Feature array.

  • y – Target array.

Returns:

Dict with overall stats and per-filter breakdown.

get_mask(X: ndarray, y: ndarray | None = None) → ndarray[source]

Compute combined mask from all sub-filters.

Parameters:

  • X – Feature array of shape (n_samples, n_features).

  • y – Target array of shape (n_samples,) or (n_samples, n_targets).

Returns:

Boolean array where True = keep, False = exclude.

For “any” mode: keep if ALL filters say keep For “all” mode: keep if ANY filter says keep

Return type:

np.ndarray