fix: resolve 10 bugs from a second library audit (#87-#96)

docs/core_concepts/configuration.md

@@ -51,6 +51,31 @@ example `{"feature_0": "rbf"}`. Check `numerical_features_` / `categorical_featu
 overrides, rather than guessing the column order.
 ```
 
+## Polars DataFrame input
+
+A `polars.DataFrame` is accepted as input to the `Preprocessor`, `RepresentationSearchCV` and
+`CrossFittedTransformer` wherever a pandas `DataFrame` is (temporal columns are not supported),
+so a scikit-learn `Pipeline` configured with `set_output(transform="polars")` can hand its
+polars output straight to the `Preprocessor`. PreTab reads the frame through pandas with the same column names, order
+and dtypes, then detects column types as usual: `Float*` columns are numerical, `Int*` /
+`UInt*` columns follow the integer cardinality rule (an integer column with nulls becomes
+float, as in pandas), and `String`, `Categorical`, `Enum` and `Boolean` columns are
+categorical. Polars nulls become missing values, so imputation and `missing_policy` treat them
+like `NaN` in a pandas frame. The conversion does not need `pyarrow`.
+
+```python
+import polars as pl
+from sklearn.impute import SimpleImputer
+from sklearn.pipeline import Pipeline
+
+X = pl.DataFrame({"age": [25.0, None, 47.0, 33.0], "income": [3.1e4, 5.2e4, None, 4.4e4]})
+y = [0.0, 1.0, 1.0, 0.0]
+
+pipe = Pipeline([("impute", SimpleImputer()), ("pretab", Preprocessor(numerical_method="minmax"))])
+pipe.set_output(transform="polars")
+pipe.fit_transform(X, y)   # a polars.DataFrame with columns num_age, num_income
+```
+
 ## Per-feature overrides
 
 Columns rarely want identical treatment. The `feature_preprocessing` dict assigns a strategy
@@ -119,7 +144,9 @@ regression and classification differently.
 | `"adaptive"` | task-dependent     | `"int"`              | -            | `True`     | `7`              | `15`             |
 
 `numerical_method` resolves to `"bspline"` when `task="regression"` (the default) and to
-`"ple"` when `task="classification"`, for every preset:
+`"ple"` when `task="classification"`, for every preset. `task` is matched exactly: any other
+value, such as `"Regression"`, raises an `InvalidParamError` from `fit` and
+`get_resolved_config()` instead of falling back to the classification preset.
 
 ```python
 standard_regression = Preprocessor(preset="standard", task="regression")
@@ -182,7 +209,8 @@ The resolution order is deterministic. Later layers win.
 
 ```{warning}
 Configuration is validated at `fit` time, not silently coerced. An invalid combination, such
-as a method that requires the target used with `target_aware=False`, raises a typed error.
+as a method that requires the target used with `target_aware=False`, or a `task` other than
+`"regression"` / `"classification"`, raises a typed error.
 This is intentional: it surfaces mistakes early rather than producing a quietly wrong
 representation.
 ```

docs/core_concepts/missing_values.md

@@ -36,6 +36,15 @@ unchanged at `transform`. PreTab never drops a row for missing values: every inp
 produces an output row.
 ```
 
+```{note}
+A column with no observed value at `fit` (for example an optional field that is empty in a
+training window or a cross-validation fold) still keeps its block, so the output layout does not
+depend on which rows were seen. The imputers fill it with `0` (or `fill_value` for
+`strategy="constant"`), which makes it a constant column, and `fit` emits a `DataWarning`
+naming it. See [Edge-case behaviour](../representations/choosing_a_method.md) for the methods
+that cannot be fitted on a constant column.
+```
+
 ## The `missing_policy` control
 
 For finer control, `missing_policy` selects one of five behaviours for the whole

docs/core_concepts/target_awareness.md

@@ -70,7 +70,14 @@ this for the training features specifically. It splits the training data into fo
 fresh copy of the transformer on all folds _except_ one, and uses that copy to transform the
 held-out fold, so every training row is transformed by a model that never saw its own target.
 `transform` on genuinely new data (a validation or test set) instead uses one model fit on all
-the training data, since there is no leakage risk there.
+the training data, since there is no leakage risk there. Both return the same kind of output,
+the one the wrapped transformer produces: a dense array of the same dtype, a sparse matrix in
+the same format, or a DataFrame.
+
+A wrapped `Preprocessor` is not refit from scratch on each fold: every fold reuses its all-data
+fit and refits only the target-aware representations. Column types, category codes,
+missing-value indicators and the blocks that do not use `y` are shared, so the out-of-fold
+features have exactly the columns and encoding `transform` produces.
 
 ```python
 import numpy as np
@@ -158,6 +165,31 @@ search.best_method_
 best_preprocessor_` and `search.best_estimator_` are refit on all the data with `bspline` and
 ready to call `.predict(X_new)`.
 
+When several rows belong to the same unit, such as repeated measurements of one patient, a
+row-level split validates each candidate on units it has already seen in training. Pass a
+group splitter as `cv` and the group labels to `fit`, so no group is on both sides of a
+train/validation split:
+
+```python
+from sklearn.model_selection import GroupKFold
+
+patient_id = np.repeat(np.arange(30), 10)  # 30 patients with 10 rows each
+
+search = RepresentationSearchCV(Ridge(), methods=["minmax", "bspline"], cv=GroupKFold(5))
+search.fit(X, y, groups=patient_id)
+```
+
+Any other keyword argument to `fit`, such as `sample_weight`, is passed to the estimator's
+`fit` on every fold (restricted to that fold's training rows) and on the final refit. The
+`Preprocessor` and the scorer do not see it, so placement and validation scores are
+unweighted.
+
+The search takes the estimator type of the estimator it wraps. Around a classifier it is a
+classifier itself and delegates `classes_`, `predict_proba`, `predict_log_proba` and
+`decision_function` (whichever the estimator provides) to the refit `best_estimator_`. It can
+therefore be evaluated as a whole in an outer, stratified cross-validation, for example with
+`cross_val_score(search, X, y_class, scoring="roc_auc")`.
+
 ```{note}
 This is deliberately narrow: it only searches the single `numerical_method` axis with one
 global method for every numerical column, not a per-column `feature_preprocessing` search.

docs/getting_started/installation.md

@@ -67,7 +67,9 @@ pip install "pretab[polars]"
 `polars` is only needed for the `set_output(transform="polars")` output path; every other
 output (`output_structure="matrix"`/`"blocks"`, `output_format="dense"`/`"sparse"`,
 `set_output(transform="pandas")`) works without it. Requesting `"polars"` output without the
-extra installed raises a clear `OptionalDependencyError`.
+extra installed raises a clear `OptionalDependencyError`. A `polars.DataFrame` is also accepted
+as input to `fit` / `transform` (see
+[Polars DataFrame input](../core_concepts/configuration.md#polars-dataframe-input)).
 ```
 
 Use the convenience `all` extra to install every optional dependency at once:

docs/homepage.md

@@ -34,8 +34,8 @@ compose standalone representations with scikit-learn transformers and estimators
 :::
 
 :::{grid-item-card} 🧠 Smart defaults
-Automatic feature-type detection (numerical vs. categorical) with support for both
-`pandas.DataFrame` and `numpy.ndarray` inputs.
+Automatic feature-type detection (numerical vs. categorical) with support for
+`pandas.DataFrame`, `polars.DataFrame` and `numpy.ndarray` inputs.
 :::
 
 ::::

docs/representations/choosing_a_method.md

@@ -98,6 +98,15 @@ PreTab is explicit about degenerate inputs rather than failing silently.
   row for it.
 - **NaN into a finite-only method**: raises a typed error unless imputation is configured. See
   [Missing values](../core_concepts/missing_values.md).
+- **Column with no observed value at fit** (missing on every row, e.g. an optional field that is
+  empty in a training window or a cross-validation fold): `Preprocessor` keeps the column's
+  block and emits a `DataWarning` naming it. The imputers fill it with `0` (or `fill_value`
+  for `strategy="constant"`), so it is fitted as a constant column; a method that cannot be
+  fitted on a constant column (the splines above, or Box-Cox) raises a typed error naming the
+  column. With imputation disabled the column stays missing: the scalers, `"quantile"` and
+  `"none"` keep its block as missing values, and every other method raises the same typed
+  error. `policy={"constant": "error"}` rejects an empty numerical column like any other
+  constant one.
 
 ## Non-goals
 

docs/tutorials/custom_representation.md

@@ -71,7 +71,9 @@ The four class attributes are the declarative contract.
 
 `supervision`
 : `"unsupervised"`, `"optional"` (uses `y` only when `target_aware=True`), or `"supervised"`
-(always needs `y`).
+(always needs `y`). An `"optional"` class must take a `target_aware` constructor parameter:
+`Preprocessor` sets it from its own `target_aware`, and registration rejects a class without
+one.
 
 ```{tip}
 Implement `_output_sizes` to return the number of output columns each input contributes. The

pretab/compose/config.py

@@ -3,10 +3,10 @@
 :class:`PreprocessorConfig` is the frozen, canonical view of the user-supplied
 Preprocessor parameters. It normalizes the global method names (resolving
 aliases and separator/case variants, mapping ``None`` to ``"none"``) and
-validates the global ``target_aware`` / ``placement_strategy`` contract up front.
-The user's original constructor arguments stay untouched on the estimator (as
-scikit-learn requires); this object is the internal, normalized counterpart the
-composition layer consumes.
+validates the global ``target_aware`` / ``placement_strategy`` contract and the
+``task`` up front. The user's original constructor arguments stay untouched on
+the estimator (as scikit-learn requires); this object is the internal, normalized
+counterpart the composition layer consumes.
 
 Per-column overrides in ``feature_preprocessing`` are kept verbatim because the
 namespace they resolve in (numerical vs categorical) depends on the column type,
@@ -18,7 +18,7 @@
 
 from dataclasses import dataclass
 
-from ..core.parameters import validate_placement
+from ..core.parameters import validate_placement, validate_task
 from ..exceptions import invalid_param_error
 from .registry import (
     CATEGORICAL_ALIASES,
@@ -47,8 +47,8 @@ class PreprocessorConfig:
     """Frozen, normalized configuration derived from Preprocessor parameters.
 
     Built via :meth:`from_params`, which normalizes the global method names and
-    validates the placement contract. All other knobs are carried through as-is
-    for the factory and orchestration layers.
+    validates the placement contract and the task. All other knobs are carried
+    through as-is for the factory and orchestration layers.
     """
 
     numerical_method: str
@@ -103,9 +103,12 @@ def from_params(
         ------
         InvalidParamError
             If the ``target_aware`` / ``placement_strategy`` combination is
-            invalid (via :func:`~pretab.core.parameters.validate_placement`).
+            invalid (via :func:`~pretab.core.parameters.validate_placement`), or
+            ``task`` is not ``"regression"`` / ``"classification"`` (via
+            :func:`~pretab.core.parameters.validate_task`).
         """
         validate_placement(target_aware, placement_strategy)
+        validate_task(task, "Preprocessor")
         if missing_policy is not None and missing_policy not in MISSING_POLICIES:
             raise invalid_param_error(
                 "Preprocessor",

pretab/compose/factory.py

@@ -69,6 +69,18 @@ def _filter_kwargs(allowed, kwargs):
     return {key: kwargs[key] for key in allowed if key in kwargs}
 
 
+def _imputer(strategy, add_indicator, imputer_kwargs):
+    """Build the leading :class:`~sklearn.impute.SimpleImputer` of a per-column pipeline.
+
+    A column with no observed value at fit is kept (``keep_empty_features=True``)
+    and filled with ``0``, or with ``fill_value`` for ``strategy="constant"``.
+    scikit-learn's default drops it, which leaves the rest of the pipeline an
+    input without columns: the block then fails to fit or vanishes from the output.
+    """
+    imputer_kwargs = {"keep_empty_features": True, **(imputer_kwargs or {})}
+    return SimpleImputer(strategy=strategy, add_indicator=add_indicator, **imputer_kwargs)
+
+
 def _clamp_spline_basis(output_dim):
     """Clamp a requested output dimension into the supported B/M/I spline range.
 
@@ -90,23 +102,24 @@ def _placement_kwargs(spec: TransformerSpec, kwargs):
     """Return the placement kwargs to inject for a method, honouring its capability.
 
     Mirrors the shared-placement contract: methods with optional target awareness
-    (feature maps and freely-placed knot splines) receive ``target_aware`` plus
-    the ``placement_strategy`` (when set); the always-target-aware ``ple`` receives
-    a supervised ``placement_strategy`` only when target-aware; the unsupervised-only
-    penalized splines receive an unsupervised ``placement_strategy`` only when not
-    target-aware. Methods without data-driven placement receive nothing.
+    (feature maps, freely-placed knot splines, and registered ``"optional"``
+    representations) always receive ``target_aware``, which decides whether they
+    use ``y`` at all, plus the ``placement_strategy`` (when set) if they declare
+    placement strategies; the always-target-aware ``ple`` receives a supervised
+    ``placement_strategy`` only when target-aware; the unsupervised-only penalized
+    splines receive an unsupervised ``placement_strategy`` only when not
+    target-aware. Other methods without data-driven placement receive nothing.
     """
-    if not spec.placement_strategies:
-        return {}
-
     target_aware = bool(kwargs.get("target_aware", False))
     placement_strategy = kwargs.get("placement_strategy")
 
     if spec.target_usage == "optional":
         out = {"target_aware": target_aware}
-        if placement_strategy is not None:
+        if placement_strategy is not None and spec.placement_strategies:
             out["placement_strategy"] = placement_strategy
         return out
+    if not spec.placement_strategies:
+        return {}
     if spec.target_usage == "required":
         if target_aware and placement_strategy in ("cart", "lightgbm"):
             return {"placement_strategy": placement_strategy}
@@ -131,10 +144,7 @@ def get_numerical_transformer_steps(
     steps = []
 
     if add_imputer:
-        imputer_kwargs = imputer_kwargs or {}
-        steps.append(
-            ("imputer", SimpleImputer(strategy=imputer_strategy, add_indicator=add_missing_indicator, **imputer_kwargs))
-        )
+        steps.append(("imputer", _imputer(imputer_strategy, add_missing_indicator, imputer_kwargs)))
 
     # Optional scaling step, added only when it is not already the chosen method.
     scalers = {
@@ -208,10 +218,7 @@ def get_categorical_transformer_steps(
     steps = []
 
     if add_imputer:
-        imputer_kwargs = imputer_kwargs or {}
-        steps.append(
-            ("imputer", SimpleImputer(strategy=imputer_strategy, add_indicator=add_missing_indicator, **imputer_kwargs))
-        )
+        steps.append(("imputer", _imputer(imputer_strategy, add_missing_indicator, imputer_kwargs)))
 
     if method not in CATEGORICAL_METHODS:
         raise invalid_param_error(

pretab/compose/feature_detection.py

@@ -1,4 +1,4 @@
-"""Coerce inputs to DataFrames and classify columns as numerical or categorical.
+"""Coerce inputs to pandas DataFrames and classify columns as numerical or categorical.
 
 Feature-type detection decides which construction path each column takes. It is
 kept here, separate from orchestration, so the Preprocessor's ``fit`` reads as a
@@ -10,27 +10,60 @@
 import numpy as np
 import pandas as pd
 
+from ..core.validation import is_polars_frame, polars_to_pandas
 from ..exceptions import PretabDataError, invalid_param_error
 
-__all__ = ["bool_columns_as_object", "detect_column_types", "to_dataframe", "with_string_labels"]
+__all__ = [
+    "bool_columns_as_object",
+    "detect_column_types",
+    "has_column_labels",
+    "to_dataframe",
+    "with_string_labels",
+]
+
+
+def has_column_labels(X) -> bool:
+    """Return True if ``X`` carries its own column labels.
+
+    DataFrames (pandas or polars) and dicts of columns are matched by label;
+    anything else (a NumPy array, a list of rows, ...) is matched by position.
+    """
+    return isinstance(X, dict) or hasattr(X, "columns")
 
 
 def to_dataframe(X, *, copy: bool = False) -> pd.DataFrame:
-    """Return ``X`` as a DataFrame, naming array columns ``feature_0``, ``feature_1`` ....
+    """Return ``X`` as a pandas DataFrame, naming array columns ``feature_0``, ``feature_1`` ....
 
-    Dicts and NumPy arrays are wrapped in a fresh DataFrame; an existing
-    DataFrame is returned as-is, or copied when ``copy`` is True.
+    Dicts are wrapped in a fresh DataFrame and a polars DataFrame is converted to
+    one with the same column names, order and (as far as possible) dtypes. Input
+    without column labels -- a NumPy array, a list of rows -- is read as a 2D
+    array. An existing pandas DataFrame is returned as-is, or copied when
+    ``copy`` is True.
 
     Raises
     ------
     PretabDataError
-        If the resulting columns contain a duplicate label. A
+        If ``X`` is a pandas Series or array-like input that is not 2D, or if the
+        resulting columns contain a duplicate label. A
         :class:`~sklearn.compose.ColumnTransformer` keys its per-column steps by
         name, so a duplicate cannot be routed unambiguously.
     """
+    if isinstance(X, pd.Series):
+        raise PretabDataError(
+            "Expected 2D input (a DataFrame or a 2D array), got a pandas Series.\n"
+            "Fix: pass X.to_frame() to use the Series as a single column."
+        )
     if isinstance(X, dict):
         X = pd.DataFrame(X)
-    elif isinstance(X, np.ndarray):
+    elif is_polars_frame(X):
+        X = polars_to_pandas(X)
+    elif not has_column_labels(X):
+        X = np.asarray(X)
+        if X.ndim != 2:
+            raise PretabDataError(
+                f"Expected 2D input (a DataFrame or a 2D array), got an array with {X.ndim} dimension(s).\n"
+                "Fix: reshape a single feature to one column, e.g. X.reshape(-1, 1)."
+            )
         X = pd.DataFrame(X, columns=pd.Index([f"feature_{i}" for i in range(X.shape[1])]))
     else:
         X = X.copy() if copy else X