[dependabot]

Updates the requirements on EvoTrees to permit the latest version.

Release notes

Sourced from EvoTrees's releases.

v0.20.0

EvoTrees v0.20.0

Diff since v0.19.0

Breaking changes

Model structure

• The intercept is stored on EvoTree as bias::Vector{Float32} (unconstrained / link space). trees holds only boosting rounds, so length(m.trees) == nrounds * bagging_size (and is empty when nrounds = 0).
• predict always applies bias, then the first ntree_limit trees. ntree_limit=0 is bias only.
• treeplot defaults to n=1 (the first learned tree).
• max_depth is the maximum number of splits on a leaf path (1 is a split with 2 leaves).

Plotting

• Tree plots now use a Makie recipe instead of Plots.jl / RecipesBase. Load a backend (CairoMakie or GLMakie) and call treeplot(model) or plot(model, 1).
• RecipesBase and NetworkLayout are no longer dependencies. Layout is computed in EvoTrees; the recipe is a package extension loaded with Makie.
• Split labels show the comparison (≤ for numeric/ordered features, = for unordered categoricals). Multi-output leaves show every predicted parameter.

Merged pull requests:

• Dev (#370) (@​jeremiedb)
• Check the column count of a matrix against the model's features (#371) (@​AJ0070)
• Validate eval data the way training data is validated (#372) (@​AJ0070)
• up (#373) (@​jeremiedb)
• Ranking: group support, per-query NDCG and a LambdaRank objective (#375) (@​AJ0070)
• Reuse train bins for eval and binarize GPU matrices on device. (#380) (@​AdityaPandeyCN)
• Jdb/groups (#382) (@​jeremiedb)
• Accumulate every tree of a round in the eval callback (#383) (@​AJ0070)
• Jdb/v20 (#384) (@​jeremiedb)
• Reject negative targets for Tweedie and Poisson (#385) (@​AJ0070)
• Score each target against its own location in the grouped corr metric (#386) (@​AJ0070)
• Load models saved before the intercept moved out of the first tree (#387) (@​AJ0070)
• Reject a non-finite target instead of returning an all-NaN model (#392) (@​AJ0070)
• Jdb/v20 (#394) (@​jeremiedb)
• Jdb/tmp (#395) (@​jeremiedb)
• Jdb/tmp (#396) (@​jeremiedb)
• Count each oblivious depth's split gain once in feature importance (#399) (@​AJ0070)
• Drop count_nodes_kernel!, pick sibling by weight sum from nodes_sum (#403) (@​AdityaPandeyCN)
• Reject GPU splits whose side weight equals min_weight, matching CPU (#405) (@​AdityaPandeyCN)
• Update Random requirement from 1 to <0.0.1, 1 (#411) (@​dependabot[bot])
• Let load read the IO stream that save writes to (#414) (@​AJ0070)
• Make :ndcg and :lambdarank independent of row order within a group (#416) (@​AJ0070)
• Update KernelAbstractions requirement from 0.9 to 0.9, 0.10 (#418) (@​dependabot[bot])
• Dev (#419) (@​jeremiedb)
• Route categorical splits in shap as predict does (#420) (@​AJ0070)
• Skip lambdarank pairs that both rank below ndcg_k (#421) (@​AJ0070)

Closed issues:

• Unexpected behaviour in learning curves and tuning of hyperparameters (#344)
• MLJ update continues boosting when hyperparameters other than nrounds change (#345)

Changelog

Sourced from EvoTrees's changelog.

v0.20.0

Decorrelation penalty

• ctrl_lambda penalises the dependence between the predictions and one or more control variables, measured by distance covariance, which also captures nonlinear dependence. Pass the controls to EvoTrees.fit with ctrl_name on a table or ctrl_train alongside x_train, and their relative weights with ctrl_weights.
• ctrl_within_group = true computes the penalty within each group of group_name / group_train.
• Available for :mse, :logloss, :poisson, :gamma, :tweedie and :gaussian_mle (on the location), on CPU and GPU.
• With evaluation data, the loss's own metric includes the penalty, so early stopping follows the penalised objective. The evaluation data then needs the same controls.
• ctrl_lambda scales with the target. Past some ctrl_lambda * eta the fit oscillates between rounds, so a larger weight needs a smaller eta.

config = EvoTreeRegressor(; loss=:mse, ctrl_lambda=10.0, ctrl_within_group=true)
m = fit(config, dtrain; target_name="y", group_name="group", ctrl_name=["c1", "c2"])

Model structure

• The intercept is stored on EvoTree as bias::Vector{Float32} (unconstrained / link space). trees holds only boosting rounds, so length(m.trees) == nrounds * bagging_size (and is empty when nrounds = 0).
• predict always applies bias, then the first ntree_limit trees. ntree_limit=0 is bias only.
• treeplot defaults to n=1 (the first learned tree).
• max_depth is the maximum number of splits on a leaf path (1 is a split with 2 leaves).

Plotting

• Tree plots now use a Makie recipe instead of Plots.jl / RecipesBase. Load a backend (CairoMakie or GLMakie) and call treeplot(model) or plot(model, 1).
• RecipesBase and NetworkLayout are no longer dependencies. Layout is computed in EvoTrees; the recipe is a package extension loaded with Makie.
• Split labels show the comparison (≤ for numeric/ordered features, = for unordered categoricals). Multi-output leaves show every predicted parameter.

XE-NDCG ranking loss

• loss=:xendcg: the XE-NDCG listwise ranking loss (Bruch, WWW 2021), linear in the size of each group. metric defaults to :xendcg, the same loss on the evaluation groups.
• :lambdarank and :xendcg reject a multi-column target, which :lambdarank used to fit on the wrong values.

Pearson loss

• loss=:pearson fits the Pearson correlation between the predictions and the target within each group of group_name / group_train, so it is only available through EvoTrees.fit. metric defaults to :pearson.

v0.18.8

Saved :gaussian_mle / :logistic_mle models from ≤0.18.7 remain compatible: the unconstrained scale is still exp(x).

Behavioral changes

• Multiclass Hessian. :mlogloss now uses the softmax diagonal p(1-p). Retraining a classifier can yield different trees; already-saved models still predict as before.
• Invalid input now errors instead of logging and continuing. In particular: a single-level classification target, a non-positive target under :gamma, an empty row subsample, and L2 / bagging_size / early_stopping_rounds outside their valid ranges.
• Classification eval labels are encoded against the training levels. y_eval must only contain classes seen in y_train; a different level order no longer silently scores the wrong prediction columns.
• MLJ update continues an existing fit only when nrounds is the sole hyper-parameter change (and is not reduced). Any other change triggers a full refit.

MLE losses: Fisher information

• :gaussian_mle and :logistic_mle now take Newton steps with the Fisher information (expected Hessian) rather than the observed Hessian. The Fisher matrix is diagonal and positive definite for these location-scale families, which stabilizes split finding and leaf weights.
• The unconstrained scale parameter remains exp(x). predict still returns the positive scale (σ / s). User-facing offsets remain the positive scale and are mapped internally with log.

Multi-target regression

• Several losses can now be fit jointly on a vector of targets, with shared tree structure and a vector-valued leaf. Pass a matrix y_train of size (nobs, n_targets), matching x_train's (nobs, nfeats), or target_name=["y1", "y2", ...] on a table.
• Supported: :mse, :logloss, :poisson, :gamma, :tweedie, :mae, :quantile, :cred_std, :cred_var, :gaussian_mle, :logistic_mle. Not supported: :mlogloss, :multiquantile.
• Predictions are (nobs, n_targets). MLE losses return (nobs, 2 * n_targets) with interleaved location and positive scale per target.

... (truncated)

Commits

• 6fefbe5 bench
• 1172b62 Skip lambdarank pairs below the ndcg_k cutoff, whose NDCG change is zero (#421)
• 8cabf90 Route categorical splits in shap as predict does (#420)
• 37d3ac0 Make :ndcg and :lambdarank independent of row order within a group (#416)
• 919a666 Update KernelAbstractions requirement from 0.9 to 0.9, 0.10 (#418)
• e0ecaee Dev (#419)
• a754959 CUDA dep
• 3101c34 Update Random requirement from 1 to <0.0.1, 1 (#411)
• 284c552 Let load read the IO stream that save writes to (#414)
• f82894c Drop count_nodes_kernel!, pick sibling by weight sum from nodes_sum (#403)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)