Metrics

fyt.metrics

Metrics registry for model evaluation.

This module implements a registry pattern for computing evaluation metrics.

MetricFunction

Bases: Protocol

Protocol for metric computation functions.

__call__(y_true, y_pred, y_pred_proba)

Compute a metric.

Parameters:

Name	Type	Description	Default
y_true	Series	True labels	required
y_pred	Series	Predicted labels	required
y_pred_proba	DataFrame | None	Predicted probabilities (optional)	required

Returns:

Type	Description
MetricValue	Computed metric value

MetricsRegistry

Bases: ComponentRegistry[Callable]

Registry for metric computation functions using the Registry pattern.

compute(metric_type, y_true, y_pred, y_pred_proba=None, **kwargs) classmethod

Compute a metric using the registered function.

Parameters:

Name	Type	Description	Default
metric_type	str	Type of metric to compute (string or enum).	required
y_true	Series	True labels	required
y_pred	Series	Predicted labels	required
y_pred_proba	DataFrame | None	Predicted probabilities (optional)	None
**kwargs	Any	Extra options (e.g. average, labels). Only the options a metric function declares are forwarded, so custom metrics with the plain three-argument signature keep working.	{}

Returns:

Type	Description
MetricValue	Computed metric value

Raises:

Type	Description
ValueError	If metric type is not registered

get_available_metrics() classmethod

Get list of available registered metrics.

Returns:

Type	Description
list[str]	List of registered metric keys.

get_sklearn_scorer(metric_type) classmethod

Get the sklearn scorer name for a metric type.

Parameters:

Name	Type	Description	Default
metric_type	str	Metric type (enum or string)	required

Returns:

Type	Description
str | None	Sklearn scorer name or None if not available for cross-validation

register_sklearn_scorer(metric_type, scorer_name) classmethod

Register a custom sklearn scorer mapping.

Parameters:

Name	Type	Description	Default
metric_type	str	Metric type (enum or string)	required
scorer_name	str	Sklearn scorer name	required

compute_accuracy(y_true, y_pred, y_pred_proba=None)

Compute accuracy score.

Parameters:

Name	Type	Description	Default
y_true	Series	True labels	required
y_pred	Series	Predicted labels	required
y_pred_proba	DataFrame | None	Not used for accuracy	None

Returns:

Type	Description
float	Accuracy score

compute_balanced_accuracy(y_true, y_pred, y_pred_proba=None)

Compute balanced accuracy (mean per-class recall).

compute_confusion_matrix(y_true, y_pred, y_pred_proba=None)

Compute confusion matrix.

Parameters:

Name	Type	Description	Default
y_true	Series	True labels	required
y_pred	Series	Predicted labels	required
y_pred_proba	DataFrame | None	Not used for confusion matrix	None

Returns:

Type	Description
list[list[int]]	Confusion matrix as a 2D list

compute_f1_score(y_true, y_pred, y_pred_proba=None, average='auto')

Compute F1 score.

Parameters:

Name	Type	Description	Default
y_true	Series	True labels	required
y_pred	Series	Predicted labels	required
y_pred_proba	DataFrame | None	Not used for F1 score	None
average	str	Averaging strategy; "auto" resolves to "binary" for two-class data and "weighted" otherwise.	'auto'

Returns:

Type	Description
float	F1 score

compute_log_loss(y_true, y_pred, y_pred_proba=None, labels=None)

Compute log loss.

Parameters:

Name	Type	Description	Default
y_true	Series	True labels	required
y_pred	Series	Not used for log loss	required
y_pred_proba	DataFrame | None	Predicted probabilities (required)	None
labels	list | None	Full class list (e.g. model.classes_). Required when the evaluation slice is missing some classes, otherwise sklearn infers the classes from y_true and the column count mismatches.	None

Returns:

Type	Description
float | None	Log loss or None if computation fails

compute_mae(y_true, y_pred, y_pred_proba=None)

Compute mean absolute error.

compute_mcc(y_true, y_pred, y_pred_proba=None)

Compute Matthews correlation coefficient.

compute_mse(y_true, y_pred, y_pred_proba=None)

Compute mean squared error.

compute_pr_auc(y_true, y_pred, y_pred_proba=None, labels=None)

Compute area under the precision-recall curve (average precision).

Parameters:

Name	Type	Description	Default
y_true	Series	True labels	required
y_pred	Series	Not used for PR AUC	required
y_pred_proba	DataFrame | None	Predicted probabilities (required)	None
labels	list | None	Full class list (e.g. model.classes_); required for a correct multiclass score when the slice is missing classes.	None

Returns:

Type	Description
float | None	Average precision (binary: positive class; multiclass: weighted
float | None	one-vs-rest) or None if probabilities are unavailable.

compute_precision(y_true, y_pred, y_pred_proba=None, average='auto')

Compute precision score.

Parameters:

Name	Type	Description	Default
y_true	Series	True labels	required
y_pred	Series	Predicted labels	required
y_pred_proba	DataFrame | None	Not used for precision	None
average	str	Averaging strategy; "auto" resolves to "binary" for two-class data and "weighted" otherwise.	'auto'

Returns:

Type	Description
float	Precision score

compute_r2(y_true, y_pred, y_pred_proba=None)

Compute the coefficient of determination (R^2).

compute_recall(y_true, y_pred, y_pred_proba=None, average='auto')

Compute recall score.

Parameters:

Name	Type	Description	Default
y_true	Series	True labels	required
y_pred	Series	Predicted labels	required
y_pred_proba	DataFrame | None	Not used for recall	None
average	str	Averaging strategy; "auto" resolves to "binary" for two-class data and "weighted" otherwise.	'auto'

Returns:

Type	Description
float	Recall score

compute_rmse(y_true, y_pred, y_pred_proba=None)

Compute root mean squared error.

compute_roc_auc(y_true, y_pred, y_pred_proba=None, labels=None)

Compute ROC AUC.

Parameters:

Name	Type	Description	Default
y_true	Series	True labels	required
y_pred	Series	Not used for ROC AUC	required
y_pred_proba	DataFrame | None	Predicted probabilities (required)	None
labels	list | None	Full class list (e.g. model.classes_). Required for a correct score when the evaluation slice is missing some classes.	None

Returns:

Type	Description
float | None	ROC AUC score or None if computation is not possible

resolve_average(y_true, y_pred, requested='auto')

Resolve the averaging strategy for precision/recall/f1.

Parameters:

Name	Type	Description	Default
y_true	Series	True labels.	required
y_pred	Series	Predicted labels.	required
requested	str	Requested strategy; "auto" picks "binary" for two-class data and "weighted" otherwise.	'auto'

Returns:

Type	Description
str	A concrete sklearn average value.