functional_metrics¶

Functions

`_get_axis_by_multioutput`(multioutput)
`mape`(y_true, y_pred[, eps, multioutput])	Mean absolute percentage error.
`max_deviation`(y_true, y_pred[, multioutput])	Max Deviation metric.
`sign`(y_true, y_pred[, multioutput])	Sign error metric.
`smape`(y_true, y_pred[, eps, multioutput])	Symmetric mean absolute percentage error.
`wape`(y_true, y_pred[, multioutput])	Weighted average percentage Error metric.

Classes

FunctionalMetricMultioutput(value)

Enum for different functional metric multioutput modes.

mae(y_true, y_pred, *, sample_weight=None, multioutput='uniform_average')¶

Mean absolute error regression loss.

Read more in the User Guide.

Parameters

y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Ground truth (correct) target values.
y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Estimated target values.
sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.
multioutput ({'raw_values', 'uniform_average'} or array-like of shape (n_outputs,), default='uniform_average') –
Defines aggregating of multiple output values. Array-like value defines weights used to average errors.

’raw_values’ :
Returns a full set of errors in case of multioutput input.

’uniform_average’ :
Errors of all outputs are averaged with uniform weight.

Returns

loss – If multioutput is ‘raw_values’, then mean absolute error is returned for each output separately. If multioutput is ‘uniform_average’ or an ndarray of weights, then the weighted average of all output errors is returned.

MAE output is non-negative floating point. The best value is 0.0.

Return type

float or ndarray of floats

Examples

>>> from sklearn.metrics import mean_absolute_error
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> mean_absolute_error(y_true, y_pred)
0.5
>>> y_true = [[0.5, 1], [-1, 1], [7, -6]]
>>> y_pred = [[0, 2], [-1, 2], [8, -5]]
>>> mean_absolute_error(y_true, y_pred)
0.75
>>> mean_absolute_error(y_true, y_pred, multioutput='raw_values')
array([0.5, 1. ])
>>> mean_absolute_error(y_true, y_pred, multioutput=[0.3, 0.7])
0.85...

mape(y_true: Union[float, Sequence[float], Sequence[Sequence[float]]], y_pred: Union[float, Sequence[float], Sequence[Sequence[float]]], eps: float = 1e-15, multioutput: str = 'joint') → Union[float, Sequence[float], Sequence[Sequence[float]]][source]¶

Mean absolute percentage error.

Wikipedia entry on the Mean absolute percentage error

Parameters

y_true (Union[float, Sequence[float], Sequence[Sequence[float]]]) –
array-like of shape (n_samples,) or (n_samples, n_outputs)

Ground truth (correct) target values.
y_pred (Union[float, Sequence[float], Sequence[Sequence[float]]]) –
array-like of shape (n_samples,) or (n_samples, n_outputs)

Estimated target values.
eps (float) – MAPE is undefined for y_true[i]==0 for any i, so all zeros y_true[i] are clipped to max(eps, abs(y_true)).
multioutput (str) – Defines aggregating of multiple output values (see FunctionalMetricMultioutput).

Returns

A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.

Return type

Union[float, Sequence[float], Sequence[Sequence[float]]]

max_deviation(y_true: Union[float, Sequence[float], Sequence[Sequence[float]]], y_pred: Union[float, Sequence[float], Sequence[Sequence[float]]], multioutput: str = 'joint') → Union[float, Sequence[float], Sequence[Sequence[float]]][source]¶

Max Deviation metric.

Parameters

y_true (Union[float, Sequence[float], Sequence[Sequence[float]]]) –
array-like of shape (n_samples,) or (n_samples, n_outputs)

Ground truth (correct) target values.
y_pred (Union[float, Sequence[float], Sequence[Sequence[float]]]) –
array-like of shape (n_samples,) or (n_samples, n_outputs)

Estimated target values.
multioutput (str) – Defines aggregating of multiple output values (see FunctionalMetricMultioutput).

Returns

A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.

Return type

Union[float, Sequence[float], Sequence[Sequence[float]]]

medae(y_true, y_pred, *, multioutput='uniform_average', sample_weight=None)¶

Median absolute error regression loss.

Median absolute error output is non-negative floating point. The best value is 0.0. Read more in the User Guide.

Parameters

y_true (array-like of shape = (n_samples) or (n_samples, n_outputs)) – Ground truth (correct) target values.
y_pred (array-like of shape = (n_samples) or (n_samples, n_outputs)) – Estimated target values.
multioutput ({'raw_values', 'uniform_average'} or array-like of shape (n_outputs,), default='uniform_average') –
Defines aggregating of multiple output values. Array-like value defines weights used to average errors.

’raw_values’ :
Returns a full set of errors in case of multioutput input.

’uniform_average’ :
Errors of all outputs are averaged with uniform weight.
sample_weight (array-like of shape (n_samples,), default=None) –
Sample weights.

New in version 0.24.

Returns

loss – If multioutput is ‘raw_values’, then mean absolute error is returned for each output separately. If multioutput is ‘uniform_average’ or an ndarray of weights, then the weighted average of all output errors is returned.

Return type

float or ndarray of floats

Examples

>>> from sklearn.metrics import median_absolute_error
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> median_absolute_error(y_true, y_pred)
0.5
>>> y_true = [[0.5, 1], [-1, 1], [7, -6]]
>>> y_pred = [[0, 2], [-1, 2], [8, -5]]
>>> median_absolute_error(y_true, y_pred)
0.75
>>> median_absolute_error(y_true, y_pred, multioutput='raw_values')
array([0.5, 1. ])
>>> median_absolute_error(y_true, y_pred, multioutput=[0.3, 0.7])
0.85

mse(y_true, y_pred, *, sample_weight=None, multioutput='uniform_average', squared=True)¶

Mean squared error regression loss.

Read more in the User Guide.

Parameters

y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Ground truth (correct) target values.
y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Estimated target values.
sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.
multioutput ({'raw_values', 'uniform_average'} or array-like of shape (n_outputs,), default='uniform_average') –
Defines aggregating of multiple output values. Array-like value defines weights used to average errors.

’raw_values’ :
Returns a full set of errors in case of multioutput input.

’uniform_average’ :
Errors of all outputs are averaged with uniform weight.
squared (bool, default=True) – If True returns MSE value, if False returns RMSE value.

Returns

loss – A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.

Return type

float or ndarray of floats

Examples

>>> from sklearn.metrics import mean_squared_error
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> mean_squared_error(y_true, y_pred)
0.375
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> mean_squared_error(y_true, y_pred, squared=False)
0.612...
>>> y_true = [[0.5, 1],[-1, 1],[7, -6]]
>>> y_pred = [[0, 2],[-1, 2],[8, -5]]
>>> mean_squared_error(y_true, y_pred)
0.708...
>>> mean_squared_error(y_true, y_pred, squared=False)
0.822...
>>> mean_squared_error(y_true, y_pred, multioutput='raw_values')
array([0.41666667, 1.        ])
>>> mean_squared_error(y_true, y_pred, multioutput=[0.3, 0.7])
0.825...

msle(y_true, y_pred, *, sample_weight=None, multioutput='uniform_average', squared=True)¶

Mean squared logarithmic error regression loss.

Read more in the User Guide.

Parameters

y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Ground truth (correct) target values.
y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Estimated target values.
sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.
multioutput ({'raw_values', 'uniform_average'} or array-like of shape (n_outputs,), default='uniform_average') –
Defines aggregating of multiple output values. Array-like value defines weights used to average errors.

’raw_values’ :
Returns a full set of errors when the input is of multioutput format.

’uniform_average’ :
Errors of all outputs are averaged with uniform weight.
squared (bool, default=True) – If True returns MSLE (mean squared log error) value. If False returns RMSLE (root mean squared log error) value.

Returns

loss – A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.

Return type

float or ndarray of floats

Examples

>>> from sklearn.metrics import mean_squared_log_error
>>> y_true = [3, 5, 2.5, 7]
>>> y_pred = [2.5, 5, 4, 8]
>>> mean_squared_log_error(y_true, y_pred)
0.039...
>>> mean_squared_log_error(y_true, y_pred, squared=False)
0.199...
>>> y_true = [[0.5, 1], [1, 2], [7, 6]]
>>> y_pred = [[0.5, 2], [1, 2.5], [8, 8]]
>>> mean_squared_log_error(y_true, y_pred)
0.044...
>>> mean_squared_log_error(y_true, y_pred, multioutput='raw_values')
array([0.00462428, 0.08377444])
>>> mean_squared_log_error(y_true, y_pred, multioutput=[0.3, 0.7])
0.060...

r2_score(y_true, y_pred, *, sample_weight=None, multioutput='uniform_average')[source]¶

\(R^2\) (coefficient of determination) regression score function.

Best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a \(R^2\) score of 0.0.

Read more in the User Guide.

Parameters

y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Ground truth (correct) target values.
y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Estimated target values.
sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.
multioutput ({'raw_values', 'uniform_average', 'variance_weighted'}, array-like of shape (n_outputs,) or None, default='uniform_average') –
Defines aggregating of multiple output scores. Array-like value defines weights used to average scores. Default is “uniform_average”.

’raw_values’ :
Returns a full set of scores in case of multioutput input.

’uniform_average’ :
Scores of all outputs are averaged with uniform weight.

’variance_weighted’ :
Scores of all outputs are averaged, weighted by the variances of each individual output.

Changed in version 0.19: Default value of multioutput is ‘uniform_average’.

Returns

z – The \(R^2\) score or ndarray of scores if ‘multioutput’ is ‘raw_values’.

Return type

float or ndarray of floats

Notes

This is not a symmetric function.

Unlike most other scores, \(R^2\) score may be negative (it need not actually be the square of a quantity R).

This metric is not well-defined for single samples and will return a NaN value if n_samples is less than two.

References

1: Wikipedia entry on the Coefficient of determination

Examples

>>> from sklearn.metrics import r2_score
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> r2_score(y_true, y_pred)
0.948...
>>> y_true = [[0.5, 1], [-1, 1], [7, -6]]
>>> y_pred = [[0, 2], [-1, 2], [8, -5]]
>>> r2_score(y_true, y_pred,
...          multioutput='variance_weighted')
0.938...
>>> y_true = [1, 2, 3]
>>> y_pred = [1, 2, 3]
>>> r2_score(y_true, y_pred)
1.0
>>> y_true = [1, 2, 3]
>>> y_pred = [2, 2, 2]
>>> r2_score(y_true, y_pred)
0.0
>>> y_true = [1, 2, 3]
>>> y_pred = [3, 2, 1]
>>> r2_score(y_true, y_pred)
-3.0

rmse(y_true, y_pred, *, sample_weight=None, multioutput='uniform_average', squared=False)¶

Mean squared error regression loss.

Read more in the User Guide.

Parameters

y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Ground truth (correct) target values.
y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Estimated target values.
sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.
multioutput ({'raw_values', 'uniform_average'} or array-like of shape (n_outputs,), default='uniform_average') –
Defines aggregating of multiple output values. Array-like value defines weights used to average errors.

’raw_values’ :
Returns a full set of errors in case of multioutput input.

’uniform_average’ :
Errors of all outputs are averaged with uniform weight.
squared (bool, default=True) – If True returns MSE value, if False returns RMSE value.

Returns

loss – A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.

Return type

float or ndarray of floats

Examples

>>> from sklearn.metrics import mean_squared_error
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> mean_squared_error(y_true, y_pred)
0.375
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> mean_squared_error(y_true, y_pred, squared=False)
0.612...
>>> y_true = [[0.5, 1],[-1, 1],[7, -6]]
>>> y_pred = [[0, 2],[-1, 2],[8, -5]]
>>> mean_squared_error(y_true, y_pred)
0.708...
>>> mean_squared_error(y_true, y_pred, squared=False)
0.822...
>>> mean_squared_error(y_true, y_pred, multioutput='raw_values')
array([0.41666667, 1.        ])
>>> mean_squared_error(y_true, y_pred, multioutput=[0.3, 0.7])
0.825...

sign(y_true: Union[float, Sequence[float], Sequence[Sequence[float]]], y_pred: Union[float, Sequence[float], Sequence[Sequence[float]]], multioutput: str = 'joint') → Union[float, Sequence[float], Sequence[Sequence[float]]][source]¶

Sign error metric.

\[Sign(y\_true, y\_pred) = \frac{1}{n}\cdot\sum_{i=0}^{n - 1}{sign(y\_true_i - y\_pred_i)}\]

Parameters

y_true (Union[float, Sequence[float], Sequence[Sequence[float]]]) –
array-like of shape (n_samples,) or (n_samples, n_outputs)

Ground truth (correct) target values.
y_pred (Union[float, Sequence[float], Sequence[Sequence[float]]]) –
array-like of shape (n_samples,) or (n_samples, n_outputs)

Estimated target values.
multioutput (str) – Defines aggregating of multiple output values (see FunctionalMetricMultioutput).

Returns

A floating point value, or an array of floating point values, one for each individual target.

Return type

Union[float, Sequence[float], Sequence[Sequence[float]]]

smape(y_true: Union[float, Sequence[float], Sequence[Sequence[float]]], y_pred: Union[float, Sequence[float], Sequence[Sequence[float]]], eps: float = 1e-15, multioutput: str = 'joint') → Union[float, Sequence[float], Sequence[Sequence[float]]][source]¶

Symmetric mean absolute percentage error.

Wikipedia entry on the Symmetric mean absolute percentage error

\[SMAPE = \dfrac{100}{n}\sum_{t=1}^{n}\dfrac{|ytrue_{t}-ypred_{t}|}{(|ypred_{t}|+|ytrue_{t}|) / 2}\]

Parameters

y_true (Union[float, Sequence[float], Sequence[Sequence[float]]]) –
array-like of shape (n_samples,) or (n_samples, n_outputs)

Ground truth (correct) target values.
y_pred (Union[float, Sequence[float], Sequence[Sequence[float]]]) –
array-like of shape (n_samples,) or (n_samples, n_outputs)

Estimated target values.
eps (float=1e-15) – SMAPE is undefined for y_true[i] + y_pred[i] == 0 for any i, so all zeros y_true[i] + y_pred[i] are clipped to max(eps, abs(y_true) + abs(y_pred)).
multioutput (str) – Defines aggregating of multiple output values (see FunctionalMetricMultioutput).

Returns

A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.

Return type

Union[float, Sequence[float], Sequence[Sequence[float]]]

wape(y_true: Union[float, Sequence[float], Sequence[Sequence[float]]], y_pred: Union[float, Sequence[float], Sequence[Sequence[float]]], multioutput: str = 'joint') → Union[float, Sequence[float], Sequence[Sequence[float]]][source]¶

Weighted average percentage Error metric.

\[WAPE(y\_true, y\_pred) = \frac{\sum_{i=0}^{n} |y\_true_i - y\_pred_i|}{\sum_{i=0}^{n}|y\_true_i|}\]

Parameters

y_true (Union[float, Sequence[float], Sequence[Sequence[float]]]) –
array-like of shape (n_samples,) or (n_samples, n_outputs)

Ground truth (correct) target values.
y_pred (Union[float, Sequence[float], Sequence[Sequence[float]]]) –
array-like of shape (n_samples,) or (n_samples, n_outputs)

Estimated target values.
multioutput (str) – Defines aggregating of multiple output values (see FunctionalMetricMultioutput).

Returns

A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.

Return type

Union[float, Sequence[float], Sequence[Sequence[float]]]