Metrics

This module contains the fundamental metrics computations. The ConfusionMatrix class relies on the implementations in this module.

acceptance_rate(matrix)[source]

Acceptance Rate. Alias for topr().

Parameters:: matrix (ndarray) –
Return type:: Union[ndarray, float]

accuracy(matrix)[source]

Accuracy for confusion matrices.

Parameters:: matrix (ndarray) – Array of shape (…, N, N).
Returns:: Array of shape (…). Returns NaNs in case of zero matrix.
Return type:: Union[ndarray, float]

error_rate(matrix)[source]

Error rate for confusion matrices.

Formula:

Error Rate = 1 - Accuracy

Parameters:: matrix (ndarray) – Array of shape (…, N, N).
Returns:: Array of shape (…). Returns NaNs in case of zero matrix.
Return type:: Union[ndarray, float]

far(matrix)[source]

False Acceptance Rate. Alias for fpr().

Parameters:: matrix (ndarray) –
Return type:: Union[ndarray, float]

far_ci(matrix, alpha=0.05)[source]

Confidence interval for the False Acceptance Rate. Alias for fpr_ci().

Parameters:

matrix (ndarray) –
alpha (float) –

Return type:

ndarray

fdr(matrix)[source]

False Discovery Rate for binary confusion matrices.

Formula:

FDR = FP / (TP + FP) = FP / TOP = 1 - PPV

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…). Returns NaNs in case of no elements with prediction positive.
Return type:: Union[ndarray, float]

fn(matrix)[source]

False Negatives for binary confusion matrices. Test does not detect the condition while the condition is present.

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…).
Return type:: Union[ndarray, float]

fnr(matrix)[source]

False Negative Rate for binary confusion matrices.

Formula:

FNR = FN / (TP + FN) = FN / P

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…). Returns NaNs in case of no elements with positive condition.
Return type:: Union[ndarray, float]

fnr_ci(matrix, alpha=0.05)[source]

Confidence inferval for the False Negative Rate for binary confusion matrices.

Parameters:

matrix (ndarray) – Array of shape (…, 2, 2)
alpha (float) – Significance level. In range (0, 1).

Returns:

Array of shape (…, 2). Lower and upper limits of CI with coverage 1-alpha.

Return type:

ndarray

for_(matrix)[source]

False Omission Rate for binary confusion matrices.

Formula:

FOR = FN / (TN + FN) = FN / TON = 1 - NPV

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…). Returns NaNs in case of no elements with prediction negative.
Return type:: Union[ndarray, float]

fp(matrix)[source]

False Positives for binary confusion matrices. Test detects the condition while the condition is absent.

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…).
Return type:: Union[ndarray, float]

fpr(matrix)[source]

False Positive Rate for binary confusion matrices.

Formula:

FPR = FP / (FP + TN) = FP / N

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…). Returns NaNs in case of no elements with positive condition.
Return type:: Union[ndarray, float]

fpr_ci(matrix, alpha=0.05)[source]

Confidence inferval for the False Positive Rate for binary confusion matrices.

Parameters:

matrix (ndarray) – Array of shape (…, 2, 2)
alpha (float) – Significance level. In range (0, 1).

Returns:

Array of shape (…, 2). Lower and upper limits of CI with coverage 1-alpha.

Return type:

ndarray

frr(matrix)[source]

False Rejection Rate. Alias for fnr().

Parameters:: matrix (ndarray) –
Return type:: Union[ndarray, float]

frr_ci(matrix, alpha=0.05)[source]

Confidence interval for the False Rejection Rate. Alias for fnr_ci().

Parameters:

matrix (ndarray) –
alpha (float) –

Return type:

ndarray

n(matrix)[source]

Condition Negative for binary confusion matrices. Number of samples with condition negative.

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…).
Return type:: Union[ndarray, float]

npv(matrix)[source]

Negative Predictive Value for binary confusion matrices.

Formula:

NPV = TN / (TN + FN) = TN / TON

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…). Returns NaNs in case of no elements with prediction positive.
Return type:: Union[ndarray, float]

p(matrix)[source]

Condition Positive for binary confusion matrices. Number of samples with condition positive.

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…).
Return type:: Union[ndarray, float]

pop(matrix)[source]

Total population for confusion matrices.

Parameters:: matrix (ndarray) – Array of shape (…, N, N).
Returns:: Array of shape (…).
Return type:: Union[ndarray, float]

ppv(matrix)[source]

Positive Predictive Value for binary confusion matrices.

Formula:

PPV = TP / (TP + FP) = TP / TOP

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…). Returns NaNs in case of no elements with prediction positive.
Return type:: Union[ndarray, float]

rejection_rate(matrix)[source]

Rejection Rate. Alias for tonr().

Parameters:: matrix (ndarray) –
Return type:: Union[ndarray, float]

tar(matrix)[source]

True Acceptance Rate. Alias for tpr().

Parameters:: matrix (ndarray) –
Return type:: Union[ndarray, float]

tar_ci(matrix, alpha=0.05)[source]

Confidence interval for the True Acceptance Rate. Alias for tpr_ci().

Parameters:

matrix (ndarray) –
alpha (float) –

Return type:

ndarray

tn(matrix)[source]

True Negatives for binary confusion matrices. Test does not detect the condition and the condition is absent.

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…).
Return type:: Union[ndarray, float]

tnr(matrix)[source]

True Negative Rate for binary confusion matrices.

Formula:

TNR = TN / (FP + TN) = TN / N

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…). Returns NaNs in case of no elements with positive condition.
Return type:: Union[ndarray, float]

tnr_ci(matrix, alpha=0.05)[source]

Confidence inferval for the True Negative Rate for binary confusion matrices.

Parameters:

matrix (ndarray) – Array of shape (…, 2, 2)
alpha (float) – Significance level. In range (0, 1).

Returns:

Array of shape (…, 2). Lower and upper limits of CI with coverage 1-alpha.

Return type:

ndarray

ton(matrix)[source]

Test Outcome Negative. Number of samples where test does not detect condition.

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…).
Return type:: Union[ndarray, float]

tonr(matrix)[source]

Test Outcome Negative Rate. Proportion of samples where test does not detect condition.

Formula:

TONR = TON / N = (TNR + FNR) / (TPR + FPR + TNR + FNR)

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…).
Return type:: Union[ndarray, float]

top(matrix)[source]

Test Outcome Positive. Number of samples where test detects condition.

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…).
Return type:: Union[ndarray, float]

topr(matrix)[source]

Test Outcome Positive Rate. Proportion of samples where test detects condition.

Formula:

TOPR = TOP / N = (TPR + FPR) / (TPR + FPR + TNR + FNR)

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…).
Return type:: Union[ndarray, float]

tp(matrix)[source]

True Positives for binary confusion matrices. Test detects the condition while the condition is present.

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…).
Return type:: Union[ndarray, float]

tpr(matrix)[source]

True Positive Rate for binary confusion matrices.

Formula:

TPR = TP / (TP + FN) = TP / P

Parameters:: matrix (ndarray) – Array of shape (…, 2, 2).
Returns:: Array of shape (…). Returns NaNs in case of no elements with positive condition.
Return type:: Union[ndarray, float]

tpr_ci(matrix, alpha=0.05)[source]

Confidence inferval for the True Positive Rate for binary confusion matrices.

Parameters:

matrix (ndarray) – Array of shape (…, 2, 2)
alpha (float) – Significance level. In range (0, 1).

Returns:

Array of shape (…, 2). Lower and upper limits of CI with coverage 1-alpha.

Return type:

ndarray

trr(matrix)[source]

True Rejection Rate. Alias for tnr().

Parameters:: matrix (ndarray) –
Return type:: Union[ndarray, float]

trr_ci(matrix, alpha=0.05)[source]

Confidence interval for the True Rejection Rate. Alias for tnr_ci().

Parameters:

matrix (ndarray) –
alpha (float) –

Return type:

ndarray