ROC Curve

This module contains ROC curve calculations, including confidence bands for ROC curves.

class ROCCurve(fnr, fpr, thresholds, fnr_ci=None, fpr_ci=None)[source]

Class to hold the values and thresholds for an ROC curve, optionally with confidence bands.

Parameters:

  • fnr (ndarray) – False Negative Rate.

  • fpr (ndarray) – False Positive Rate.

  • thresholds (ndarray) – Thresholds.

  • fnr_ci (Optional[ndarray]) – (N, 2) array with FNR confidence bands.

  • fpr_ci (Optional[ndarray]) – (N, 2) array with FPR confidence bands.

property far

False Acceptance Rate (alias for FPR)

property far_ci

FAR CIs (alias for FPR CIs)

property frr

False Rejection Rate (alias for FNR)

property frr_ci

FRR CIs (alias for FNR CIs)

property tar

True Acceptance Rate (alias for TPR)

property tar_ci

TAR CIs (Alias for FPR CIs)

property tnr

True Negative Rate

property tnr_ci

TNR CIs

property tpr

True Positive Rate

property tpr_ci

TPR CIs

property trr

True Rejection Rate (alias for TNR)

property trr_ci

TRR CIs (alias for TNR CIs)

roc(scores, *, fnr=None, fpr=None, thresholds=None, nb_points=100, x_axis='fpr')[source]

Computes the ROC curve at the given FNR, FPR or threshold values.

We can provide the FNR, FPR and threshold values at which to compute the ROC curve. We will use the union of these points for the ROC curve.

If neither FNR, FPR, nor threshold values are provided, if nb_points is set, we use linearly spaced points between the smallest positive and negative scores.

If nb_points is None, we use all positive and negative scores for the ROC curve. This gives the highest accuracy, but can be compute-intensive if we are working with large amount of scores.

Parameters:

  • scores (Scores) – Scores for which to compute ROC curve.

  • fnr (Optional[ArrayLike]) – FNR points at which to compute the ROC curve.

  • fpr (Optional[ArrayLike]) – FPR points at which to compute the ROC curve.

  • thresholds (Optional[ArrayLike]) – Thresholds at which to compute the ROC curve.

  • nb_points (Optional[int]) – Number of linearly spaced points to use, if neither one of FNR, FPR, nor thresholds are provided. If None, we use all scores for the ROC curve.

  • x_axis (str) – The values for x-axis metric will be increasing.

Returns:

A ROCCurve object with points on the ROC curve and the corresponding thresholds.

Return type:

ROCCurve

roc_with_ci(scores, *, fnr=None, fpr=None, thresholds=None, nb_points=None, x_axis='fpr', alpha=0.05, config=BootstrapConfig(nb_samples=1000, bootstrap_method='bca', sampling_method='dynamic', stratified_sampling=None, smoothing=False, ratio=None))[source]

Function to compute the confidence band around an ROC curve.

We can provide the FNR, FPR and threshold values at which to compute the ROC curve. We will use the union of these points for the ROC curve.

If neither FNR, FPR, nor threshold values are provided, if nb_points is set, we use linearly spaced points between the smallest positive and negative scores.

If nb_points is None, we use all positive and negative scores for the ROC curve. This gives the highest accuracy, but can be compute-intensive if we are working with large amount of scores.

There are two ways to plot an ROC curve, depending on which metric (FPR or FNR) ir plotted on the x-axis. We expect the user to provide either FPR or FNR values (but not both). The provided values are plotted on the x-axis and used to set a threshold and then compute the other metric at those thresholds. The confidence band is computed using the union of confidence rectangles for both FNR and FPR.

Parameters:

  • scores (Scores) – The Scores object whose ROC curve we compute.

  • fnr (Optional[ArrayLike]) – Optional FNR support points on the ROC curve.

  • fpr (Optional[ArrayLike]) – Optional FPR support points on the ROC curve.

  • thresholds (Optional[ArrayLike]) – Optional threshold support points on the ROC curve.

  • nb_points (Optional[int]) – If at least one of fnr, fpr and threshold is provided, this is the additional number of support points beyond the range of the specified points. If none of fnr, fpr or threshold are provided, we use this number of support points linearly spaced along the FNR and FPR axes.

  • x_axis (str) – The values for x-axis metric will be increasing.

  • alpha (float) – Significance level. In range (0, 1).

  • config (BootstrapConfig) – Bootstrap config.

Returns:

ROCCurve object with point values for the ROC curve and the lower and upper bounds of the confidence band values for both metric values.

Return type:

ROCCurve