ride.metrics

Module Contents

Classes

OptimisationDirection	Generic enumeration.
MetricMixin	Abstract base class for Ride modules
MeanAveragePrecisionMetric	Mean Average Precision (mAP) metric
FlopsMetric	Computes Floating Point Operations (FLOPs) for the model and adds it as metric
FlopsWeightedAccuracyMetric	Computes acc * (flops / target_gflops) ** (-0.07)

Functions

sort_out_figures(→ Tuple[MetricDict, FigureDict])
MetricSelector(→ MetricMixin)
TopKAccuracyMetric(→ MetricMixin)
topks_correct(→ List[torch.Tensor])	Given the predictions, labels, and a list of top-k values, compute the
topk_errors(preds, labels, ks)	Computes the top-k error for each k.
topk_accuracies(preds, labels, ks)	Computes the top-k accuracy for each k.
flops(model)	Compute the Floating Point Operations per Second for the model
params_count(model)	Compute the number of parameters.
make_confusion_matrix(→ matplotlib.figure.Figure)

Attributes

ExtendedMetricDict
MetricDict
FigureDict
StepOutputs
logger

ride.metrics.ExtendedMetricDict

ride.metrics.MetricDict

ride.metrics.FigureDict

ride.metrics.StepOutputs

ride.metrics.logger

ride.metrics.sort_out_figures(d: ExtendedMetricDict) → Tuple[MetricDict, FigureDict]

class ride.metrics.OptimisationDirection

Bases: enum.Enum

Generic enumeration.

Derive from this class to define new enumerations.

MIN = 'min'

MAX = 'max'

class ride.metrics.MetricMixin(hparams: pytorch_lightning.utilities.parsing.AttributeDict, *args, **kwargs)

Bases: ride.core.RideMixin

Abstract base class for Ride modules

classmethod __init_subclass__()

classmethod metrics() → Dict[str, str]

classmethod metric_names() → List[str]

metrics_step(*args, **kwargs) → MetricDict

metrics_epoch(preds: torch.Tensor, targets: torch.Tensor, prefix: str = '', *args, **kwargs) → MetricDict

collect_metrics(preds: torch.Tensor, targets: torch.Tensor) → MetricDict

collect_epoch_metrics(preds: torch.Tensor, targets: torch.Tensor, prefix: str = None) → ExtendedMetricDict

ride.metrics.MetricSelector(mapping: Dict[str, Union[MetricMixin, Iterable[MetricMixin]]] = None, default_config: str = '', **kwargs: Union[MetricMixin, Iterable[MetricMixin]]) → MetricMixin

class ride.metrics.MeanAveragePrecisionMetric(hparams: pytorch_lightning.utilities.parsing.AttributeDict, *args, **kwargs)

Bases: MetricMixin

Mean Average Precision (mAP) metric

validate_attributes()

_compute_mean_average_precision(preds, targets)

classmethod _metrics()

metrics_step(preds: torch.Tensor, targets: torch.Tensor, *args, **kwargs) → MetricDict

metrics_epoch(preds: torch.Tensor, targets: torch.Tensor, *args, **kwargs) → MetricDict

ride.metrics.TopKAccuracyMetric(*Ks) → MetricMixin

class ride.metrics.FlopsMetric(hparams: pytorch_lightning.utilities.parsing.AttributeDict, *args, **kwargs)

Bases: MetricMixin

Computes Floating Point Operations (FLOPs) for the model and adds it as metric

classmethod _metrics()

on_init_end(*args, **kwargs)

metrics_step(preds: torch.Tensor, targets: torch.Tensor, **kwargs) → MetricDict

class ride.metrics.FlopsWeightedAccuracyMetric(hparams: pytorch_lightning.utilities.parsing.AttributeDict, *args, **kwargs)

Bases: FlopsMetric

Computes acc * (flops / target_gflops) ** (-0.07)

classmethod _metrics()

validate_attributes()

static configs() → ride.core.Configs

metrics_step(preds: torch.Tensor, targets: torch.Tensor, **kwargs) → MetricDict

ride.metrics.topks_correct(preds: torch.Tensor, labels: torch.Tensor, ks: List[int]) → List[torch.Tensor]

Given the predictions, labels, and a list of top-k values, compute the number of correct predictions for each top-k value.

Parameters:

• preds (array) – array of predictions. Dimension is batchsize N x ClassNum.

• labels (array) – array of labels. Dimension is batchsize N.

• ks (list) – list of top-k values. For example, ks = [1, 5] correspods to top-1 and top-5.

Returns:

list of numbers, where the i-th entry

corresponds to the number of top-ks[i] correct predictions.

Return type:

topks_correct (list)

ride.metrics.topk_errors(preds: torch.Tensor, labels: torch.Tensor, ks: List[int])

Computes the top-k error for each k. :param preds: array of predictions. Dimension is N. :type preds: array :param labels: array of labels. Dimension is N. :type labels: array :param ks: list of ks to calculate the top accuracies. :type ks: list

ride.metrics.topk_accuracies(preds: torch.Tensor, labels: torch.Tensor, ks: List[int])

Computes the top-k accuracy for each k. :param preds: array of predictions. Dimension is N. :type preds: array :param labels: array of labels. Dimension is N. :type labels: array :param ks: list of ks to calculate the top accuracies. :type ks: list

ride.metrics.flops(model: torch.nn.Module)

Compute the Floating Point Operations per Second for the model

ride.metrics.params_count(model: torch.nn.Module)

Compute the number of parameters. :param model: model to count the number of parameters. :type model: model

ride.metrics.make_confusion_matrix(preds: torch.Tensor, targets: torch.Tensor, classes: List[str]) → matplotlib.figure.Figure