clinicadl.metrics.Metric

class clinicadl.metrics.Metric → None

To define metrics to evaluate a model.

Adapted from monai.metrics.CumulativeIterationMetric.

A metric must inherit from this class to work with ClinicaDL.

_aggregate() and _accumulate() must be implemented.

The user must also define the attribute _optimum:

• use “min” when a lower metric value indicates better performance;

• use “max” when a higher metric value indicates better performance.

Finally, __init__ can be overwritten, but don’t forget to call super().__init__() inside.

Examples

from clinicadl.metrics import Metric

class MyMetric(Metric):
    def __init__(self, ...):
        ...

    def _aggregate(self, data: TensorOrList) -> float:
        ...

    def _accumulate(self, batch: Batch) -> TensorOrList:
        ...

metric = Metric(...)

>>> loader_iterator = iter(dataloader)
>>> metric(next(loader_iterator))
tensor([0., 1., 0.])    # metric value for the 3 images of the batch
>>> metric(next(loader_iterator))
tensor([0., 1., 0.])
>>> metric.aggregate()
0.3333333333333333      # here it is the average on all the images

abstract _aggregate(data: Tensor | Sequence[Tensor]) → float

Aggregation logic.

This function tells how to aggregate the data returned by _accumulate() to compute the metric.

Parameters:

data (TensorOrList) – Data useful to compute the metric, as returned by _accumulate().

Returns:

float – The aggregated metric.

abstract _accumulate(batch: Batch) → Tensor | Sequence[Tensor]

To accumulate data useful for the final metric computation.

For example, for segmentation, to compute the accuracy, this function would just return the confusion matrix for each element of the batch.

Parameters:

batch (Batch) – The batch of DataPoint, passed via a Batch.

Returns:

TensorOrList – Useful results for the final aggregation, as a “batch-first” tensor, or a sequence of “batch-first” tensors.

property optimum: Optimum

Optimization criterion for the metric.

aggregate() → float

See monai.metrics.Cumulative.aggregate().

__call__(batch: Batch) → Tensor

See monai.metrics.CumulativeIterationMetric.__call__().

It is modified to accept a batch of DataPoint, and to get the metric for each element of the batch, whereas the original method only accumulates.

Parameters:

batch (Batch) – The batch of DataPoint, passed via a Batch.

Returns:

torch.Tensor – The metric value for each element of the batch.