clinicadl.io.maps.Maps

class clinicadl.io.maps.Maps(path: Path | str)[source]

Python equivalent of a MAPS directory produced by a Trainer.

A MAPS directory is the core of ClinicaDL’s experiment management. It contains all the outputs produced by a training phase (models, metrics, logs, etc.), as well as all the hyperparameters used for training. Finally, it also contains the results of test and inference on new datasets.

Given that a MAPS directory is quite large and can be tedious to handle, this class allows the user to easily access all the files stored in it.

Here are the details of the files and subdirectories that you can find in a MAPS directory, as well as their Python “equivalent” (pathpython equivalent):

maps_dir

General information on the MAPS directory.

architecture.log → maps.architecture_log

Details on the neural network architecture.

callbacks.json → maps.callbacks_json

Details on the callbacks used.

metrics.json → maps.metrics_json

Details on all the metrics computed.

model.json → maps.model_json

Details on the Model().

nn_summary.txt → maps.nn_summary_txt

Summary of the neural network provided by Model.get_summary.

summary.log → maps.summary_log

Summary of the Maps directory.

env

A snapshot of the Python environment, captured automatically when the MAPS is created, so that it can be recreated later. More information in the user guide.

environment.txt → maps.env.environment_txt

Output of a pip freeze command. Always generated. It pins every package installed with pip to its exact version.

environment.yml → maps.env.environment_yml

Output of a conda env export command. Generated only when ClinicaDL runs inside a conda environment. It pins exact versions and build strings, making it the most faithful copy for reproducing the environment on the same machine.

environment_portable.yml → maps.env.environment_portable_yml

Output of a conda env export --no-builds --ignore-channels command. Generated only inside a conda environment. Dropping the build strings and channels makes the environment more likely to be recreated on another machine without errors.

training

Here are stored the outputs and the information related to Trainer.train and Trainer.validate.

optimization.json → maps.training.optimization_json

Details on the optimization parameters used during training.

data

Details on the data used during training.

data.tsv → maps.training.data.data_tsv

List of all (participant, session) pairs used for training.

train

Details on the train sets.

split-0

Details on the train set of split 0.

data.tsv → maps.training.data.train.splits[0].data_tsv

List of all (participant, session) pairs used for the train set of the split 0.

dataloader.json → maps.training.data.train.splits[0].dataloader_json

Details on the train dataloader for split 0.

dataset.json → maps.training.data.train.splits[0].dataset_json

Details on the train dataset for split 0.

validation

Details on the validation sets.

split-0

Details on the validation set of split 0.

data.tsv → maps.training.data.validation.splits[0].data_tsv

List of all (participant, session) pairs used for the validation set of the split 0.

dataloader.json → maps.training.data.train.splits[0].dataloader_json

Details on the validation dataloader for split 0.

dataset.json → maps.training.data.validation.splits[0].dataset_json

Details on the validation dataset for split 0.

split-0

Training results for split 0.

computational.json → maps.training.splits[0].computational_json

Details on the computational parameters used during training of split 0.

summary.log → maps.training.splits[0].summary_log

Summary of the training of split 0, along with computational metrics provided by MonitorCallback.

warning.log → maps.training.splits[0].warning_log

Potential warning logs saved by LoggerCallback (if save_logs=True) that have been raised when training.

logs

Logs saved during training.

training_loss.tsv → maps.training.splits[0].logs.training_loss_tsv

Training loss for each batch seen during training.

computational.tsv → maps.training.splits[0].logs.computational_tsv

Computational metrics saved by MonitorCallback.

learning_rates → maps.training.splits[0].logs.learning_rates

If one or several LRSchedulerCallback are activated, this is where the history of the learning rates will be saved.

models

All the models saved during training.

best_models

Best models obtained with respect to the metrics monitored in ModelCheckpointCallback.

best-mse

Results for the best model obtained with respect to the metric "mse".

warning.log → maps.training.splits[0].models.best_models.metrics["mse"].warning_log

Potential warning logs saved by LoggerCallback (if save_logs=True) that have been raised when evaluating this model.

model.pt → maps.training.splits[0].models.best_models.metrics["mse"].model_pt

The weights of the model.

validation_metrics

Validation metrics obtained by the best model obtained with respect to the metric "mse".

aggregated.tsv → maps.training.splits[0].models.best_models.metrics["mse"].validation_metrics.aggregated_tsv

Aggregated validation metrics.

details.tsv → maps.training.splits[0].models.best_models.metrics["mse"].validation_metrics.details_tsv

Validation metrics for each image.

checkpoints

The model at the epochs defined in ModelCheckpointCallback.

epoch-10

Results for the model at epoch 10.

warning.log → maps.training.splits[0].models.checkpoints.epochs[10].warning_log

Potential warning logs saved by LoggerCallback (if save_logs=True) that have been raised when evaluating this model.

model.pt → maps.training.splits[0].models.checkpoints.epochs[10].model_pt

The weights of the model.

validation_metrics

Validation metrics obtained by the model at epoch 10.

aggregated.tsv → maps.training.splits[0].models.checkpoints.epochs[10].validation_metrics.aggregated_tsv

Aggregated validation metrics.

details.tsv → maps.training.splits[0].models.checkpoints.epochs[10].validation_metrics.details_tsv

Validation metrics for each image.

final

The model at the end of training, saved unless save_last=False in ModelCheckpointCallback.

warning.log → maps.training.splits[0].models.final.warning_log

Potential warning logs saved by LoggerCallback (if save_logs=True) that have been raised when evaluating this model.

model.pt → maps.training.splits[0].models.final.model_pt

The weights of the model.

validation_metrics

Validation metrics obtained by the final model.

aggregated.tsv → maps.training.splits[0].models.final.validation_metrics.aggregated_tsv

Aggregated validation metrics.

details.tsv → maps.training.splits[0].models.final.validation_metrics.details_tsv

Validation metrics for each image.

tmp

Checkpoints saved by TrainingCheckpointCallback in order to resume training in case of failure. This directory will be emptied at the end of training.

epoch-15

Checkpoint at the end of epoch 15.

model.pt → maps.training.splits[0].tmp.epochs[15].model_pt

The weights of the model.

optimizer.pt → maps.training.splits[0].tmp.epochs[15].optimizer_pt

The state of the optimizer(s).

scaler.pt → maps.training.splits[0].tmp.epochs[15].scaler_pt

The state of the Gradient Scaler.

state.json → maps.training.splits[0].tmp.epochs[15].state_json
callbacks → maps.training.splits[0].tmp.epochs[15].callbacks

Where callbacks’ checkpoints are saved.

validation_metrics

Validation metrics at epoch 15.

aggregated.tsv → maps.training.splits[0].tmp.epochs[15].validation_metrics.aggregated_tsv

Aggregated validation metrics.

details.tsv → maps.training.splits[0].tmp.epochs[15].validation_metrics.details_tsv

Validation metrics for each image.

validation_metrics

Validation metrics at every validation step.

aggregated.tsv → maps.training.splits[0].validation_metrics.aggregated_tsv

Aggregated validation metrics.

details.tsv → maps.training.splits[0].validation_metrics.details_tsv

Validation metrics for each image.

test

Results of Trainer.test.

group-X

Results of test on the group "X".

data.tsv → maps.test.groups["X"].data_tsv

List of all (participant, session) pairs in the group "X".

dataloader.json → maps.test.groups["X"].dataloader_json

Details on the dataloader used for the group "X".

dataset.json → maps.test.groups["X"].dataset_json

Details on the dataset used for the group "X".

results
split-0

Results of test on the group "X" obtained with models trained on split 0 (i.e. the models in maps_dir/training/split-0/models).

epoch-10

Results obtained with the model at epoch 10, trained on split 0 (i.e. the model in maps_dir/training/split-0/models/checkpoints/epoch-10).

warning.log → maps.test.groups["X"].results.splits[0].models["epoch-10"].warning_log

Potential warning logs saved by LoggerCallback (if save_logs=True) that have been raised when testing with this model.

computational.json → maps.test.groups["X"].results.splits[0].models["epoch-10"].computational_json

Details on the computational parameters used when testing.

metrics

Metrics on the group "X".

aggregated.tsv → maps.test.groups["X"].results.splits[0].models["epoch-10"].metrics.aggregated_tsv

Aggregated metrics.

details.tsv → maps.test.groups["X"].results.splits[0].models["epoch-10"].metrics.details_tsv

Metrics for each image.

final

Results obtained with the final model trained on the split 0 (i.e. the model in maps_dir/training/split-0/models/final).

warning.log → maps.test.groups["X"].results.splits[0].models["final"].warning_log

Potential warning logs saved by LoggerCallback (if save_logs=True) that have been raised when testing with this model.

computational.json → maps.test.groups["X"].results.splits[0].models["final"].computational_json

Details on the computational parameters used when testing.

metrics

Metrics on the group "X".

aggregated.tsv → maps.test.groups["X"].results.splits[0].models["final"].metrics.aggregated_tsv

Aggregated metrics.

details.tsv → maps.test.groups["X"].results.splits[0].models["final"].metrics.details_tsv

Metrics for each image.

best-mse

Results obtained with the best model with respect to the metric "mse", trained on the split 0 (i.e. the model in maps_dir/training/split-0/models/best_models/best-mse).

warning.log → maps.test.groups["X"].results.splits[0].models["best-mse"].warning_log

Potential warning logs saved by LoggerCallback (if save_logs=True) that have been raised when testing with this model.

computational.json → maps.test.groups["X"].results.splits[0].models["best-mse"].computational_json

Details on the computational parameters used when testing.

metrics

Metrics on the group "X".

aggregated.tsv → maps.test.groups["X"].results.splits[0].models["best-mse"].metrics.aggregated_tsv

Aggregated metrics.

details.tsv → maps.test.groups["X"].results.splits[0].models["best-mse"].metrics.details_tsv

Metrics for each image.

exec

Logging messages saved by LoggerCallback (if save_logs=True).

run-train_2025_12_31_23_59_59

Logging messages saved when Trainer.train was run (the end of the filename is the date).

debug.log → maps.exec.runs["train_2025_12_31_23_59_59"].debug_log

Debug logs (if debug=True in LoggerCallback).

error.log → maps.exec.runs["train_2025_12_31_23_59_59"].error_log

Error logs.

info.log → maps.exec.runs["train_2025_12_31_23_59_59"].info_log

Info and warning logs.

Examples

Let’s say we have a MAPS directory in maps_dir:

from clinicadl.io.maps import Maps

maps = Maps("maps_dir")
maps.read()

To access the checkpoint of our neural network trained on the split 0 at the end of the \(10^{th}\) epoch:

>>> maps.training.splits[0].models.checkpoints.epochs[10].model_pt
PosixPath('maps_dir/training/split-0/models/checkpoints/epoch-10/model.pt')

To get the list of all saved checkpoints:

>>> maps.training.splits[0].models.checkpoints.epochs_list
[0, 5, 10]
create(overwrite: bool = False, exist_ok: bool = False) None[source]

Creates the MAPS directory.

Parameters:
  • overwrite (bool, default=False) – Whether to overwrite the current directory.

  • exists_ok (bool, default=False) – If the directory already exists and overwrite=False, the function succeeds when exist_ok=True.

static open_file(path: Path | str) Any[source]

To read and load any file inside the MAPS directory.

Parameters:

path (PathType) – The path in the MAPS directory.

Returns:

Any – The content of the file.

Raises:

FileNotFoundError – If the path does not match any existing file.

Examples

from clinicadl.io.maps import Maps

maps = Maps("maps_dir")
maps.read()
>>> maps.open_file(maps.metrics_json)
{
    "metrics": {
        "mse": {
            "name_": "MSEMetric",
            "get_not_nans": False,
            "pred_key": "output",
            "label_key": "label",
            "postprocessing": [],
            "reduction": "mean",
        },
    }
}
>>> maps.open_file(maps.training.splits[0].models.checkpoints.epochs[1].model_pt)
OrderedDict([('conv0.weight',
  tensor([[[[ 7.2531e-03,  5.7384e-03,  1.4988e-02,  ..., -3.1380e-02,
             -2.2103e-02,  5.8432e-02],
...
>>> maps.open_file(maps.training.data.data_tsv)
    participant_id  session_id
0   sub-001 ses-M000
1   sub-002 ses-M000
...

See also

save_file()

static save_file(obj: Any, path: Path | str, overwrite: bool = False) None[source]

To save an object in any file of the MAPS directory.

Parameters:
  • obj (Any) – The object to save.

  • path (PathType) – The path in the MAPS directory.

  • overwrite (bool, default=False) – To overwrite the file if it exists.

Raises:
  • IsADirectoryError – If the path is not a file path.

  • FileExistsError – If the file exists and overwrite=False.

  • ValueError – If the file type is not supported. Supported file types in a MAPS directory are .json, .txt, .log, .tsv, and .pt.

Examples

from clinicadl.io.maps import Maps

maps = Maps("maps_dir")
maps.read()
>>> metrics = {
        "metrics": {
            "mse": {
                "name_": "MSEMetric",
                "get_not_nans": False,
                "pred_key": "output",
                "label_key": "label",
                "postprocessing": [],
                "reduction": "mean",
            },
        }
    }
>>> maps.save_file(metrics, maps.metrics_json)

See also

open_file()

delete_split(split_idx: int) None[source]

To properly remove a split from the MAPS directory.

This function will delete the directory <maps>/training/split-<split_idx>, will update <maps>/training/data, and will delete all the results associated to models trained on this split in <maps>/test.

Parameters:

split_idx (int) – The id of the split to delete.

read() None

To read an existing MAPS directory.

Maps will scan the entire directory to identify all files and subdirectories. It first checks that all required elements are present, meaning that the MAPS directory must be complete in order to be read. Maps also identifies “optional” files and directories, and keeps track of them.

For example, if you call maps.training.splits[1].summary_log before calling maps.read(), it will raise a KeyError because Maps doesn’t know that there is a split 1. Once you will have called maps.read(), Maps will know all the splits present in the MAPS directory.

Raises:

FileNotFoundError – If a directory or a file expected in the MAPS directory is missing.

remove(non_empty_ok: bool = False) None

To delete the MAPS directory.

Parameters:

non_empty_ok (bool, default=False) – Whether to remove the directory even if it is non-empty.