clinicadl.io.maps.Maps¶
- class clinicadl.io.maps.Maps(path: Path | str)[source]¶
Python equivalent of a
MAPSdirectory produced by aTrainer.A
MAPSdirectory is the core ofClinicaDL’s experiment management. It contains all the outputs produced by a training phase (models, metrics, logs, etc.), as well as all the hyperparameters used for training. Finally, it also contains the results of test and inference on new datasets.Given that a
MAPSdirectory is quite large and can be tedious to handle, this class allows the user to easily access all the files stored in it.Here are the details of the files and subdirectories that you can find in a
MAPSdirectory, as well as their Python “equivalent” (path→python equivalent):maps_dir
General information on the
MAPSdirectory.architecture.log →
maps.architecture_logDetails on the neural network architecture.
callbacks.json →
maps.callbacks_jsonDetails on the
callbacksused.metrics.json →
maps.metrics_jsonDetails on all the
metricscomputed.model.json →
maps.model_jsonDetails on the
Model().nn_summary.txt →
maps.nn_summary_txtSummary of the neural network provided by
Model.get_summary.summary.log →
maps.summary_logSummary of the
Mapsdirectory.env
A snapshot of the Python environment, captured automatically when the MAPS is created, so that it can be recreated later. More information in the user guide.
environment.txt →
maps.env.environment_txtOutput of a
pip freezecommand. Always generated. It pins every package installed withpipto its exact version.environment.yml →
maps.env.environment_ymlOutput of a
conda env exportcommand. Generated only whenClinicaDLruns inside a conda environment. It pins exact versions and build strings, making it the most faithful copy for reproducing the environment on the same machine.environment_portable.yml →
maps.env.environment_portable_ymlOutput of a
conda env export --no-builds --ignore-channelscommand. Generated only inside a conda environment. Dropping the build strings and channels makes the environment more likely to be recreated on another machine without errors.training
Here are stored the outputs and the information related to
Trainer.trainandTrainer.validate.optimization.json →
maps.training.optimization_jsonDetails on the
optimization parametersused during training.data
Details on the data used during training.
data.tsv →
maps.training.data.data_tsvList of all (participant, session) pairs used for training.
train
Details on the train sets.
split-0
Details on the train set of split
0.data.tsv →
maps.training.data.train.splits[0].data_tsvList of all (participant, session) pairs used for the train set of the split
0.dataloader.json →
maps.training.data.train.splits[0].dataloader_jsonDetails on the train
dataloaderfor split0.dataset.json →
maps.training.data.train.splits[0].dataset_jsonDetails on the train
datasetfor split0.validation
Details on the validation sets.
split-0
Details on the validation set of split
0.data.tsv →
maps.training.data.validation.splits[0].data_tsvList of all (participant, session) pairs used for the validation set of the split
0.dataloader.json →
maps.training.data.train.splits[0].dataloader_jsonDetails on the validation
dataloaderfor split0.dataset.json →
maps.training.data.validation.splits[0].dataset_jsonDetails on the validation
datasetfor split0.split-0
Training results for split
0.computational.json →
maps.training.splits[0].computational_jsonDetails on the
computational parametersused during training of split0.summary.log →
maps.training.splits[0].summary_logSummary of the training of split
0, along with computational metrics provided byMonitorCallback.warning.log →
maps.training.splits[0].warning_logPotential warning logs saved by
LoggerCallback(ifsave_logs=True) that have been raised when training.logs
Logs saved during training.
training_loss.tsv →
maps.training.splits[0].logs.training_loss_tsvTraining loss for each batch seen during training.
computational.tsv →
maps.training.splits[0].logs.computational_tsvComputational metrics saved by
MonitorCallback.learning_rates → maps.training.splits[0].logs.learning_rates
If one or several
LRSchedulerCallbackare activated, this is where the history of the learning rates will be saved.models
All the models saved during training.
best_models
Best models obtained with respect to the metrics monitored in
ModelCheckpointCallback.best-mse
Results for the best model obtained with respect to the metric
"mse".warning.log →
maps.training.splits[0].models.best_models.metrics["mse"].warning_logPotential warning logs saved by
LoggerCallback(ifsave_logs=True) that have been raised when evaluating this model.model.pt →
maps.training.splits[0].models.best_models.metrics["mse"].model_ptThe weights of the model.
validation_metrics
Validation metrics obtained by the best model obtained with respect to the metric
"mse".aggregated.tsv →
maps.training.splits[0].models.best_models.metrics["mse"].validation_metrics.aggregated_tsvAggregated validation metrics.
details.tsv →
maps.training.splits[0].models.best_models.metrics["mse"].validation_metrics.details_tsvValidation metrics for each image.
checkpoints
The model at the epochs defined in
ModelCheckpointCallback.epoch-10
Results for the model at epoch
10.warning.log →
maps.training.splits[0].models.checkpoints.epochs[10].warning_logPotential warning logs saved by
LoggerCallback(ifsave_logs=True) that have been raised when evaluating this model.model.pt →
maps.training.splits[0].models.checkpoints.epochs[10].model_ptThe weights of the model.
validation_metrics
Validation metrics obtained by the model at epoch
10.aggregated.tsv →
maps.training.splits[0].models.checkpoints.epochs[10].validation_metrics.aggregated_tsvAggregated validation metrics.
details.tsv →
maps.training.splits[0].models.checkpoints.epochs[10].validation_metrics.details_tsvValidation metrics for each image.
final
The model at the end of training, saved unless
save_last=FalseinModelCheckpointCallback.warning.log →
maps.training.splits[0].models.final.warning_logPotential warning logs saved by
LoggerCallback(ifsave_logs=True) that have been raised when evaluating this model.model.pt →
maps.training.splits[0].models.final.model_ptThe weights of the model.
validation_metrics
Validation metrics obtained by the final model.
aggregated.tsv →
maps.training.splits[0].models.final.validation_metrics.aggregated_tsvAggregated validation metrics.
details.tsv →
maps.training.splits[0].models.final.validation_metrics.details_tsvValidation metrics for each image.
tmp
Checkpoints saved by
TrainingCheckpointCallbackin order to resume training in case of failure. This directory will be emptied at the end of training.epoch-15
Checkpoint at the end of epoch
15.model.pt →
maps.training.splits[0].tmp.epochs[15].model_ptThe weights of the model.
optimizer.pt →
maps.training.splits[0].tmp.epochs[15].optimizer_ptThe state of the optimizer(s).
scaler.pt →
maps.training.splits[0].tmp.epochs[15].scaler_ptThe state of the Gradient Scaler.
state.json →
maps.training.splits[0].tmp.epochs[15].state_jsonThe
state of the Trainer.callbacks → maps.training.splits[0].tmp.epochs[15].callbacks
Where
callbacks’ checkpoints are saved.validation_metrics
Validation metrics at epoch
15.aggregated.tsv →
maps.training.splits[0].tmp.epochs[15].validation_metrics.aggregated_tsvAggregated validation metrics.
details.tsv →
maps.training.splits[0].tmp.epochs[15].validation_metrics.details_tsvValidation metrics for each image.
validation_metrics
Validation metrics at every validation step.
aggregated.tsv →
maps.training.splits[0].validation_metrics.aggregated_tsvAggregated validation metrics.
details.tsv →
maps.training.splits[0].validation_metrics.details_tsvValidation metrics for each image.
test
Results of
Trainer.test.group-X
Results of test on the group
"X".data.tsv →
maps.test.groups["X"].data_tsvList of all (participant, session) pairs in the group
"X".dataloader.json →
maps.test.groups["X"].dataloader_jsonDetails on the
dataloaderused for the group"X".dataset.json →
maps.test.groups["X"].dataset_jsonDetails on the
datasetused for the group"X".results
split-0
Results of test on the group
"X"obtained with models trained on split0(i.e. the models inmaps_dir/training/split-0/models).epoch-10
Results obtained with the model at epoch
10, trained on split0(i.e. the model inmaps_dir/training/split-0/models/checkpoints/epoch-10).warning.log →
maps.test.groups["X"].results.splits[0].models["epoch-10"].warning_logPotential warning logs saved by
LoggerCallback(ifsave_logs=True) that have been raised when testing with this model.computational.json →
maps.test.groups["X"].results.splits[0].models["epoch-10"].computational_jsonDetails on the
computational parametersused when testing.metrics
Metrics on the group
"X".aggregated.tsv →
maps.test.groups["X"].results.splits[0].models["epoch-10"].metrics.aggregated_tsvAggregated metrics.
details.tsv →
maps.test.groups["X"].results.splits[0].models["epoch-10"].metrics.details_tsvMetrics for each image.
final
Results obtained with the final model trained on the split
0(i.e. the model inmaps_dir/training/split-0/models/final).warning.log →
maps.test.groups["X"].results.splits[0].models["final"].warning_logPotential warning logs saved by
LoggerCallback(ifsave_logs=True) that have been raised when testing with this model.computational.json →
maps.test.groups["X"].results.splits[0].models["final"].computational_jsonDetails on the
computational parametersused when testing.metrics
Metrics on the group
"X".aggregated.tsv →
maps.test.groups["X"].results.splits[0].models["final"].metrics.aggregated_tsvAggregated metrics.
details.tsv →
maps.test.groups["X"].results.splits[0].models["final"].metrics.details_tsvMetrics for each image.
best-mse
Results obtained with the best model with respect to the metric
"mse", trained on the split0(i.e. the model inmaps_dir/training/split-0/models/best_models/best-mse).warning.log →
maps.test.groups["X"].results.splits[0].models["best-mse"].warning_logPotential warning logs saved by
LoggerCallback(ifsave_logs=True) that have been raised when testing with this model.computational.json →
maps.test.groups["X"].results.splits[0].models["best-mse"].computational_jsonDetails on the
computational parametersused when testing.metrics
Metrics on the group
"X".aggregated.tsv →
maps.test.groups["X"].results.splits[0].models["best-mse"].metrics.aggregated_tsvAggregated metrics.
details.tsv →
maps.test.groups["X"].results.splits[0].models["best-mse"].metrics.details_tsvMetrics for each image.
exec
Logging messages saved by
LoggerCallback(ifsave_logs=True).run-train_2025_12_31_23_59_59
Logging messages saved when
Trainer.trainwas run (the end of the filename is the date).debug.log →
maps.exec.runs["train_2025_12_31_23_59_59"].debug_logDebug logs (if
debug=TrueinLoggerCallback).error.log →
maps.exec.runs["train_2025_12_31_23_59_59"].error_logError logs.
info.log →
maps.exec.runs["train_2025_12_31_23_59_59"].info_logInfo and warning logs.
Examples
Let’s say we have a
MAPSdirectory inmaps_dir:from clinicadl.io.maps import Maps maps = Maps("maps_dir") maps.read()
To access the checkpoint of our neural network trained on the split
0at the end of the \(10^{th}\) epoch:>>> maps.training.splits[0].models.checkpoints.epochs[10].model_pt PosixPath('maps_dir/training/split-0/models/checkpoints/epoch-10/model.pt')
To get the list of all saved checkpoints:
>>> maps.training.splits[0].models.checkpoints.epochs_list [0, 5, 10]
- static open_file(path: Path | str) Any[source]¶
To read and load any file inside the
MAPSdirectory.- Parameters:
path (PathType) – The path in the
MAPSdirectory.- Returns:
Any – The content of the file.
- Raises:
FileNotFoundError – If the path does not match any existing file.
Examples
from clinicadl.io.maps import Maps maps = Maps("maps_dir") maps.read()
>>> maps.open_file(maps.metrics_json) { "metrics": { "mse": { "name_": "MSEMetric", "get_not_nans": False, "pred_key": "output", "label_key": "label", "postprocessing": [], "reduction": "mean", }, } } >>> maps.open_file(maps.training.splits[0].models.checkpoints.epochs[1].model_pt) OrderedDict([('conv0.weight', tensor([[[[ 7.2531e-03, 5.7384e-03, 1.4988e-02, ..., -3.1380e-02, -2.2103e-02, 5.8432e-02], ... >>> maps.open_file(maps.training.data.data_tsv) participant_id session_id 0 sub-001 ses-M000 1 sub-002 ses-M000 ...
See also
- static save_file(obj: Any, path: Path | str, overwrite: bool = False) None[source]¶
To save an object in any file of the
MAPSdirectory.- Parameters:
obj (Any) – The object to save.
path (PathType) – The path in the
MAPSdirectory.overwrite (bool, default=False) – To overwrite the file if it exists.
- Raises:
IsADirectoryError – If the path is not a file path.
FileExistsError – If the file exists and
overwrite=False.ValueError – If the file type is not supported. Supported file types in a
MAPSdirectory are.json,.txt,.log,.tsv, and.pt.
Examples
from clinicadl.io.maps import Maps maps = Maps("maps_dir") maps.read()
>>> metrics = { "metrics": { "mse": { "name_": "MSEMetric", "get_not_nans": False, "pred_key": "output", "label_key": "label", "postprocessing": [], "reduction": "mean", }, } } >>> maps.save_file(metrics, maps.metrics_json)
See also
- delete_split(split_idx: int) None[source]¶
To properly remove a split from the
MAPSdirectory.This function will delete the directory
<maps>/training/split-<split_idx>, will update<maps>/training/data, and will delete all the results associated to models trained on this split in<maps>/test.- Parameters:
split_idx (int) – The id of the split to delete.
- read() None¶
To read an existing
MAPSdirectory.Mapswill scan the entire directory to identify all files and subdirectories. It first checks that all required elements are present, meaning that theMAPSdirectory must be complete in order to be read.Mapsalso identifies “optional” files and directories, and keeps track of them.For example, if you call
maps.training.splits[1].summary_logbefore callingmaps.read(), it will raise aKeyErrorbecauseMapsdoesn’t know that there is a split1. Once you will have calledmaps.read(),Mapswill know all the splits present in theMAPSdirectory.- Raises:
FileNotFoundError – If a directory or a file expected in the
MAPSdirectory is missing.