clinicadl.networks.nn.MLP

class clinicadl.networks.nn.MLP(num_inputs: int, num_outputs: int, hidden_dims: Sequence[int], act: ActFunction | tuple[ActFunction, dict[str, Any]] | None = ActFunction.PRELU, output_act: ActFunction | tuple[ActFunction, dict[str, Any]] | None = None, norm: NormLayer | tuple[NormLayer, dict[str, Any]] | None = NormLayer.BATCH, dropout: float | None = None, bias: bool = True, adn_ordering: str = 'NDA') → None

Simple fully-connected neural network (or Multi-Layer Perceptron) with linear, normalization, activation and dropout layers.

It contains a flattening layer, so it works with images (only batched images).

Parameters:

• num_inputs (int) – Number of input features (after flattening).

• num_outputs (int) – Number of outputs.

• hidden_dims (Sequence[int]) – Number of outputs for each hidden layer. Thus, this parameter also controls the number of hidden layers (equal to the length of the sequence).

• act (Optional[ActivationParameters], default="prelu") –

  The activation function used after a linear layer, and optionally its arguments. Must be passed as activation_name or (activation_name, arguments), where arguments is a dictionary. If None, no activation will be used.

  activation_name can be any value in {"celu", "elu", "gelu", "leakyrelu", "logsoftmax", "mish", "prelu", "relu", "relu6", "selu", "sigmoid", "softmax", "tanh"}. Please refer to PyTorch activation functions to know the arguments for each of them.

• output_act (Optional[ActivationParameters], default=None) – A potential activation layer applied to the output of the network. Must be passed in the same way as act. If None, no last activation will be applied.

• norm (Optional[NormalizationParameters], default="batch") –

  The normalization layer used after a linear layer, and optionally its arguments. Must be passed as norm_type or (norm_type, parameters). If None, no normalization will be performed.

  norm_type can be any value in {"batch", "group", "instance", "syncbatch"}. Please refer to PyTorch normalization layers to know the arguments for each of them.

  Note

  Please note that there’s no need to pass the arguments num_channels, num_features and normalized_shape of the normalization layer, as they are automatically inferred from the output of the previous layer in the network.

• dropout (Optional[float], default=None) – Dropout ratio. If None, no dropout.

• bias (bool, default=True) – Whether to have a bias term in linear layers.

• adn_ordering (str) – Order of operations Activation, Dropout and Normalization, after a linear layer (except the last one). Cannot contain duplicated letters. For example if "ND" is passed, Normalization and then Dropout will be performed (without Activation).

Raises:

ValueError – If the activation or normalization layer requires a mandatory argument, which is not passed by the user (via a dictionary in act or norm).

See also

torch.nn.Module

To see all the methods of this neural network.

Examples

>>> MLP(
        num_inputs=12,
        num_outputs=2,
        hidden_dims=[8, 4],
        dropout=0.1,
        act=("elu", {"alpha": 0.5}),
        norm=("group", {"num_groups": 2}),
        bias=True,
        adn_ordering="ADN",
        output_act="softmax",
    )
MLP(
    (flatten): Flatten(start_dim=1, end_dim=-1)
    (hidden0): Sequential(
        (linear): Linear(in_features=12, out_features=8, bias=True)
        (adn): ADN(
            (A): ELU(alpha=0.5)
            (D): Dropout(p=0.1, inplace=False)
            (N): GroupNorm(2, 8, eps=1e-05, affine=True)
        )
    )
    (hidden1): Sequential(
        (linear): Linear(in_features=8, out_features=4, bias=True)
        (adn): ADN(
            (A): ELU(alpha=0.5)
            (D): Dropout(p=0.1, inplace=False)
            (N): GroupNorm(2, 4, eps=1e-05, affine=True)
        )
    )
    (output): Sequential(
        (linear): Linear(in_features=4, out_features=2, bias=True)
        (output_act): Softmax(dim=None)
    )
)