Skip to content

Modeling Module

Classifier

Bases: Module

Multi-class classifier for adversarial training in n-modal latent space alignment.

Attributes:

Name Type Description
input_dim

Dimension of the input features.
n_modalities

Number of modalities (classes) to classify.
n_hidden

Number of hidden units in the classifier network.
Source code in src/autoencodix/modeling/_classifier.py 
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
class Classifier(nn.Module):
    """Multi-class classifier for adversarial training in n-modal latent space alignment.


    Attributes:
        input_dim: Dimension of the input features.
        n_modalities: Number of modalities (classes) to classify.
        n_hidden: Number of hidden units in the classifier network.

    """

    def __init__(self, input_dim: int, n_modalities: int, n_hidden: int = 64) -> None:
        super().__init__()
        self.input_dim = input_dim
        self.n_modalities = n_modalities
        self.classifier = nn.Sequential(
            nn.Linear(input_dim, n_hidden),
            nn.ReLU(inplace=False),
            nn.Dropout(0.1),
            nn.Linear(n_hidden, n_hidden // 2),
            nn.ReLU(inplace=False),
            nn.Dropout(0.1),
            nn.Linear(n_hidden // 2, n_modalities),
        )
        self.apply(self._init_weights)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        """Forward pass through the classifier.

        Args:
            x: Input tensor of shape (batch_size, input_dim).
        Returns:
            Output tensor of shape (batch_size, n_modalities) representing class scores.
        """
        return self.classifier(x)

    def _init_weights(self, m: nn.Module) -> None:
        if isinstance(m, nn.Linear):
            nn.init.xavier_uniform_(m.weight)
            if m.bias is not None:
                m.bias.data.fill_(0.01)

forward(x)

Forward pass through the classifier.

Parameters:

Name Type Description Default
x Tensor

Input tensor of shape (batch_size, input_dim).

 required

Returns: Output tensor of shape (batch_size, n_modalities) representing class scores.

Source code in src/autoencodix/modeling/_classifier.py 
31
32
33
34
35
36
37
38
39
def forward(self, x: torch.Tensor) -> torch.Tensor:
    """Forward pass through the classifier.

    Args:
        x: Input tensor of shape (batch_size, input_dim).
    Returns:
        Output tensor of shape (batch_size, n_modalities) representing class scores.
    """
    return self.classifier(x)

ImageVAEArchitecture

Bases: BaseAutoencoder

This class defines a VAE, based on a CNN for images

It takes as input an image and of shape (C,W,H) and reconstructs it. We ensure to have a latent space of shape and img_in.shape = img_out.shape We have a fixed kernel_size=4, padding=1 and stride=2 (given from https://github.com/uhlerlab/cross-modal-auto_encoders/tree/master)

So we need to calculate how the image dimension changes after each Convolution (we assume W=H) Applying the formular: W_out = (((W - kernel_size + 2padding)/stride) + 1) We get: W_out = (((W-4+2*1)/2)+1) = = (W-2/2)+1 = = (2(0.5W-1)/2) +1 # factor 2 out = 0.5W - 1 + 1 W_out = 0.5W So in this configuration the output shape halfs after every convolutional step (assuming W=H)

Attributes:

Name Type Description
input_dim int

(C,W,H) the input image shape
config

Configuration object containing model architecture parameters
_encoder Optional[Module]

Encoder network of the autoencoder
_decoder Optional[Module]

Decoder network of the autoencoder
latent_dim int

Dimension of the latent space
nc int

number of channels in the input image
h int

height of the input image
w int

width of the input image
img_shape Tuple[int, int, int]

(C,W,H) the input image shape
hidden_dim int

number of filters in the first convolutional layer
Source code in src/autoencodix/modeling/_imagevae_architecture.py 
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
class ImageVAEArchitecture(BaseAutoencoder):
    """This class defines a VAE, based on a CNN for images

    It takes as input an image and of shape (C,W,H) and reconstructs it.
    We ensure to have a latent space of shape <batchsize,1,LatentDim> and img_in.shape = img_out.shape
    We have a fixed kernel_size=4, padding=1 and stride=2 (given from https://github.com/uhlerlab/cross-modal-auto_encoders/tree/master)

    So we need to calculate how the image dimension changes after each Convolution (we assume W=H)
    Applying the formular:
        W_out = (((W - kernel_size + 2padding)/stride) + 1)
    We get:
        W_out = (((W-4+2*1)/2)+1) =
        = (W-2/2)+1 =
        = (2(0.5W-1)/2) +1 # factor 2 out
        = 0.5W - 1 + 1
        W_out = 0.5W
    So in this configuration the output shape halfs after every convolutional step (assuming W=H)


    Attributes:
        input_dim: (C,W,H) the input image shape
        config: Configuration object containing model architecture parameters
        _encoder: Encoder network of the autoencoder
        _decoder: Decoder network of the autoencoder
        latent_dim: Dimension of the latent space
        nc: number of channels in the input image
        h: height of the input image
        w: width of the input image
        img_shape: (C,W,H) the input image shape
        hidden_dim: number of filters in the first convolutional layer
    """

    def __init__(
        self,
        input_dim: Tuple[int, int, int],  # (C,W,H) the input image shape
        config: Optional[DefaultConfig],
        ontologies: Optional[Union[Tuple, Dict]] = None,
        feature_order: Optional[Union[Tuple, Dict]] = None,
        # the input_dim is the number of channels in the image, e.g. 3
    ):
        """Initialize the ImageVAEArchitecture with the given configuration.

        Args:
            input_dim: (C,W,H) the input image shape
            config: Configuration object containing model parameters.
            hidden_dim: number of filters in the first convolutional layer
        """
        if config is None:
            config = DefaultConfig()
        self._config: DefaultConfig = config
        super().__init__(config=config, input_dim=input_dim)
        self.input_dim: int = input_dim
        self.latent_dim: int = self._config.latent_dim
        self.nc, self.h, self.w = input_dim
        self.img_shape: Tuple[int, int, int] = input_dim
        self.hidden_dim: int = self._config.hidden_dim
        self._build_network()
        self.apply(self._init_weights)

    def _build_network(self):
        """Construct the encoder and decoder networks."""
        self._encoder = nn.Sequential(
            nn.Conv2d(
                in_channels=self.nc,
                out_channels=self.hidden_dim,
                kernel_size=4,
                stride=2,
                padding=1,
                bias=False,
            ),
            nn.LeakyReLU(0.2, inplace=False),
            nn.Conv2d(
                in_channels=self.hidden_dim,
                out_channels=self.hidden_dim * 2,
                kernel_size=4,
                stride=2,
                padding=1,
                bias=False,
            ),
            nn.BatchNorm2d(self.hidden_dim * 2),
            nn.LeakyReLU(0.2, inplace=False),
            nn.Conv2d(
                in_channels=self.hidden_dim * 2,
                out_channels=self.hidden_dim * 4,
                kernel_size=4,
                stride=2,
                padding=1,
                bias=False,
            ),
            nn.BatchNorm2d(self.hidden_dim * 4),
            nn.LeakyReLU(0.2, inplace=False),
            nn.Conv2d(
                in_channels=self.hidden_dim * 4,
                out_channels=self.hidden_dim * 8,
                kernel_size=4,
                stride=2,
                padding=1,
                bias=False,
            ),
            nn.BatchNorm2d(self.hidden_dim * 8),
            nn.LeakyReLU(0.2, inplace=False),
            nn.Conv2d(
                in_channels=self.hidden_dim * 8,
                out_channels=self.hidden_dim * 8,
                kernel_size=4,
                stride=2,
                padding=1,
                bias=False,
            ),
            nn.BatchNorm2d(self.hidden_dim * 8),
            nn.LeakyReLU(0.2, inplace=False),
        )

        # to Calculate the image shape after the _encoder, we need to know the number of layers
        # because the shape halfs after every Conv2D layer
        self.num__encoder_layers = sum(
            1 for _ in self._encoder.children() if isinstance(_, nn.Conv2d)
        )
        # So the output shape after all layers is in_shape / 2**N_layers
        # We showed above in the DocString why the shape halfs
        self.spatial_dim = self.h // (2**self.num__encoder_layers)
        # In the Linear mu and logvar layer we need to flatten the 3D output to a 2D matrix
        # Therefore we need to multiply the size of every out diemension of the input layer to the Linear layers
        # This is hidden_dim * 8 (the number of filter/channel layer) * spatial dim (the widht of the image) * spatial diem (the height of the image)
        # assuimg width = height
        # The original paper had a fixed spatial dimension of 2, which only worked for images with 64x64 shape
        self.mu = nn.Linear(
            self.hidden_dim * 8 * self.spatial_dim * self.spatial_dim, self.latent_dim
        )
        self.logvar = nn.Linear(
            self.hidden_dim * 8 * self.spatial_dim * self.spatial_dim, self.latent_dim
        )

        # the same logic goes for the first _decoder layer, which takes the latent_dim as inshape
        # which is the outshape of the previous mu/logvar layer
        # and the shape of the first ConvTranspose2D layer is the last outpus shape of the _encoder layer
        # This the same multiplication as above
        self.d1 = nn.Sequential(
            nn.Linear(
                self.latent_dim,
                self.hidden_dim * 8 * self.spatial_dim * self.spatial_dim,
            ),
            nn.ReLU(inplace=False),
        )
        self._decoder = nn.Sequential(
            nn.ConvTranspose2d(
                in_channels=self.hidden_dim * 8,
                out_channels=self.hidden_dim * 8,
                kernel_size=4,
                stride=2,
                padding=1,
                bias=False,
            ),
            nn.BatchNorm2d(self.hidden_dim * 8),
            nn.LeakyReLU(0.2, inplace=False),
            nn.ConvTranspose2d(
                in_channels=self.hidden_dim * 8,
                out_channels=self.hidden_dim * 4,
                kernel_size=4,
                stride=2,
                padding=1,
                bias=False,
            ),
            nn.BatchNorm2d(self.hidden_dim * 4),
            nn.LeakyReLU(0.2, inplace=False),
            nn.ConvTranspose2d(
                in_channels=self.hidden_dim * 4,
                out_channels=self.hidden_dim * 2,
                kernel_size=4,
                stride=2,
                padding=1,
                bias=False,
            ),
            nn.BatchNorm2d(self.hidden_dim * 2),
            nn.LeakyReLU(0.2, inplace=False),
            nn.ConvTranspose2d(
                in_channels=self.hidden_dim * 2,
                out_channels=self.hidden_dim,
                kernel_size=4,
                stride=2,
                padding=1,
                bias=False,
            ),
            nn.BatchNorm2d(self.hidden_dim),
            nn.LeakyReLU(0.2, inplace=False),
            nn.ConvTranspose2d(
                in_channels=self.hidden_dim,
                out_channels=self.nc,
                kernel_size=4,
                stride=2,
                padding=1,
                bias=False,
            ),
        )

    def _get_spatial_dim(self) -> int:
        return self.spatial_dim

    def encode(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
        """Encodes the input tensor x.

        Args:
            x: Input tensor
        Returns:
            The encoded latent space representation, or mu and logvar for VAEs.

        """
        h = self._encoder(x)
        # this makes sure we get the <batchsize, 1, latent_dim> shape for our latent space in the next step
        # because we put all dimensionaltiy in the second dimension of the output shape.
        # By covering all dimensionality here, we are sure that the rest is
        h = h.view(-1, self.hidden_dim * 8 * self.spatial_dim * self.spatial_dim)
        logvar = self.logvar(h)
        mu = self.mu(h)
        # prevent  mu and logvar from being too close to zero, this increased
        # numerical stability
        logvar = torch.clamp(logvar, 0.1, 20)
        # replace mu when mu < 0.00000001 with 0.1
        mu = torch.where(mu < 0.000001, torch.zeros_like(mu), mu)
        return mu, logvar

    def reparameterize(self, mu: torch.Tensor, logvar: torch.Tensor) -> torch.Tensor:
        """Reparameterization trick for VAE.

        Args:
             mu: mean of the latent distribution
             logvar: log-variance of the latent distribution
        Returns:
                z: sampled latent vector
        """
        std = torch.exp(0.5 * logvar)
        eps = torch.randn_like(std)
        return mu + eps * std

    def get_latent_space(self, x: torch.Tensor) -> torch.Tensor:
        """Returns the latent space representation of the input.

        Args:
            x: Input tensor
        Returns:
            Latent space representation

        """
        mu, logvar = self.encode(x)
        return self.reparameterize(mu, logvar)

    def decode(self, x: torch.Tensor) -> torch.Tensor:
        """Decode the latent tensor x
        Args:
            x: Latent tensor
        Returns:
            Decoded tensor, reconstructed from the latent space
        """
        h = self.d1(x)
        # here we do a similar thing as in the _encoder,
        # but instead of ensuring the correct dimension for the latent space,
        # we ensure the correct dimension for the first Conv2DTranspose layer
        # so we make sure that the last 3 dimension are (n_filters, reduced_img_dim, reduced_img_dim)
        h = h.view(-1, self.hidden_dim * 8, self.spatial_dim, self.spatial_dim)
        return self._decoder(h)

    def translate(self, z: torch.Tensor) -> torch.Tensor:
        """Reshapes the output to get actual images

        Args:
            z: Latent tensor
        Returns:
            Reconstructed image of shape (C,W,H)
        """
        out = self.decode(z)
        return out.view(-1, *self.img_shape)

    def forward(self, x: torch.Tensor) -> ModelOutput:
        """Forward pass of the model.
        Args:
            x: Input tensor
        Returns:
            ModelOutput object containing the reconstructed tensor and latent tensor
        """
        mu, logvar = self.encode(x)
        z = self.reparameterize(mu, logvar)
        return ModelOutput(
            reconstruction=self.translate(z),
            latentspace=z,
            latent_mean=mu,
            latent_logvar=logvar,
            additional_info=None,
        )

__init__(input_dim, config, ontologies=None, feature_order=None)

Initialize the ImageVAEArchitecture with the given configuration.

Parameters:

Name Type Description Default
input_dim Tuple[int, int, int]

(C,W,H) the input image shape

 required
config Optional[DefaultConfig]

Configuration object containing model parameters.

 required
hidden_dim

number of filters in the first convolutional layer

 required
Source code in src/autoencodix/modeling/_imagevae_architecture.py 
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
def __init__(
    self,
    input_dim: Tuple[int, int, int],  # (C,W,H) the input image shape
    config: Optional[DefaultConfig],
    ontologies: Optional[Union[Tuple, Dict]] = None,
    feature_order: Optional[Union[Tuple, Dict]] = None,
    # the input_dim is the number of channels in the image, e.g. 3
):
    """Initialize the ImageVAEArchitecture with the given configuration.

    Args:
        input_dim: (C,W,H) the input image shape
        config: Configuration object containing model parameters.
        hidden_dim: number of filters in the first convolutional layer
    """
    if config is None:
        config = DefaultConfig()
    self._config: DefaultConfig = config
    super().__init__(config=config, input_dim=input_dim)
    self.input_dim: int = input_dim
    self.latent_dim: int = self._config.latent_dim
    self.nc, self.h, self.w = input_dim
    self.img_shape: Tuple[int, int, int] = input_dim
    self.hidden_dim: int = self._config.hidden_dim
    self._build_network()
    self.apply(self._init_weights)

decode(x)

Decode the latent tensor x Args: x: Latent tensor Returns: Decoded tensor, reconstructed from the latent space

Source code in src/autoencodix/modeling/_imagevae_architecture.py 
255
256
257
258
259
260
261
262
263
264
265
266
267
268
def decode(self, x: torch.Tensor) -> torch.Tensor:
    """Decode the latent tensor x
    Args:
        x: Latent tensor
    Returns:
        Decoded tensor, reconstructed from the latent space
    """
    h = self.d1(x)
    # here we do a similar thing as in the _encoder,
    # but instead of ensuring the correct dimension for the latent space,
    # we ensure the correct dimension for the first Conv2DTranspose layer
    # so we make sure that the last 3 dimension are (n_filters, reduced_img_dim, reduced_img_dim)
    h = h.view(-1, self.hidden_dim * 8, self.spatial_dim, self.spatial_dim)
    return self._decoder(h)

encode(x)

Encodes the input tensor x.

Parameters:

Name Type Description Default
x Tensor

Input tensor

 required

Returns: The encoded latent space representation, or mu and logvar for VAEs.

Source code in src/autoencodix/modeling/_imagevae_architecture.py 
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
def encode(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
    """Encodes the input tensor x.

    Args:
        x: Input tensor
    Returns:
        The encoded latent space representation, or mu and logvar for VAEs.

    """
    h = self._encoder(x)
    # this makes sure we get the <batchsize, 1, latent_dim> shape for our latent space in the next step
    # because we put all dimensionaltiy in the second dimension of the output shape.
    # By covering all dimensionality here, we are sure that the rest is
    h = h.view(-1, self.hidden_dim * 8 * self.spatial_dim * self.spatial_dim)
    logvar = self.logvar(h)
    mu = self.mu(h)
    # prevent  mu and logvar from being too close to zero, this increased
    # numerical stability
    logvar = torch.clamp(logvar, 0.1, 20)
    # replace mu when mu < 0.00000001 with 0.1
    mu = torch.where(mu < 0.000001, torch.zeros_like(mu), mu)
    return mu, logvar

forward(x)

Forward pass of the model. Args: x: Input tensor Returns: ModelOutput object containing the reconstructed tensor and latent tensor

Source code in src/autoencodix/modeling/_imagevae_architecture.py 
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
def forward(self, x: torch.Tensor) -> ModelOutput:
    """Forward pass of the model.
    Args:
        x: Input tensor
    Returns:
        ModelOutput object containing the reconstructed tensor and latent tensor
    """
    mu, logvar = self.encode(x)
    z = self.reparameterize(mu, logvar)
    return ModelOutput(
        reconstruction=self.translate(z),
        latentspace=z,
        latent_mean=mu,
        latent_logvar=logvar,
        additional_info=None,
    )

get_latent_space(x)

Returns the latent space representation of the input.

Parameters:

Name Type Description Default
x Tensor

Input tensor

 required

Returns: Latent space representation

Source code in src/autoencodix/modeling/_imagevae_architecture.py 
243
244
245
246
247
248
249
250
251
252
253
def get_latent_space(self, x: torch.Tensor) -> torch.Tensor:
    """Returns the latent space representation of the input.

    Args:
        x: Input tensor
    Returns:
        Latent space representation

    """
    mu, logvar = self.encode(x)
    return self.reparameterize(mu, logvar)

reparameterize(mu, logvar)

Reparameterization trick for VAE.

Parameters:

Name Type Description Default
mu Tensor

mean of the latent distribution

 required
logvar Tensor

log-variance of the latent distribution

 required

Returns: z: sampled latent vector

Source code in src/autoencodix/modeling/_imagevae_architecture.py 
230
231
232
233
234
235
236
237
238
239
240
241
def reparameterize(self, mu: torch.Tensor, logvar: torch.Tensor) -> torch.Tensor:
    """Reparameterization trick for VAE.

    Args:
         mu: mean of the latent distribution
         logvar: log-variance of the latent distribution
    Returns:
            z: sampled latent vector
    """
    std = torch.exp(0.5 * logvar)
    eps = torch.randn_like(std)
    return mu + eps * std

translate(z)

Reshapes the output to get actual images

Parameters:

Name Type Description Default
z Tensor

Latent tensor

 required

Returns: Reconstructed image of shape (C,W,H)

Source code in src/autoencodix/modeling/_imagevae_architecture.py 
270
271
272
273
274
275
276
277
278
279
def translate(self, z: torch.Tensor) -> torch.Tensor:
    """Reshapes the output to get actual images

    Args:
        z: Latent tensor
    Returns:
        Reconstructed image of shape (C,W,H)
    """
    out = self.decode(z)
    return out.view(-1, *self.img_shape)

LayerFactory

Factory for creating configurable neural network layers.

Source code in src/autoencodix/modeling/_layer_factory.py 
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
class LayerFactory:
    """Factory for creating configurable neural network layers."""

    @staticmethod
    def get_layer_dimensions(
        feature_dim: int, latent_dim: int, n_layers: int, enc_factor: float
    ) -> List[int]:
        """Calculate progressive layer dimensions.

        Args:
        feature_dim: Input feature dimension
        latent_dim: Target latent dimension
        n_layers: Number of layers
        enc_factor: Reduction factor for layer sizes

        Returns:
            Calculated layer dimensions
        """
        if n_layers == 0:
            return [feature_dim, latent_dim]  # Direct projection from input to latent

        layer_dimensions = [feature_dim]
        for _ in range(n_layers):
            prev_layer_size = layer_dimensions[-1]
            next_layer_size = max(int(prev_layer_size / enc_factor), latent_dim)
            layer_dimensions.append(next_layer_size)
        layer_dimensions.append(latent_dim)

        return layer_dimensions

    @staticmethod
    def create_layer(
        in_features: int,
        out_features: int,
        dropout_p: float = 0.1,
        last_layer: bool = False,
    ) -> List[nn.Module]:
        """Create a configurable layer with optional components.

        Args:
            in_features: Input feature dimension
            out_features: Output feature dimension
            dropout_p: Dropout probability, by default 0.1
            last_layer: Flag to skip activation/dropout for final layer, by default False

        Returns:
            List of layer components
        """
        if last_layer:
            return [nn.Linear(in_features, out_features)]

        return [
            nn.Linear(in_features, out_features),
            nn.BatchNorm1d(out_features),
            nn.Dropout(dropout_p),
            nn.ReLU(),
        ]

create_layer(in_features, out_features, dropout_p=0.1, last_layer=False) staticmethod

Create a configurable layer with optional components.

Parameters:

Name Type Description Default
in_features int

Input feature dimension

 required
out_features int

Output feature dimension

 required
dropout_p float

Dropout probability, by default 0.1

 0.1
last_layer bool

Flag to skip activation/dropout for final layer, by default False

 False

Returns:

Type Description
List[Module]

List of layer components
Source code in src/autoencodix/modeling/_layer_factory.py 
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
@staticmethod
def create_layer(
    in_features: int,
    out_features: int,
    dropout_p: float = 0.1,
    last_layer: bool = False,
) -> List[nn.Module]:
    """Create a configurable layer with optional components.

    Args:
        in_features: Input feature dimension
        out_features: Output feature dimension
        dropout_p: Dropout probability, by default 0.1
        last_layer: Flag to skip activation/dropout for final layer, by default False

    Returns:
        List of layer components
    """
    if last_layer:
        return [nn.Linear(in_features, out_features)]

    return [
        nn.Linear(in_features, out_features),
        nn.BatchNorm1d(out_features),
        nn.Dropout(dropout_p),
        nn.ReLU(),
    ]

get_layer_dimensions(feature_dim, latent_dim, n_layers, enc_factor) staticmethod

Calculate progressive layer dimensions.

Args: feature_dim: Input feature dimension latent_dim: Target latent dimension n_layers: Number of layers enc_factor: Reduction factor for layer sizes

Returns:

Type Description
List[int]

Calculated layer dimensions
Source code in src/autoencodix/modeling/_layer_factory.py 
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
@staticmethod
def get_layer_dimensions(
    feature_dim: int, latent_dim: int, n_layers: int, enc_factor: float
) -> List[int]:
    """Calculate progressive layer dimensions.

    Args:
    feature_dim: Input feature dimension
    latent_dim: Target latent dimension
    n_layers: Number of layers
    enc_factor: Reduction factor for layer sizes

    Returns:
        Calculated layer dimensions
    """
    if n_layers == 0:
        return [feature_dim, latent_dim]  # Direct projection from input to latent

    layer_dimensions = [feature_dim]
    for _ in range(n_layers):
        prev_layer_size = layer_dimensions[-1]
        next_layer_size = max(int(prev_layer_size / enc_factor), latent_dim)
        layer_dimensions.append(next_layer_size)
    layer_dimensions.append(latent_dim)

    return layer_dimensions

MaskixArchitectureVanilla

Bases: BaseAutoencoder

Masked Autoencoder Architecture that follows https://doi.org/10.1093/bioinformatics/btae020

To closely mimic the publication, the network is not build with our LayerFactory as in other architectures.

Attributes:

Name Type Description
input_dim Union[int, Tuple[int, ...]]

number of input features
config

Configuration object containing model architecture parameters
encoder

Encoder network of the autoencoder
decoder

Decoder network of the autoencoder
Source code in src/autoencodix/modeling/_maskix_architecture.py 
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
class MaskixArchitectureVanilla(BaseAutoencoder):
    """Masked Autoencoder Architecture that follows https://doi.org/10.1093/bioinformatics/btae020

    To closely mimic the publication, the network is not build with our LayerFactory as in
    other architectures.

    Attributes:
        input_dim: number of input features
        config: Configuration object containing model architecture parameters
        encoder: Encoder network of the autoencoder
        decoder: Decoder network of the autoencoder


    """

    def __init__(
        self,
        config: Optional[DefaultConfig],
        input_dim: Union[int, Tuple[int, ...]],
        ontologies: Optional[Union[Tuple, Dict]] = None,
        feature_order: Optional[Union[Tuple, Dict]] = None,
    ):
        if config is None:
            config = DefaultConfig()
        self._config: DefaultConfig = config
        super().__init__(config, input_dim)
        self.input_dim: Union[int, Tuple[int, ...]] = input_dim
        if not isinstance(self.input_dim, int):
            raise TypeError(
                f"input dim needs to be int for MaskixArchitecture, got {type(self.input_dim)}"
            )
        self.latent_dim: int = self._config.latent_dim

        # populate self.encoder and self.decoder
        self._build_network()
        self.apply(self._init_weights)

    def _build_network(self):
        self._encoder = nn.Sequential(
            nn.Dropout(p=self.config.drop_p),
            nn.Linear(self.input_dim, self._config.maskix_hidden_dim),
            nn.LayerNorm(self._config.maskix_hidden_dim),
            nn.Mish(inplace=True),
            nn.Linear(self._config.maskix_hidden_dim, self.latent_dim),
            nn.LayerNorm(self.latent_dim),
            nn.Mish(inplace=True),
            nn.Linear(self.latent_dim, self.latent_dim),
        )

        self._mask_predictor = nn.Linear(self.latent_dim, self.input_dim)
        self._decoder = nn.Linear(
            in_features=self.latent_dim + self.input_dim, out_features=self.input_dim
        )

    def encode(self, x: torch.Tensor) -> torch.Tensor:
        """Encodes the input data.

        Args:
            x: input Tensor
        Returns:
            torch.Tensor

        """
        return self._encoder(x)

    def get_latent_space(self, x: torch.Tensor) -> torch.Tensor:
        """Returns the latent space representation of the input data.

        Args:
            x: input Tensor
        Returns:
            torch.Tensor

        """
        return self.encode(x)

    def decode(self, x: torch.Tensor) -> torch.Tensor:
        """Decodes the latent representation.

        Args:
            x: input Tensor
        Returns:
            torch.Tensor

        """
        return self._decoder(x)

    def forward_mask(self, x):
        latent = self.encoder(x)
        predicted_mask = self.mask_predictor(latent)
        reconstruction = self.decoder(torch.cat([latent, predicted_mask], dim=1))

        return latent, predicted_mask, reconstruction

    def forward(self, x: torch.Tensor) -> ModelOutput:
        latent: torch.Tensor = self.encode(x=x)
        predicted_mask: torch.Tensor = self._mask_predictor(latent)
        return ModelOutput(
            reconstruction=self.decode(torch.cat([latent, predicted_mask], dim=1)),
            latentspace=latent,
            latent_mean=None,
            latent_logvar=None,
            additional_info={"predicted_mask": predicted_mask},
        )

decode(x)

Decodes the latent representation.

Parameters:

Name Type Description Default
x Tensor

input Tensor

 required

Returns: torch.Tensor

Source code in src/autoencodix/modeling/_maskix_architecture.py 
85
86
87
88
89
90
91
92
93
94
def decode(self, x: torch.Tensor) -> torch.Tensor:
    """Decodes the latent representation.

    Args:
        x: input Tensor
    Returns:
        torch.Tensor

    """
    return self._decoder(x)

encode(x)

Encodes the input data.

Parameters:

Name Type Description Default
x Tensor

input Tensor

 required

Returns: torch.Tensor

Source code in src/autoencodix/modeling/_maskix_architecture.py 
63
64
65
66
67
68
69
70
71
72
def encode(self, x: torch.Tensor) -> torch.Tensor:
    """Encodes the input data.

    Args:
        x: input Tensor
    Returns:
        torch.Tensor

    """
    return self._encoder(x)

get_latent_space(x)

Returns the latent space representation of the input data.

Parameters:

Name Type Description Default
x Tensor

input Tensor

 required

Returns: torch.Tensor

Source code in src/autoencodix/modeling/_maskix_architecture.py 
74
75
76
77
78
79
80
81
82
83
def get_latent_space(self, x: torch.Tensor) -> torch.Tensor:
    """Returns the latent space representation of the input data.

    Args:
        x: input Tensor
    Returns:
        torch.Tensor

    """
    return self.encode(x)

OntixArchitecture

Bases: BaseAutoencoder

Ontology Autoencoder implementation with separate encoder and decoder construction.

Attributes: input_dim: number of input features config: Configuration object containing model architecture parameters _encoder: Encoder network of the autoencoder _decoder: Decoder network of the autoencoder mu: Linear layer to compute the mean of the latent distribution logvar: Linear layer to compute the log-variance of the latent distribution masks: Tuple of weight masks for the decoder layers based on ontology latent_dim: Dimension of the latent space, inferred from the first mask ontologies: Ontology information. feature_order: Order of features for input data.

Source code in src/autoencodix/modeling/_ontix_architecture.py 
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
class OntixArchitecture(BaseAutoencoder):
    """Ontology Autoencoder implementation with separate encoder and decoder construction.

    Attributes:
    input_dim: number of input features
    config: Configuration object containing model architecture parameters
    _encoder: Encoder network of the autoencoder
    _decoder: Decoder network of the autoencoder
    mu: Linear layer to compute the mean of the latent distribution
    logvar: Linear layer to compute the log-variance of the latent distribution
    masks: Tuple of weight masks for the decoder layers based on ontology
    latent_dim: Dimension of the latent space, inferred from the first mask
    ontologies: Ontology information.
    feature_order: Order of features for input data.

    """

    def __init__(
        self,
        config: Optional[Union[None, DefaultConfig]],
        input_dim: int,
        ontologies: tuple,
        feature_order: list,
    ) -> None:
        """Initialize the Vanilla Autoencoder with the given configuration.

        Args:
            config: Configuration object containing model parameters.
            input_dim: Number of input features.
            ontologies: Ontology information.
            feature_order: Order of features for input data.
        """
        if config is None:
            config = DefaultConfig()
        self._config = config
        super().__init__(config, input_dim)
        self.input_dim = input_dim
        self._mu: nn.Module
        self._logvar: nn.Module
        # create masks for sparse decoder
        self.ontologies = ontologies
        self.feature_order = feature_order
        self.masks = self._make_masks(config=self._config, feature_order=feature_order)
        self.latent_dim = self.masks[0].shape[1]
        print("Latent Dim: " + str(self.latent_dim))
        # populate self.encoder and self.decoder
        self._build_network()
        self.apply(self._init_weights)
        self._decoder.apply(
            self._positive_dec
        )  # Sparse decoder only has positive weights

        # Apply weight mask to create ontology-based decoder
        with torch.no_grad():
            # Check that the decoder has the same number of layers as masks
            if len(self.masks) != len(self._decoder):
                print(len(self.masks), len(self._decoder))
                print(self._encoder)
                print(self._decoder)
                raise ValueError(
                    "Number of masks does not match number of decoder layers"
                )
            else:
                for i, mask in enumerate(self.masks):
                    self._decoder[i].weight.mul_(mask)

    def _make_masks(
        self, config: DefaultConfig, feature_order: list
    ) -> Tuple[torch.Tensor, ...]:
        """Create masks for sparse decoder based on ontology via config

        Args:
            config: Configuration object containing model parameters
            feature_order: Order of features for input data

        Returns:
            Tuple containing the masks for the decoder network

        """
        # Read ontology from config

        masks = tuple()
        # feature_names are all values in the last ontology layer
        all_feature_names = set()
        for key, values in self.ontologies[-1].items():
            all_feature_names.update(values)
        all_feature_names = list(all_feature_names)
        print("Ontix checks:")
        print(f"All possible feature names length: {len(all_feature_names)}")
        print(f"Feature order length: {len(feature_order)}")
        # Check if all features in feature_order are present in all_feature_names
        feature_names = [f for f in feature_order]
        missing_features = [f for f in feature_order if f not in all_feature_names]
        if missing_features:
            print(
                f"Features in feature_order not found in all_feature_names: {missing_features}"
            )
        print(f"Feature names without filtering: {len(feature_names)}")

        # Enumerate through the ontologies
        for x, ont_dic in enumerate(self.ontologies):
            prev_lay_dim = len(ont_dic.keys())

            if x == len(self.ontologies) - 1:
                # fixed sort of feature list
                node_list = feature_names
            else:
                node_list = list(self.ontologies[x + 1].keys())
            next_lay_dim = len(node_list)
            # create masks for sparse decoder
            mask = torch.zeros(next_lay_dim, prev_lay_dim)
            p_int = 0
            if len(node_list) == next_lay_dim:
                if len(ont_dic.keys()) == prev_lay_dim:
                    for p_id in ont_dic:
                        feature_list = ont_dic[p_id]
                        for f_id in feature_list:
                            if f_id in node_list:
                                f_int = node_list.index(f_id)
                                mask[f_int, p_int] = 1

                        p_int += 1
                else:
                    print(
                        "Mask layer cannot be calculated. Ontology key list does not match previous layer dimension"
                    )
                    print("Returning zero mask")
            else:
                print(f"node list: {len(node_list)} vs next_lay_dim:{next_lay_dim}")
                print(
                    "Mask layer cannot be calculated. Output layer list does not match next layer dimension"
                )
                print("Returning zero mask")
            print(
                f"Mask layer {x} with shape {mask.shape} and {torch.sum(mask)} connections"
            )
            masks += (mask,)

        if torch.max(mask) < 1:
            print(
                "You provided an ontology with no connections between layers in the decoder. Please check your ontology definition."
            )

        return masks

    def _build_network(self) -> None:
        """Construct the encoder and decoder networks.

        Handles cases where `n_layers=0` by skipping the encoder and using only mu/logvar.
        """
        #### Encoder copied from varix architecture ####
        enc_dim = LayerFactory.get_layer_dimensions(
            feature_dim=self.input_dim,
            latent_dim=self.latent_dim,
            n_layers=self._config.n_layers,
            enc_factor=self._config.enc_factor,
        )
        #

        # Case 1: No Hidden Layers (Direct Mapping)
        self._encoder = nn.Sequential()
        self._mu = nn.Linear(self.input_dim, self.latent_dim)
        self._logvar = nn.Linear(self.input_dim, self.latent_dim)

        # Case 2: At Least One Hidden Layer
        if self._config.n_layers > 0:
            encoder_layers = []
            # print(enc_dim)
            for i, (in_features, out_features) in enumerate(
                zip(enc_dim[:-1], enc_dim[1:])
            ):
                # since we add mu and logvar, we will remove the last layer
                if i == len(enc_dim) - 2:
                    break
                encoder_layers.extend(
                    LayerFactory.create_layer(
                        in_features=in_features,
                        out_features=out_features,
                        dropout_p=self._config.drop_p,
                        last_layer=False,  # only for decoder relevant
                    )
                )

            self._encoder = nn.Sequential(*encoder_layers)
            self._mu = nn.Linear(enc_dim[-2], self.latent_dim)
            self._logvar = nn.Linear(enc_dim[-2], self.latent_dim)
        #### Encoder copied from varix architecture ####

        # Construct Decoder with Sparse Connections via masks
        # Decoder dimension is determined by the masks
        dec_dim = [self.latent_dim] + [
            mask.shape[0] for mask in self.masks
        ]  # + [self.input_dim]
        decoder_layers = []
        for i, (in_features, out_features) in enumerate(zip(dec_dim[:-1], dec_dim[1:])):
            # last_layer = i == len(dec_dim) - 2
            last_layer = True  ## Only linear layers in sparse decoder
            decoder_layers.extend(
                LayerFactory.create_layer(
                    in_features=in_features,
                    out_features=out_features,
                    dropout_p=0,  ## No dropout in sparse decoder
                    last_layer=last_layer,
                    # only_linear=True,
                )
            )

        self._decoder = nn.Sequential(*decoder_layers)

    def _positive_dec(self, m):
        if isinstance(m, nn.Linear):
            m.weight.data = m.weight.data.clamp(min=0)

    def encode(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
        """Encode the input tensor x

        Args:
            x: Input tensor

        Returns:
            Encoded tensor
        """
        latent = x  # for case where n_layers=0
        if len(self._encoder) > 0:
            latent = self._encoder(x)
        mu = self._mu(latent)
        logvar = self._logvar(latent)
        # numeric stability
        logvar = torch.clamp(logvar, 0.01, 20)
        mu = torch.where(mu < 0.0000001, torch.zeros_like(mu), mu)
        return mu, logvar

    def reparameterize(self, mu: torch.Tensor, logvar: torch.Tensor) -> torch.Tensor:
        """Reparameterization trick for VAE

        Args:
            mu: Mean tensor
            logvar: Log variance tensor

        Returns:
            Reparameterized latent tensor
        """
        std = torch.exp(0.5 * logvar)
        eps = torch.randn_like(std)
        return mu + eps * std

    def decode(self, x: torch.Tensor) -> torch.Tensor:
        """Decode the latent tensor x

        Args:
            x: Latent tensor

        Returns:
            torch.Tensor
            Decoded tensor

        """
        return self._decoder(x)

    def forward(self, x: torch.Tensor) -> ModelOutput:
        """Forward pass of the model, fill

        Args:
            x: Input tensor

        Returns:
            ModelOutput object containing the reconstructed tensor and latent tensor

        """
        mu, logvar = self.encode(x)
        z = self.reparameterize(mu, logvar)
        x_hat = self.decode(z)
        return ModelOutput(
            reconstruction=x_hat,
            latentspace=z,
            latent_mean=mu,
            latent_logvar=logvar,
            additional_info=None,
        )

    def get_latent_space(self, x: torch.Tensor) -> torch.Tensor:
        """Returns the latent space representation of the input.

        Args:
            x: Input tensor

        Returns:
            Latent space representation

        """
        mu, logvar = self.encode(x)
        z = self.reparameterize(mu, logvar)
        return z

__init__(config, input_dim, ontologies, feature_order)

Initialize the Vanilla Autoencoder with the given configuration.

Parameters:

Name Type Description Default
config Optional[Union[None, DefaultConfig]]

Configuration object containing model parameters.

 required
input_dim int

Number of input features.

 required
ontologies tuple

Ontology information.

 required
feature_order list

Order of features for input data.

 required
Source code in src/autoencodix/modeling/_ontix_architecture.py 
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
def __init__(
    self,
    config: Optional[Union[None, DefaultConfig]],
    input_dim: int,
    ontologies: tuple,
    feature_order: list,
) -> None:
    """Initialize the Vanilla Autoencoder with the given configuration.

    Args:
        config: Configuration object containing model parameters.
        input_dim: Number of input features.
        ontologies: Ontology information.
        feature_order: Order of features for input data.
    """
    if config is None:
        config = DefaultConfig()
    self._config = config
    super().__init__(config, input_dim)
    self.input_dim = input_dim
    self._mu: nn.Module
    self._logvar: nn.Module
    # create masks for sparse decoder
    self.ontologies = ontologies
    self.feature_order = feature_order
    self.masks = self._make_masks(config=self._config, feature_order=feature_order)
    self.latent_dim = self.masks[0].shape[1]
    print("Latent Dim: " + str(self.latent_dim))
    # populate self.encoder and self.decoder
    self._build_network()
    self.apply(self._init_weights)
    self._decoder.apply(
        self._positive_dec
    )  # Sparse decoder only has positive weights

    # Apply weight mask to create ontology-based decoder
    with torch.no_grad():
        # Check that the decoder has the same number of layers as masks
        if len(self.masks) != len(self._decoder):
            print(len(self.masks), len(self._decoder))
            print(self._encoder)
            print(self._decoder)
            raise ValueError(
                "Number of masks does not match number of decoder layers"
            )
        else:
            for i, mask in enumerate(self.masks):
                self._decoder[i].weight.mul_(mask)

decode(x)

Decode the latent tensor x

Parameters:

Name Type Description Default
x Tensor

Latent tensor

 required

Returns:

Type Description
Tensor

torch.Tensor
Tensor

Decoded tensor
Source code in src/autoencodix/modeling/_ontix_architecture.py 
259
260
261
262
263
264
265
266
267
268
269
270
def decode(self, x: torch.Tensor) -> torch.Tensor:
    """Decode the latent tensor x

    Args:
        x: Latent tensor

    Returns:
        torch.Tensor
        Decoded tensor

    """
    return self._decoder(x)

encode(x)

Encode the input tensor x

Parameters:

Name Type Description Default
x Tensor

Input tensor

 required

Returns:

Type Description
Tuple[Tensor, Tensor]

Encoded tensor
Source code in src/autoencodix/modeling/_ontix_architecture.py 
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
def encode(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
    """Encode the input tensor x

    Args:
        x: Input tensor

    Returns:
        Encoded tensor
    """
    latent = x  # for case where n_layers=0
    if len(self._encoder) > 0:
        latent = self._encoder(x)
    mu = self._mu(latent)
    logvar = self._logvar(latent)
    # numeric stability
    logvar = torch.clamp(logvar, 0.01, 20)
    mu = torch.where(mu < 0.0000001, torch.zeros_like(mu), mu)
    return mu, logvar

forward(x)

Forward pass of the model, fill

Parameters:

Name Type Description Default
x Tensor

Input tensor

 required

Returns:

Type Description
ModelOutput

ModelOutput object containing the reconstructed tensor and latent tensor
Source code in src/autoencodix/modeling/_ontix_architecture.py 
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
def forward(self, x: torch.Tensor) -> ModelOutput:
    """Forward pass of the model, fill

    Args:
        x: Input tensor

    Returns:
        ModelOutput object containing the reconstructed tensor and latent tensor

    """
    mu, logvar = self.encode(x)
    z = self.reparameterize(mu, logvar)
    x_hat = self.decode(z)
    return ModelOutput(
        reconstruction=x_hat,
        latentspace=z,
        latent_mean=mu,
        latent_logvar=logvar,
        additional_info=None,
    )

get_latent_space(x)

Returns the latent space representation of the input.

Parameters:

Name Type Description Default
x Tensor

Input tensor

 required

Returns:

Type Description
Tensor

Latent space representation
Source code in src/autoencodix/modeling/_ontix_architecture.py 
293
294
295
296
297
298
299
300
301
302
303
304
305
def get_latent_space(self, x: torch.Tensor) -> torch.Tensor:
    """Returns the latent space representation of the input.

    Args:
        x: Input tensor

    Returns:
        Latent space representation

    """
    mu, logvar = self.encode(x)
    z = self.reparameterize(mu, logvar)
    return z

reparameterize(mu, logvar)

Reparameterization trick for VAE

Parameters:

Name Type Description Default
mu Tensor

Mean tensor

 required
logvar Tensor

Log variance tensor

 required

Returns:

Type Description
Tensor

Reparameterized latent tensor
Source code in src/autoencodix/modeling/_ontix_architecture.py 
245
246
247
248
249
250
251
252
253
254
255
256
257
def reparameterize(self, mu: torch.Tensor, logvar: torch.Tensor) -> torch.Tensor:
    """Reparameterization trick for VAE

    Args:
        mu: Mean tensor
        logvar: Log variance tensor

    Returns:
        Reparameterized latent tensor
    """
    std = torch.exp(0.5 * logvar)
    eps = torch.randn_like(std)
    return mu + eps * std

VanillixArchitecture

Bases: BaseAutoencoder

Vanilla Autoencoder implementation with separate encoder and decoder construction.

Attributes:

Name Type Description
input_dim

number of input features
config

Configuration object containing model architecture parameters
encoder

Encoder network of the autoencoder
decoder

Decoder network of the autoencoder
Source code in src/autoencodix/modeling/_vanillix_architecture.py 
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
class VanillixArchitecture(BaseAutoencoder):
    """Vanilla Autoencoder implementation with separate encoder and decoder construction.

    Attributes:
        input_dim: number of input features
        config: Configuration object containing model architecture parameters
        encoder: Encoder network of the autoencoder
        decoder: Decoder network of the autoencoder

    """

    def __init__(
        self,
        config: Optional[Union[None, DefaultConfig]],
        input_dim: int,
        ontologies: Optional[Union[Tuple, Dict]] = None,
        feature_order: Optional[Union[Tuple, Dict]] = None,
    ) -> None:
        """Initialize the Vanilla Autoencoder with the given configuration.

        Args:
            config: Configuration object containing model parameters.
            input_dim: Number of input features.
        """

        if config is None:
            config = DefaultConfig()
        self._config = config
        super().__init__(config, input_dim)
        self.input_dim = input_dim

        # populate self.encoder and self.decoder
        self._build_network()
        self.apply(self._init_weights)

    def _build_network(self) -> None:
        """Construct the encoder with linear layers."""
        # Calculate layer dimensions
        enc_dim = LayerFactory.get_layer_dimensions(
            feature_dim=self.input_dim,
            latent_dim=self._config.latent_dim,
            n_layers=self._config.n_layers,
            enc_factor=self._config.enc_factor,
        )

        encoder_layers = []
        for i, (in_features, out_features) in enumerate(zip(enc_dim[:-1], enc_dim[1:])):
            last_layer = i == len(enc_dim) - 2
            encoder_layers.extend(
                LayerFactory.create_layer(
                    in_features=in_features,
                    out_features=out_features,
                    dropout_p=self._config.drop_p,
                    last_layer=last_layer,
                )
            )

        dec_dim = enc_dim[::-1]  # Reverse the dimensions and copy
        decoder_layers = []
        for i, (in_features, out_features) in enumerate(zip(dec_dim[:-1], dec_dim[1:])):
            last_layer = i == len(dec_dim) - 2
            decoder_layers.extend(
                LayerFactory.create_layer(
                    in_features=in_features,
                    out_features=out_features,
                    dropout_p=self._config.drop_p,
                    last_layer=last_layer,
                )
            )
        self._encoder = nn.Sequential(*encoder_layers)
        self._decoder = nn.Sequential(*decoder_layers)

    def encode(self, x: torch.Tensor) -> torch.Tensor:
        """Encodes the input data.

        Args:
            x: input Tensor
        Returns:
            torch.Tensor

        """
        return self._encoder(x)

    def get_latent_space(self, x: torch.Tensor) -> torch.Tensor:
        """Returns the latent space representation of the input data.

        Args:
            x: input Tensor
        Returns:
            torch.Tensor

        """
        return self.encode(x)

    def decode(self, x: torch.Tensor) -> torch.Tensor:
        """Decodes the latent representation.

        Args:
            x: input Tensor
        Returns:
            torch.Tensor

        """
        return self._decoder(x)

    def forward(self, x: torch.Tensor) -> ModelOutput:
        """Forward pass of the model.

        Args:
            x: input Tensor
        Returns:
            ModelOutput

        """
        latent = self.encode(x)
        return ModelOutput(
            reconstruction=self.decode(latent),
            latentspace=latent,
            latent_mean=None,
            latent_logvar=None,
            additional_info=None,
        )

__init__(config, input_dim, ontologies=None, feature_order=None)

Initialize the Vanilla Autoencoder with the given configuration.

Parameters:

Name Type Description Default
config Optional[Union[None, DefaultConfig]]

Configuration object containing model parameters.

 required
input_dim int

Number of input features.

 required
Source code in src/autoencodix/modeling/_vanillix_architecture.py 
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
def __init__(
    self,
    config: Optional[Union[None, DefaultConfig]],
    input_dim: int,
    ontologies: Optional[Union[Tuple, Dict]] = None,
    feature_order: Optional[Union[Tuple, Dict]] = None,
) -> None:
    """Initialize the Vanilla Autoencoder with the given configuration.

    Args:
        config: Configuration object containing model parameters.
        input_dim: Number of input features.
    """

    if config is None:
        config = DefaultConfig()
    self._config = config
    super().__init__(config, input_dim)
    self.input_dim = input_dim

    # populate self.encoder and self.decoder
    self._build_network()
    self.apply(self._init_weights)

decode(x)

Decodes the latent representation.

Parameters:

Name Type Description Default
x Tensor

input Tensor

 required

Returns: torch.Tensor

Source code in src/autoencodix/modeling/_vanillix_architecture.py 
109
110
111
112
113
114
115
116
117
118
def decode(self, x: torch.Tensor) -> torch.Tensor:
    """Decodes the latent representation.

    Args:
        x: input Tensor
    Returns:
        torch.Tensor

    """
    return self._decoder(x)

encode(x)

Encodes the input data.

Parameters:

Name Type Description Default
x Tensor

input Tensor

 required

Returns: torch.Tensor

Source code in src/autoencodix/modeling/_vanillix_architecture.py 
87
88
89
90
91
92
93
94
95
96
def encode(self, x: torch.Tensor) -> torch.Tensor:
    """Encodes the input data.

    Args:
        x: input Tensor
    Returns:
        torch.Tensor

    """
    return self._encoder(x)

forward(x)

Forward pass of the model.

Parameters:

Name Type Description Default
x Tensor

input Tensor

 required

Returns: ModelOutput

Source code in src/autoencodix/modeling/_vanillix_architecture.py 
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
def forward(self, x: torch.Tensor) -> ModelOutput:
    """Forward pass of the model.

    Args:
        x: input Tensor
    Returns:
        ModelOutput

    """
    latent = self.encode(x)
    return ModelOutput(
        reconstruction=self.decode(latent),
        latentspace=latent,
        latent_mean=None,
        latent_logvar=None,
        additional_info=None,
    )

get_latent_space(x)

Returns the latent space representation of the input data.

Parameters:

Name Type Description Default
x Tensor

input Tensor

 required

Returns: torch.Tensor

Source code in src/autoencodix/modeling/_vanillix_architecture.py 
 98
 99
100
101
102
103
104
105
106
107
def get_latent_space(self, x: torch.Tensor) -> torch.Tensor:
    """Returns the latent space representation of the input data.

    Args:
        x: input Tensor
    Returns:
        torch.Tensor

    """
    return self.encode(x)

VarixArchitecture

Bases: BaseAutoencoder

Variational Autoencoder implementation with separate encoder and decoder construction.

Attributes:

Name Type Description
input_dim int

number of input features
config

Configuration object containing model architecture parameters
encoder

Encoder network of the autoencoder
decoder

Decoder network of the autoencoder
mu

Linear layer to compute the mean of the latent distribution

logvar: Linear layer to compute the log-variance of the latent distribution

Source code in src/autoencodix/modeling/_varix_architecture.py 
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
class VarixArchitecture(BaseAutoencoder):
    """Variational Autoencoder implementation with separate encoder and decoder construction.

    Attributes:
        input_dim: number of input features
        config: Configuration object containing model architecture parameters
        encoder: Encoder network of the autoencoder
        decoder: Decoder network of the autoencoder
        mu: Linear layer to compute the mean of the latent distribution
    logvar: Linear layer to compute the log-variance of the latent distribution

    """

    def __init__(
        self,
        config: Optional[Union[None, DefaultConfig]],
        input_dim: int,
        ontologies: Optional[Union[Tuple, Dict]] = None,
        feature_order: Optional[Union[Tuple, Dict]] = None,
    ) -> None:
        """Initialize the Vanilla Autoencoder with the given configuration.

        Args:
            config: Configuration object containing model parameters.
            input_dim: Number of input features.
        """

        if config is None:
            config = DefaultConfig()
        self._config: DefaultConfig = config
        super().__init__(config=config, input_dim=input_dim)
        self.input_dim: int = input_dim
        self._mu: nn.Module
        self._logvar: nn.Module

        # populate self.encoder and self.decoder
        self._build_network()
        self.apply(self._init_weights)

    def _build_network(self) -> None:
        """Construct the encoder and decoder networks.

        Handles cases where `n_layers=0` by skipping the encoder and using only mu/logvar.
        """
        enc_dim = LayerFactory.get_layer_dimensions(
            feature_dim=self.input_dim,
            latent_dim=self._config.latent_dim,
            n_layers=self._config.n_layers,
            enc_factor=self._config.enc_factor,
        )
        #

        # Case 1: No Hidden Layers (Direct Mapping)
        self._encoder = nn.Sequential()
        self._mu = nn.Linear(self.input_dim, self._config.latent_dim)
        self._logvar = nn.Linear(self.input_dim, self._config.latent_dim)

        # Case 2: At Least One Hidden Layer
        if self._config.n_layers > 0:
            encoder_layers = []
            # print(enc_dim)
            for i, (in_features, out_features) in enumerate(
                zip(enc_dim[:-1], enc_dim[1:])
            ):
                # since we add mu and logvar, we will remove the last layer
                if i == len(enc_dim) - 2:
                    break
                encoder_layers.extend(
                    LayerFactory.create_layer(
                        in_features=in_features,
                        out_features=out_features,
                        dropout_p=self._config.drop_p,
                        last_layer=False,  # only for decoder relevant
                    )
                )

            self._encoder = nn.Sequential(*encoder_layers)
            self._mu = nn.Linear(enc_dim[-2], self._config.latent_dim)
            self._logvar = nn.Linear(enc_dim[-2], self._config.latent_dim)

        # Construct Decoder (Same for Both Cases)
        dec_dim = enc_dim[::-1]  # Reverse the dimensions and copy
        decoder_layers = []
        for i, (in_features, out_features) in enumerate(zip(dec_dim[:-1], dec_dim[1:])):
            last_layer = i == len(dec_dim) - 2
            decoder_layers.extend(
                LayerFactory.create_layer(
                    in_features=in_features,
                    out_features=out_features,
                    dropout_p=self._config.drop_p,
                    last_layer=last_layer,
                )
            )

        self._decoder = nn.Sequential(*decoder_layers)

    def encode(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
        """Encode the input tensor x.

        Args:
            x: Input tensor

        Returns:
            Encoded tensor

        """

        latent = x  # for case where n_layers=0
        if len(self._encoder) > 0:
            latent = self._encoder(x)
        mu = self._mu(latent)
        logvar = self._logvar(latent)
        # numeric stability
        logvar = torch.clamp(logvar, 0.01, 20)
        mu = torch.where(mu < 0.0000001, torch.zeros_like(mu), mu)
        return mu, logvar

    def reparameterize(self, mu: torch.Tensor, logvar: torch.Tensor) -> torch.Tensor:
        """Reparameterization trick for VAE

        Args:
            mu: torch.Tensor
            logvar: torch.Tensor

        Returns:
            torch.Tensor

        """
        std = torch.exp(0.5 * logvar)
        eps = torch.randn_like(std)
        return mu + eps * std

    def get_latent_space(self, x: torch.Tensor) -> torch.Tensor:
        """Returns the latent space representation of the input.

        Args:
            x: Input tensor

        Returns:
            Latent space representation

        """
        mu, logvar = self.encode(x)
        z = self.reparameterize(mu, logvar)
        return z

    def decode(self, x: torch.Tensor) -> torch.Tensor:
        """Decode the latent tensor x

        Args:
            x: Latent tensor

        Returns:
            Decoded tensor
        """

        return self._decoder(x)

    def forward(self, x: torch.Tensor) -> ModelOutput:
        """Forward pass of the model, fill

        Args:
            x: Input tensor

        Returns:
            ModelOutput object containing the reconstructed tensor and latent tensor

        """
        mu, logvar = self.encode(x)
        z = self.reparameterize(mu, logvar)
        x_hat = self.decode(z)
        return ModelOutput(
            reconstruction=x_hat,
            latentspace=z,
            latent_mean=mu,
            latent_logvar=logvar,
            additional_info=None,
        )

__init__(config, input_dim, ontologies=None, feature_order=None)

Initialize the Vanilla Autoencoder with the given configuration.

Parameters:

Name Type Description Default
config Optional[Union[None, DefaultConfig]]

Configuration object containing model parameters.

 required
input_dim int

Number of input features.

 required
Source code in src/autoencodix/modeling/_varix_architecture.py 
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
def __init__(
    self,
    config: Optional[Union[None, DefaultConfig]],
    input_dim: int,
    ontologies: Optional[Union[Tuple, Dict]] = None,
    feature_order: Optional[Union[Tuple, Dict]] = None,
) -> None:
    """Initialize the Vanilla Autoencoder with the given configuration.

    Args:
        config: Configuration object containing model parameters.
        input_dim: Number of input features.
    """

    if config is None:
        config = DefaultConfig()
    self._config: DefaultConfig = config
    super().__init__(config=config, input_dim=input_dim)
    self.input_dim: int = input_dim
    self._mu: nn.Module
    self._logvar: nn.Module

    # populate self.encoder and self.decoder
    self._build_network()
    self.apply(self._init_weights)

decode(x)

Decode the latent tensor x

Parameters:

Name Type Description Default
x Tensor

Latent tensor

 required

Returns:

Type Description
Tensor

Decoded tensor
Source code in src/autoencodix/modeling/_varix_architecture.py 
159
160
161
162
163
164
165
166
167
168
169
def decode(self, x: torch.Tensor) -> torch.Tensor:
    """Decode the latent tensor x

    Args:
        x: Latent tensor

    Returns:
        Decoded tensor
    """

    return self._decoder(x)

encode(x)

Encode the input tensor x.

Parameters:

Name Type Description Default
x Tensor

Input tensor

 required

Returns:

Type Description
Tuple[Tensor, Tensor]

Encoded tensor
Source code in src/autoencodix/modeling/_varix_architecture.py 
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
def encode(self, x: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
    """Encode the input tensor x.

    Args:
        x: Input tensor

    Returns:
        Encoded tensor

    """

    latent = x  # for case where n_layers=0
    if len(self._encoder) > 0:
        latent = self._encoder(x)
    mu = self._mu(latent)
    logvar = self._logvar(latent)
    # numeric stability
    logvar = torch.clamp(logvar, 0.01, 20)
    mu = torch.where(mu < 0.0000001, torch.zeros_like(mu), mu)
    return mu, logvar

forward(x)

Forward pass of the model, fill

Parameters:

Name Type Description Default
x Tensor

Input tensor

 required

Returns:

Type Description
ModelOutput

ModelOutput object containing the reconstructed tensor and latent tensor
Source code in src/autoencodix/modeling/_varix_architecture.py 
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
def forward(self, x: torch.Tensor) -> ModelOutput:
    """Forward pass of the model, fill

    Args:
        x: Input tensor

    Returns:
        ModelOutput object containing the reconstructed tensor and latent tensor

    """
    mu, logvar = self.encode(x)
    z = self.reparameterize(mu, logvar)
    x_hat = self.decode(z)
    return ModelOutput(
        reconstruction=x_hat,
        latentspace=z,
        latent_mean=mu,
        latent_logvar=logvar,
        additional_info=None,
    )

get_latent_space(x)

Returns the latent space representation of the input.

Parameters:

Name Type Description Default
x Tensor

Input tensor

 required

Returns:

Type Description
Tensor

Latent space representation
Source code in src/autoencodix/modeling/_varix_architecture.py 
145
146
147
148
149
150
151
152
153
154
155
156
157
def get_latent_space(self, x: torch.Tensor) -> torch.Tensor:
    """Returns the latent space representation of the input.

    Args:
        x: Input tensor

    Returns:
        Latent space representation

    """
    mu, logvar = self.encode(x)
    z = self.reparameterize(mu, logvar)
    return z

reparameterize(mu, logvar)

Reparameterization trick for VAE

Parameters:

Name Type Description Default
mu Tensor

torch.Tensor

 required
logvar Tensor

torch.Tensor

 required

Returns:

Type Description
Tensor

torch.Tensor
Source code in src/autoencodix/modeling/_varix_architecture.py 
130
131
132
133
134
135
136
137
138
139
140
141
142
143
def reparameterize(self, mu: torch.Tensor, logvar: torch.Tensor) -> torch.Tensor:
    """Reparameterization trick for VAE

    Args:
        mu: torch.Tensor
        logvar: torch.Tensor

    Returns:
        torch.Tensor

    """
    std = torch.exp(0.5 * logvar)
    eps = torch.randn_like(std)
    return mu + eps * std