Note: This environment variable is required for fully deterministic CuBLAS ops on CUDA >= 10.2 when reproducible=True is set below. Without it, PyTorch raises a RuntimeError instead of training deterministically. It must be set before torch is imported. See the README FAQ for details.

In [ ]:

%env CUBLAS_WORKSPACE_CONFIG=:16:8

%env CUBLAS_WORKSPACE_CONFIG=:16:8

Overview of visualization options¶

This tutorial gives a comprehensive overview of in-built visualization functions and options.

Content:

1. Model & training
   • Model weight visualization
   • Loss curves
2. Latent space
   • Heatmap representation
   • Ridgeline representation
   • 2D representation
   • Coverage and total correlation
3. Embedding evaluation
   • Performance comparison via bar plots
4. Saving and customization
5. XModalix and Imagix specialities
   • Differences in standard visualizations
   • Input vs. translation 2D check
   • Visualizing image translation capabilities

1. Model & training¶

We provide a heatmap representation of model weights and loss curves over epochs to gain insights into the autoencoder training and debugging.

Model weight visualization is limited to small models (input features <3000) for computational and visual reasons.

Let us start with a Disentanglix to show model weight and loss curve plotting.

In [1]:

from autoencodix.utils.example_data import EXAMPLE_MULTI_BULK
from autoencodix.configs.disentanglix_config import DisentanglixConfig
from autoencodix.configs.default_config import DataCase
import autoencodix as acx


my_cfg = DisentanglixConfig(
	data_case=DataCase.MULTI_BULK,
	loss_reduction="sum",
	k_filter=80,
	latent_dim=6,
	scaling="STANDARD",
	epochs=100,
	learning_rate=0.001,
	batch_size = 128,
	beta_mi = 10,
	beta_tc = 500,
	beta_dimKL= 0.5,
	use_mss = True,
	drop_p =0.1,
	global_seed=42,
	checkpoint_interval=1,
	n_layers=1)

disent = acx.Disentanglix(data=EXAMPLE_MULTI_BULK, config=my_cfg)

result = disent.run()

from autoencodix.utils.example_data import EXAMPLE_MULTI_BULK from autoencodix.configs.disentanglix_config import DisentanglixConfig from autoencodix.configs.default_config import DataCase import autoencodix as acx my_cfg = DisentanglixConfig( data_case=DataCase.MULTI_BULK, loss_reduction="sum", k_filter=80, latent_dim=6, scaling="STANDARD", epochs=100, learning_rate=0.001, batch_size = 128, beta_mi = 10, beta_tc = 500, beta_dimKL= 0.5, use_mss = True, drop_p =0.1, global_seed=42, checkpoint_interval=1, n_layers=1) disent = acx.Disentanglix(data=EXAMPLE_MULTI_BULK, config=my_cfg) result = disent.run()

in handle_direct_user_data with data: <class 'autoencodix.data.datapackage.DataPackage'>
anno key: transcriptomics
anno key: proteomics
Epoch 1 - Train Loss: 96.6301
Sub-losses: recon_loss: 96.6291, mut_info_loss: 0.0003, tot_corr_loss: 0.0005, dimwise_kl_loss: 0.0001, anneal_factor: 0.0000, effective_beta_mi_factor: 0.0005, effective_beta_tc_factor: 0.0227, effective_beta_dimKL_factor: 0.0000
Epoch 1 - Valid Loss: 90.4680
Sub-losses: recon_loss: 90.4664, mut_info_loss: 0.0003, tot_corr_loss: 0.0013, dimwise_kl_loss: 0.0000, anneal_factor: 0.0000, effective_beta_mi_factor: 0.0005, effective_beta_tc_factor: 0.0227, effective_beta_dimKL_factor: 0.0000
Epoch 2 - Train Loss: 94.7958
Sub-losses: recon_loss: 94.7943, mut_info_loss: 0.0004, tot_corr_loss: 0.0010, dimwise_kl_loss: 0.0001, anneal_factor: 0.0001, effective_beta_mi_factor: 0.0006, effective_beta_tc_factor: 0.0277, effective_beta_dimKL_factor: 0.0000
Epoch 2 - Valid Loss: 88.7186
Sub-losses: recon_loss: 88.7153, mut_info_loss: 0.0004, tot_corr_loss: 0.0028, dimwise_kl_loss: 0.0000, anneal_factor: 0.0001, effective_beta_mi_factor: 0.0006, effective_beta_tc_factor: 0.0277, effective_beta_dimKL_factor: 0.0000
Epoch 3 - Train Loss: 94.3066
Sub-losses: recon_loss: 94.3044, mut_info_loss: 0.0005, tot_corr_loss: 0.0017, dimwise_kl_loss: 0.0001, anneal_factor: 0.0001, effective_beta_mi_factor: 0.0007, effective_beta_tc_factor: 0.0339, effective_beta_dimKL_factor: 0.0000
Epoch 3 - Valid Loss: 86.3774
Sub-losses: recon_loss: 86.3727, mut_info_loss: 0.0004, tot_corr_loss: 0.0043, dimwise_kl_loss: 0.0001, anneal_factor: 0.0001, effective_beta_mi_factor: 0.0007, effective_beta_tc_factor: 0.0339, effective_beta_dimKL_factor: 0.0000
Epoch 4 - Train Loss: 92.6062
Sub-losses: recon_loss: 92.6019, mut_info_loss: 0.0007, tot_corr_loss: 0.0034, dimwise_kl_loss: 0.0001, anneal_factor: 0.0001, effective_beta_mi_factor: 0.0008, effective_beta_tc_factor: 0.0414, effective_beta_dimKL_factor: 0.0000
Epoch 4 - Valid Loss: 85.0198
Sub-losses: recon_loss: 85.0170, mut_info_loss: 0.0005, tot_corr_loss: 0.0022, dimwise_kl_loss: 0.0001, anneal_factor: 0.0001, effective_beta_mi_factor: 0.0008, effective_beta_tc_factor: 0.0414, effective_beta_dimKL_factor: 0.0000
Epoch 5 - Train Loss: 91.3575
Sub-losses: recon_loss: 91.3542, mut_info_loss: 0.0008, tot_corr_loss: 0.0024, dimwise_kl_loss: 0.0001, anneal_factor: 0.0001, effective_beta_mi_factor: 0.0010, effective_beta_tc_factor: 0.0505, effective_beta_dimKL_factor: 0.0001
Epoch 5 - Valid Loss: 86.2906
Sub-losses: recon_loss: 86.2860, mut_info_loss: 0.0007, tot_corr_loss: 0.0037, dimwise_kl_loss: 0.0001, anneal_factor: 0.0001, effective_beta_mi_factor: 0.0010, effective_beta_tc_factor: 0.0505, effective_beta_dimKL_factor: 0.0001
Epoch 6 - Train Loss: 90.0469
Sub-losses: recon_loss: 90.0414, mut_info_loss: 0.0009, tot_corr_loss: 0.0044, dimwise_kl_loss: 0.0002, anneal_factor: 0.0001, effective_beta_mi_factor: 0.0012, effective_beta_tc_factor: 0.0617, effective_beta_dimKL_factor: 0.0001
Epoch 6 - Valid Loss: 84.3421
Sub-losses: recon_loss: 84.3376, mut_info_loss: 0.0008, tot_corr_loss: 0.0035, dimwise_kl_loss: 0.0001, anneal_factor: 0.0001, effective_beta_mi_factor: 0.0012, effective_beta_tc_factor: 0.0617, effective_beta_dimKL_factor: 0.0001
Epoch 7 - Train Loss: 88.8228
Sub-losses: recon_loss: 88.8126, mut_info_loss: 0.0012, tot_corr_loss: 0.0089, dimwise_kl_loss: 0.0002, anneal_factor: 0.0002, effective_beta_mi_factor: 0.0015, effective_beta_tc_factor: 0.0754, effective_beta_dimKL_factor: 0.0001
Epoch 7 - Valid Loss: 84.7706
Sub-losses: recon_loss: 84.7691, mut_info_loss: 0.0011, tot_corr_loss: 0.0003, dimwise_kl_loss: 0.0002, anneal_factor: 0.0002, effective_beta_mi_factor: 0.0015, effective_beta_tc_factor: 0.0754, effective_beta_dimKL_factor: 0.0001
Epoch 8 - Train Loss: 87.4583
Sub-losses: recon_loss: 87.4490, mut_info_loss: 0.0014, tot_corr_loss: 0.0077, dimwise_kl_loss: 0.0002, anneal_factor: 0.0002, effective_beta_mi_factor: 0.0018, effective_beta_tc_factor: 0.0920, effective_beta_dimKL_factor: 0.0001
Epoch 8 - Valid Loss: 83.1774
Sub-losses: recon_loss: 83.1700, mut_info_loss: 0.0010, tot_corr_loss: 0.0062, dimwise_kl_loss: 0.0002, anneal_factor: 0.0002, effective_beta_mi_factor: 0.0018, effective_beta_tc_factor: 0.0920, effective_beta_dimKL_factor: 0.0001
Epoch 9 - Train Loss: 86.7168
Sub-losses: recon_loss: 86.7029, mut_info_loss: 0.0020, tot_corr_loss: 0.0116, dimwise_kl_loss: 0.0003, anneal_factor: 0.0002, effective_beta_mi_factor: 0.0022, effective_beta_tc_factor: 0.1124, effective_beta_dimKL_factor: 0.0001
Epoch 9 - Valid Loss: 82.5168
Sub-losses: recon_loss: 82.5058, mut_info_loss: 0.0016, tot_corr_loss: 0.0091, dimwise_kl_loss: 0.0003, anneal_factor: 0.0002, effective_beta_mi_factor: 0.0022, effective_beta_tc_factor: 0.1124, effective_beta_dimKL_factor: 0.0001
Epoch 10 - Train Loss: 85.6509
Sub-losses: recon_loss: 85.6385, mut_info_loss: 0.0024, tot_corr_loss: 0.0095, dimwise_kl_loss: 0.0004, anneal_factor: 0.0003, effective_beta_mi_factor: 0.0027, effective_beta_tc_factor: 0.1373, effective_beta_dimKL_factor: 0.0001
Epoch 10 - Valid Loss: 82.1123
Sub-losses: recon_loss: 82.0969, mut_info_loss: 0.0020, tot_corr_loss: 0.0132, dimwise_kl_loss: 0.0003, anneal_factor: 0.0003, effective_beta_mi_factor: 0.0027, effective_beta_tc_factor: 0.1373, effective_beta_dimKL_factor: 0.0001
Epoch 11 - Train Loss: 85.2477
Sub-losses: recon_loss: 85.2240, mut_info_loss: 0.0032, tot_corr_loss: 0.0200, dimwise_kl_loss: 0.0005, anneal_factor: 0.0003, effective_beta_mi_factor: 0.0034, effective_beta_tc_factor: 0.1677, effective_beta_dimKL_factor: 0.0002
Epoch 11 - Valid Loss: 80.3974
Sub-losses: recon_loss: 80.3853, mut_info_loss: 0.0026, tot_corr_loss: 0.0092, dimwise_kl_loss: 0.0004, anneal_factor: 0.0003, effective_beta_mi_factor: 0.0034, effective_beta_tc_factor: 0.1677, effective_beta_dimKL_factor: 0.0002
Epoch 12 - Train Loss: 83.9004
Sub-losses: recon_loss: 83.8581, mut_info_loss: 0.0042, tot_corr_loss: 0.0374, dimwise_kl_loss: 0.0007, anneal_factor: 0.0004, effective_beta_mi_factor: 0.0041, effective_beta_tc_factor: 0.2048, effective_beta_dimKL_factor: 0.0002
Epoch 12 - Valid Loss: 79.6988
Sub-losses: recon_loss: 79.6728, mut_info_loss: 0.0028, tot_corr_loss: 0.0226, dimwise_kl_loss: 0.0005, anneal_factor: 0.0004, effective_beta_mi_factor: 0.0041, effective_beta_tc_factor: 0.2048, effective_beta_dimKL_factor: 0.0002
Epoch 13 - Train Loss: 83.4865
Sub-losses: recon_loss: 83.4460, mut_info_loss: 0.0046, tot_corr_loss: 0.0352, dimwise_kl_loss: 0.0007, anneal_factor: 0.0005, effective_beta_mi_factor: 0.0050, effective_beta_tc_factor: 0.2501, effective_beta_dimKL_factor: 0.0003
Epoch 13 - Valid Loss: 79.6094
Sub-losses: recon_loss: 79.5562, mut_info_loss: 0.0050, tot_corr_loss: 0.0472, dimwise_kl_loss: 0.0010, anneal_factor: 0.0005, effective_beta_mi_factor: 0.0050, effective_beta_tc_factor: 0.2501, effective_beta_dimKL_factor: 0.0003
Epoch 14 - Train Loss: 82.8275
Sub-losses: recon_loss: 82.7652, mut_info_loss: 0.0060, tot_corr_loss: 0.0552, dimwise_kl_loss: 0.0011, anneal_factor: 0.0006, effective_beta_mi_factor: 0.0061, effective_beta_tc_factor: 0.3054, effective_beta_dimKL_factor: 0.0003
Epoch 14 - Valid Loss: 79.1420
Sub-losses: recon_loss: 79.0842, mut_info_loss: 0.0042, tot_corr_loss: 0.0529, dimwise_kl_loss: 0.0007, anneal_factor: 0.0006, effective_beta_mi_factor: 0.0061, effective_beta_tc_factor: 0.3054, effective_beta_dimKL_factor: 0.0003
Epoch 15 - Train Loss: 81.5633
Sub-losses: recon_loss: 81.4747, mut_info_loss: 0.0080, tot_corr_loss: 0.0790, dimwise_kl_loss: 0.0016, anneal_factor: 0.0007, effective_beta_mi_factor: 0.0075, effective_beta_tc_factor: 0.3730, effective_beta_dimKL_factor: 0.0004
Epoch 15 - Valid Loss: 77.9349
Sub-losses: recon_loss: 77.8719, mut_info_loss: 0.0073, tot_corr_loss: 0.0542, dimwise_kl_loss: 0.0015, anneal_factor: 0.0007, effective_beta_mi_factor: 0.0075, effective_beta_tc_factor: 0.3730, effective_beta_dimKL_factor: 0.0004
Epoch 16 - Train Loss: 80.8502
Sub-losses: recon_loss: 80.7407, mut_info_loss: 0.0110, tot_corr_loss: 0.0967, dimwise_kl_loss: 0.0018, anneal_factor: 0.0009, effective_beta_mi_factor: 0.0091, effective_beta_tc_factor: 0.4555, effective_beta_dimKL_factor: 0.0005
Epoch 16 - Valid Loss: 76.0651
Sub-losses: recon_loss: 75.9799, mut_info_loss: 0.0086, tot_corr_loss: 0.0754, dimwise_kl_loss: 0.0012, anneal_factor: 0.0009, effective_beta_mi_factor: 0.0091, effective_beta_tc_factor: 0.4555, effective_beta_dimKL_factor: 0.0005
Epoch 17 - Train Loss: 80.3795
Sub-losses: recon_loss: 80.2183, mut_info_loss: 0.0125, tot_corr_loss: 0.1466, dimwise_kl_loss: 0.0020, anneal_factor: 0.0011, effective_beta_mi_factor: 0.0111, effective_beta_tc_factor: 0.5563, effective_beta_dimKL_factor: 0.0006
Epoch 17 - Valid Loss: 75.9399
Sub-losses: recon_loss: 75.7917, mut_info_loss: 0.0096, tot_corr_loss: 0.1364, dimwise_kl_loss: 0.0021, anneal_factor: 0.0011, effective_beta_mi_factor: 0.0111, effective_beta_tc_factor: 0.5563, effective_beta_dimKL_factor: 0.0006
Epoch 18 - Train Loss: 79.6521
Sub-losses: recon_loss: 79.4392, mut_info_loss: 0.0159, tot_corr_loss: 0.1941, dimwise_kl_loss: 0.0029, anneal_factor: 0.0014, effective_beta_mi_factor: 0.0136, effective_beta_tc_factor: 0.6793, effective_beta_dimKL_factor: 0.0007
Epoch 18 - Valid Loss: 78.4353
Sub-losses: recon_loss: 78.3727, mut_info_loss: 0.0100, tot_corr_loss: 0.0506, dimwise_kl_loss: 0.0019, anneal_factor: 0.0014, effective_beta_mi_factor: 0.0136, effective_beta_tc_factor: 0.6793, effective_beta_dimKL_factor: 0.0007
Epoch 19 - Train Loss: 79.8707
Sub-losses: recon_loss: 79.6585, mut_info_loss: 0.0181, tot_corr_loss: 0.1906, dimwise_kl_loss: 0.0035, anneal_factor: 0.0017, effective_beta_mi_factor: 0.0166, effective_beta_tc_factor: 0.8294, effective_beta_dimKL_factor: 0.0008
Epoch 19 - Valid Loss: 75.1519
Sub-losses: recon_loss: 74.9945, mut_info_loss: 0.0167, tot_corr_loss: 0.1373, dimwise_kl_loss: 0.0034, anneal_factor: 0.0017, effective_beta_mi_factor: 0.0166, effective_beta_tc_factor: 0.8294, effective_beta_dimKL_factor: 0.0008
Epoch 20 - Train Loss: 78.4002
Sub-losses: recon_loss: 78.1046, mut_info_loss: 0.0257, tot_corr_loss: 0.2659, dimwise_kl_loss: 0.0040, anneal_factor: 0.0020, effective_beta_mi_factor: 0.0203, effective_beta_tc_factor: 1.0127, effective_beta_dimKL_factor: 0.0010
Epoch 20 - Valid Loss: 75.1080
Sub-losses: recon_loss: 74.8640, mut_info_loss: 0.0159, tot_corr_loss: 0.2234, dimwise_kl_loss: 0.0046, anneal_factor: 0.0020, effective_beta_mi_factor: 0.0203, effective_beta_tc_factor: 1.0127, effective_beta_dimKL_factor: 0.0010
Epoch 21 - Train Loss: 78.0921
Sub-losses: recon_loss: 77.7057, mut_info_loss: 0.0294, tot_corr_loss: 0.3515, dimwise_kl_loss: 0.0055, anneal_factor: 0.0025, effective_beta_mi_factor: 0.0247, effective_beta_tc_factor: 1.2363, effective_beta_dimKL_factor: 0.0012
Epoch 21 - Valid Loss: 73.9743
Sub-losses: recon_loss: 73.4745, mut_info_loss: 0.0219, tot_corr_loss: 0.4734, dimwise_kl_loss: 0.0045, anneal_factor: 0.0025, effective_beta_mi_factor: 0.0247, effective_beta_tc_factor: 1.2363, effective_beta_dimKL_factor: 0.0012
Epoch 22 - Train Loss: 76.9629
Sub-losses: recon_loss: 76.5342, mut_info_loss: 0.0408, tot_corr_loss: 0.3816, dimwise_kl_loss: 0.0065, anneal_factor: 0.0030, effective_beta_mi_factor: 0.0302, effective_beta_tc_factor: 1.5092, effective_beta_dimKL_factor: 0.0015
Epoch 22 - Valid Loss: 73.0017
Sub-losses: recon_loss: 72.6674, mut_info_loss: 0.0256, tot_corr_loss: 0.3005, dimwise_kl_loss: 0.0081, anneal_factor: 0.0030, effective_beta_mi_factor: 0.0302, effective_beta_tc_factor: 1.5092, effective_beta_dimKL_factor: 0.0015
Epoch 23 - Train Loss: 77.3424
Sub-losses: recon_loss: 76.7251, mut_info_loss: 0.0442, tot_corr_loss: 0.5655, dimwise_kl_loss: 0.0077, anneal_factor: 0.0037, effective_beta_mi_factor: 0.0368, effective_beta_tc_factor: 1.8421, effective_beta_dimKL_factor: 0.0018
Epoch 23 - Valid Loss: 72.9883
Sub-losses: recon_loss: 72.7368, mut_info_loss: 0.0453, tot_corr_loss: 0.1986, dimwise_kl_loss: 0.0075, anneal_factor: 0.0037, effective_beta_mi_factor: 0.0368, effective_beta_tc_factor: 1.8421, effective_beta_dimKL_factor: 0.0018
Epoch 24 - Train Loss: 76.9208
Sub-losses: recon_loss: 76.0604, mut_info_loss: 0.0591, tot_corr_loss: 0.7901, dimwise_kl_loss: 0.0112, anneal_factor: 0.0045, effective_beta_mi_factor: 0.0450, effective_beta_tc_factor: 2.2481, effective_beta_dimKL_factor: 0.0022
Epoch 24 - Valid Loss: 72.0585
Sub-losses: recon_loss: 71.4440, mut_info_loss: 0.0692, tot_corr_loss: 0.5338, dimwise_kl_loss: 0.0115, anneal_factor: 0.0045, effective_beta_mi_factor: 0.0450, effective_beta_tc_factor: 2.2481, effective_beta_dimKL_factor: 0.0022
Epoch 25 - Train Loss: 76.4887
Sub-losses: recon_loss: 75.4190, mut_info_loss: 0.0733, tot_corr_loss: 0.9832, dimwise_kl_loss: 0.0132, anneal_factor: 0.0055, effective_beta_mi_factor: 0.0549, effective_beta_tc_factor: 2.7431, effective_beta_dimKL_factor: 0.0027
Epoch 25 - Valid Loss: 72.9880
Sub-losses: recon_loss: 72.5206, mut_info_loss: 0.0567, tot_corr_loss: 0.4020, dimwise_kl_loss: 0.0086, anneal_factor: 0.0055, effective_beta_mi_factor: 0.0549, effective_beta_tc_factor: 2.7431, effective_beta_dimKL_factor: 0.0027
Epoch 26 - Train Loss: 76.1254
Sub-losses: recon_loss: 74.9595, mut_info_loss: 0.0881, tot_corr_loss: 1.0626, dimwise_kl_loss: 0.0152, anneal_factor: 0.0067, effective_beta_mi_factor: 0.0669, effective_beta_tc_factor: 3.3464, effective_beta_dimKL_factor: 0.0033
Epoch 26 - Valid Loss: 71.6176
Sub-losses: recon_loss: 70.6641, mut_info_loss: 0.0842, tot_corr_loss: 0.8583, dimwise_kl_loss: 0.0109, anneal_factor: 0.0067, effective_beta_mi_factor: 0.0669, effective_beta_tc_factor: 3.3464, effective_beta_dimKL_factor: 0.0033
Epoch 27 - Train Loss: 75.5043
Sub-losses: recon_loss: 73.8235, mut_info_loss: 0.1172, tot_corr_loss: 1.5439, dimwise_kl_loss: 0.0197, anneal_factor: 0.0082, effective_beta_mi_factor: 0.0816, effective_beta_tc_factor: 4.0813, effective_beta_dimKL_factor: 0.0041
Epoch 27 - Valid Loss: 73.2472
Sub-losses: recon_loss: 71.8493, mut_info_loss: 0.0955, tot_corr_loss: 1.2872, dimwise_kl_loss: 0.0152, anneal_factor: 0.0082, effective_beta_mi_factor: 0.0816, effective_beta_tc_factor: 4.0813, effective_beta_dimKL_factor: 0.0041
Epoch 28 - Train Loss: 75.3236
Sub-losses: recon_loss: 73.1921, mut_info_loss: 0.1481, tot_corr_loss: 1.9572, dimwise_kl_loss: 0.0263, anneal_factor: 0.0100, effective_beta_mi_factor: 0.0995, effective_beta_tc_factor: 4.9759, effective_beta_dimKL_factor: 0.0050
Epoch 28 - Valid Loss: 70.5941
Sub-losses: recon_loss: 69.1680, mut_info_loss: 0.1198, tot_corr_loss: 1.2869, dimwise_kl_loss: 0.0194, anneal_factor: 0.0100, effective_beta_mi_factor: 0.0995, effective_beta_tc_factor: 4.9759, effective_beta_dimKL_factor: 0.0050
Epoch 29 - Train Loss: 76.2183
Sub-losses: recon_loss: 73.4670, mut_info_loss: 0.1744, tot_corr_loss: 2.5454, dimwise_kl_loss: 0.0315, anneal_factor: 0.0121, effective_beta_mi_factor: 0.1213, effective_beta_tc_factor: 6.0642, effective_beta_dimKL_factor: 0.0061
Epoch 29 - Valid Loss: 71.3346
Sub-losses: recon_loss: 68.8328, mut_info_loss: 0.1722, tot_corr_loss: 2.3018, dimwise_kl_loss: 0.0277, anneal_factor: 0.0121, effective_beta_mi_factor: 0.1213, effective_beta_tc_factor: 6.0642, effective_beta_dimKL_factor: 0.0061
Epoch 30 - Train Loss: 76.2643
Sub-losses: recon_loss: 73.1496, mut_info_loss: 0.1891, tot_corr_loss: 2.8895, dimwise_kl_loss: 0.0360, anneal_factor: 0.0148, effective_beta_mi_factor: 0.1477, effective_beta_tc_factor: 7.3870, effective_beta_dimKL_factor: 0.0074
Epoch 30 - Valid Loss: 71.5880
Sub-losses: recon_loss: 70.0011, mut_info_loss: 0.1718, tot_corr_loss: 1.3828, dimwise_kl_loss: 0.0322, anneal_factor: 0.0148, effective_beta_mi_factor: 0.1477, effective_beta_tc_factor: 7.3870, effective_beta_dimKL_factor: 0.0074
Epoch 31 - Train Loss: 75.9894
Sub-losses: recon_loss: 72.2936, mut_info_loss: 0.2615, tot_corr_loss: 3.3884, dimwise_kl_loss: 0.0459, anneal_factor: 0.0180, effective_beta_mi_factor: 0.1799, effective_beta_tc_factor: 8.9931, effective_beta_dimKL_factor: 0.0090
Epoch 31 - Valid Loss: 72.9259
Sub-losses: recon_loss: 69.6424, mut_info_loss: 0.2131, tot_corr_loss: 3.0363, dimwise_kl_loss: 0.0341, anneal_factor: 0.0180, effective_beta_mi_factor: 0.1799, effective_beta_tc_factor: 8.9931, effective_beta_dimKL_factor: 0.0090
Epoch 32 - Train Loss: 76.0001
Sub-losses: recon_loss: 71.7725, mut_info_loss: 0.3215, tot_corr_loss: 3.8541, dimwise_kl_loss: 0.0519, anneal_factor: 0.0219, effective_beta_mi_factor: 0.2188, effective_beta_tc_factor: 10.9406, effective_beta_dimKL_factor: 0.0109
Epoch 32 - Valid Loss: 71.1560
Sub-losses: recon_loss: 68.6308, mut_info_loss: 0.2659, tot_corr_loss: 2.2132, dimwise_kl_loss: 0.0460, anneal_factor: 0.0219, effective_beta_mi_factor: 0.2188, effective_beta_tc_factor: 10.9406, effective_beta_dimKL_factor: 0.0109
Epoch 33 - Train Loss: 76.3094
Sub-losses: recon_loss: 71.6742, mut_info_loss: 0.3720, tot_corr_loss: 4.2016, dimwise_kl_loss: 0.0616, anneal_factor: 0.0266, effective_beta_mi_factor: 0.2660, effective_beta_tc_factor: 13.2985, effective_beta_dimKL_factor: 0.0133
Epoch 33 - Valid Loss: 70.7727
Sub-losses: recon_loss: 67.3046, mut_info_loss: 0.3527, tot_corr_loss: 3.0659, dimwise_kl_loss: 0.0495, anneal_factor: 0.0266, effective_beta_mi_factor: 0.2660, effective_beta_tc_factor: 13.2985, effective_beta_dimKL_factor: 0.0133
Epoch 34 - Train Loss: 75.1951
Sub-losses: recon_loss: 71.1592, mut_info_loss: 0.5149, tot_corr_loss: 3.4450, dimwise_kl_loss: 0.0760, anneal_factor: 0.0323, effective_beta_mi_factor: 0.3230, effective_beta_tc_factor: 16.1477, effective_beta_dimKL_factor: 0.0161
Epoch 34 - Valid Loss: 71.4594
Sub-losses: recon_loss: 68.9785, mut_info_loss: 0.4211, tot_corr_loss: 1.9929, dimwise_kl_loss: 0.0669, anneal_factor: 0.0323, effective_beta_mi_factor: 0.3230, effective_beta_tc_factor: 16.1477, effective_beta_dimKL_factor: 0.0161
Epoch 35 - Train Loss: 76.2941
Sub-losses: recon_loss: 71.2395, mut_info_loss: 0.5944, tot_corr_loss: 4.3712, dimwise_kl_loss: 0.0889, anneal_factor: 0.0392, effective_beta_mi_factor: 0.3917, effective_beta_tc_factor: 19.5829, effective_beta_dimKL_factor: 0.0196
Epoch 35 - Valid Loss: 71.1226
Sub-losses: recon_loss: 67.5429, mut_info_loss: 0.4526, tot_corr_loss: 3.0510, dimwise_kl_loss: 0.0761, anneal_factor: 0.0392, effective_beta_mi_factor: 0.3917, effective_beta_tc_factor: 19.5829, effective_beta_dimKL_factor: 0.0196
Epoch 36 - Train Loss: 77.4649
Sub-losses: recon_loss: 71.3562, mut_info_loss: 0.6472, tot_corr_loss: 5.3565, dimwise_kl_loss: 0.1050, anneal_factor: 0.0474, effective_beta_mi_factor: 0.4743, effective_beta_tc_factor: 23.7129, effective_beta_dimKL_factor: 0.0237
Epoch 36 - Valid Loss: 74.7848
Sub-losses: recon_loss: 66.8546, mut_info_loss: 0.6954, tot_corr_loss: 7.1207, dimwise_kl_loss: 0.1141, anneal_factor: 0.0474, effective_beta_mi_factor: 0.4743, effective_beta_tc_factor: 23.7129, effective_beta_dimKL_factor: 0.0237
Epoch 37 - Train Loss: 76.1208
Sub-losses: recon_loss: 70.6377, mut_info_loss: 0.8414, tot_corr_loss: 4.5259, dimwise_kl_loss: 0.1158, anneal_factor: 0.0573, effective_beta_mi_factor: 0.5732, effective_beta_tc_factor: 28.6621, effective_beta_dimKL_factor: 0.0287
Epoch 37 - Valid Loss: 70.3081
Sub-losses: recon_loss: 68.3108, mut_info_loss: 0.7203, tot_corr_loss: 1.1698, dimwise_kl_loss: 0.1072, anneal_factor: 0.0573, effective_beta_mi_factor: 0.5732, effective_beta_tc_factor: 28.6621, effective_beta_dimKL_factor: 0.0287
Epoch 38 - Train Loss: 77.8821
Sub-losses: recon_loss: 70.8597, mut_info_loss: 1.0121, tot_corr_loss: 5.8544, dimwise_kl_loss: 0.1559, anneal_factor: 0.0691, effective_beta_mi_factor: 0.6914, effective_beta_tc_factor: 34.5692, effective_beta_dimKL_factor: 0.0346
Epoch 38 - Valid Loss: 70.9222
Sub-losses: recon_loss: 67.7147, mut_info_loss: 0.8379, tot_corr_loss: 2.2409, dimwise_kl_loss: 0.1287, anneal_factor: 0.0691, effective_beta_mi_factor: 0.6914, effective_beta_tc_factor: 34.5692, effective_beta_dimKL_factor: 0.0346
Epoch 39 - Train Loss: 77.9403
Sub-losses: recon_loss: 71.1234, mut_info_loss: 1.1496, tot_corr_loss: 5.4933, dimwise_kl_loss: 0.1739, anneal_factor: 0.0832, effective_beta_mi_factor: 0.8317, effective_beta_tc_factor: 41.5863, effective_beta_dimKL_factor: 0.0416
Epoch 39 - Valid Loss: 69.6204
Sub-losses: recon_loss: 68.0568, mut_info_loss: 1.0458, tot_corr_loss: 0.3733, dimwise_kl_loss: 0.1446, anneal_factor: 0.0832, effective_beta_mi_factor: 0.8317, effective_beta_tc_factor: 41.5863, effective_beta_dimKL_factor: 0.0416
Epoch 40 - Train Loss: 76.2914
Sub-losses: recon_loss: 70.7318, mut_info_loss: 1.3396, tot_corr_loss: 4.0312, dimwise_kl_loss: 0.1889, anneal_factor: 0.0998, effective_beta_mi_factor: 0.9975, effective_beta_tc_factor: 49.8752, effective_beta_dimKL_factor: 0.0499
Epoch 40 - Valid Loss: 79.5568
Sub-losses: recon_loss: 66.7782, mut_info_loss: 1.3528, tot_corr_loss: 11.2356, dimwise_kl_loss: 0.1903, anneal_factor: 0.0998, effective_beta_mi_factor: 0.9975, effective_beta_tc_factor: 49.8752, effective_beta_dimKL_factor: 0.0499
Epoch 41 - Train Loss: 75.2537
Sub-losses: recon_loss: 72.0182, mut_info_loss: 1.5672, tot_corr_loss: 1.4461, dimwise_kl_loss: 0.2221, anneal_factor: 0.1192, effective_beta_mi_factor: 1.1920, effective_beta_tc_factor: 59.6015, effective_beta_dimKL_factor: 0.0596
Epoch 41 - Valid Loss: 75.9130
Sub-losses: recon_loss: 68.1794, mut_info_loss: 1.3176, tot_corr_loss: 6.1908, dimwise_kl_loss: 0.2251, anneal_factor: 0.1192, effective_beta_mi_factor: 1.1920, effective_beta_tc_factor: 59.6015, effective_beta_dimKL_factor: 0.0596
Epoch 42 - Train Loss: 80.2799
Sub-losses: recon_loss: 71.6301, mut_info_loss: 1.8292, tot_corr_loss: 6.5355, dimwise_kl_loss: 0.2850, anneal_factor: 0.1419, effective_beta_mi_factor: 1.4185, effective_beta_tc_factor: 70.9255, effective_beta_dimKL_factor: 0.0709
Epoch 42 - Valid Loss: 74.8966
Sub-losses: recon_loss: 70.3301, mut_info_loss: 1.1243, tot_corr_loss: 3.2370, dimwise_kl_loss: 0.2052, anneal_factor: 0.1419, effective_beta_mi_factor: 1.4185, effective_beta_tc_factor: 70.9255, effective_beta_dimKL_factor: 0.0709
Epoch 43 - Train Loss: 78.6515
Sub-losses: recon_loss: 72.3795, mut_info_loss: 2.1958, tot_corr_loss: 3.7722, dimwise_kl_loss: 0.3040, anneal_factor: 0.1680, effective_beta_mi_factor: 1.6798, effective_beta_tc_factor: 83.9908, effective_beta_dimKL_factor: 0.0840
Epoch 43 - Valid Loss: 77.9286
Sub-losses: recon_loss: 68.8093, mut_info_loss: 2.2502, tot_corr_loss: 6.5961, dimwise_kl_loss: 0.2729, anneal_factor: 0.1680, effective_beta_mi_factor: 1.6798, effective_beta_tc_factor: 83.9908, effective_beta_dimKL_factor: 0.0840
Epoch 44 - Train Loss: 78.7931
Sub-losses: recon_loss: 72.7660, mut_info_loss: 2.3060, tot_corr_loss: 3.3286, dimwise_kl_loss: 0.3925, anneal_factor: 0.1978, effective_beta_mi_factor: 1.9782, effective_beta_tc_factor: 98.9081, effective_beta_dimKL_factor: 0.0989
Epoch 44 - Valid Loss: 80.4929
Sub-losses: recon_loss: 66.1490, mut_info_loss: 2.5730, tot_corr_loss: 11.4355, dimwise_kl_loss: 0.3353, anneal_factor: 0.1978, effective_beta_mi_factor: 1.9782, effective_beta_tc_factor: 98.9081, effective_beta_dimKL_factor: 0.0989
Epoch 45 - Train Loss: 81.7900
Sub-losses: recon_loss: 72.0792, mut_info_loss: 2.7460, tot_corr_loss: 6.5602, dimwise_kl_loss: 0.4045, anneal_factor: 0.2315, effective_beta_mi_factor: 2.3148, effective_beta_tc_factor: 115.7376, effective_beta_dimKL_factor: 0.1157
Epoch 45 - Valid Loss: 76.4769
Sub-losses: recon_loss: 67.6333, mut_info_loss: 2.3496, tot_corr_loss: 6.0704, dimwise_kl_loss: 0.4235, anneal_factor: 0.2315, effective_beta_mi_factor: 2.3148, effective_beta_tc_factor: 115.7376, effective_beta_dimKL_factor: 0.1157
Epoch 46 - Train Loss: 81.7984
Sub-losses: recon_loss: 71.5667, mut_info_loss: 3.3638, tot_corr_loss: 6.3543, dimwise_kl_loss: 0.5136, anneal_factor: 0.2689, effective_beta_mi_factor: 2.6894, effective_beta_tc_factor: 134.4707, effective_beta_dimKL_factor: 0.1345
Epoch 46 - Valid Loss: 77.8737
Sub-losses: recon_loss: 68.6013, mut_info_loss: 3.3719, tot_corr_loss: 5.5012, dimwise_kl_loss: 0.3994, anneal_factor: 0.2689, effective_beta_mi_factor: 2.6894, effective_beta_tc_factor: 134.4707, effective_beta_dimKL_factor: 0.1345
Epoch 47 - Train Loss: 79.0374
Sub-losses: recon_loss: 71.2300, mut_info_loss: 3.9287, tot_corr_loss: 3.3223, dimwise_kl_loss: 0.5564, anneal_factor: 0.3100, effective_beta_mi_factor: 3.1003, effective_beta_tc_factor: 155.0128, effective_beta_dimKL_factor: 0.1550
Epoch 47 - Valid Loss: 72.9867
Sub-losses: recon_loss: 69.3901, mut_info_loss: 3.1074, tot_corr_loss: 0.0000, dimwise_kl_loss: 0.4892, anneal_factor: 0.3100, effective_beta_mi_factor: 3.1003, effective_beta_tc_factor: 155.0128, effective_beta_dimKL_factor: 0.1550
Epoch 48 - Train Loss: 81.8605
Sub-losses: recon_loss: 71.2152, mut_info_loss: 4.4250, tot_corr_loss: 5.5872, dimwise_kl_loss: 0.6331, anneal_factor: 0.3543, effective_beta_mi_factor: 3.5434, effective_beta_tc_factor: 177.1718, effective_beta_dimKL_factor: 0.1772
Epoch 48 - Valid Loss: 79.1251
Sub-losses: recon_loss: 70.4362, mut_info_loss: 3.7527, tot_corr_loss: 4.4317, dimwise_kl_loss: 0.5044, anneal_factor: 0.3543, effective_beta_mi_factor: 3.5434, effective_beta_tc_factor: 177.1718, effective_beta_dimKL_factor: 0.1772
Epoch 49 - Train Loss: 83.6371
Sub-losses: recon_loss: 72.6211, mut_info_loss: 4.4142, tot_corr_loss: 5.9197, dimwise_kl_loss: 0.6821, anneal_factor: 0.4013, effective_beta_mi_factor: 4.0131, effective_beta_tc_factor: 200.6562, effective_beta_dimKL_factor: 0.2007
Epoch 49 - Valid Loss: 73.8518
Sub-losses: recon_loss: 70.1264, mut_info_loss: 3.1653, tot_corr_loss: 0.0000, dimwise_kl_loss: 0.5600, anneal_factor: 0.4013, effective_beta_mi_factor: 4.0131, effective_beta_tc_factor: 200.6562, effective_beta_dimKL_factor: 0.2007
Epoch 50 - Train Loss: 82.8409
Sub-losses: recon_loss: 71.1285, mut_info_loss: 5.2932, tot_corr_loss: 5.6277, dimwise_kl_loss: 0.7914, anneal_factor: 0.4502, effective_beta_mi_factor: 4.5017, effective_beta_tc_factor: 225.0830, effective_beta_dimKL_factor: 0.2251
Epoch 50 - Valid Loss: 100.7160
Sub-losses: recon_loss: 68.6289, mut_info_loss: 4.7468, tot_corr_loss: 26.7500, dimwise_kl_loss: 0.5902, anneal_factor: 0.4502, effective_beta_mi_factor: 4.5017, effective_beta_tc_factor: 225.0830, effective_beta_dimKL_factor: 0.2251
Epoch 51 - Train Loss: 92.1017
Sub-losses: recon_loss: 72.0653, mut_info_loss: 5.5319, tot_corr_loss: 13.6110, dimwise_kl_loss: 0.8935, anneal_factor: 0.5000, effective_beta_mi_factor: 5.0000, effective_beta_tc_factor: 250.0000, effective_beta_dimKL_factor: 0.2500
Epoch 51 - Valid Loss: 88.3142
Sub-losses: recon_loss: 69.5591, mut_info_loss: 4.9708, tot_corr_loss: 13.1262, dimwise_kl_loss: 0.6581, anneal_factor: 0.5000, effective_beta_mi_factor: 5.0000, effective_beta_tc_factor: 250.0000, effective_beta_dimKL_factor: 0.2500
Epoch 52 - Train Loss: 82.9924
Sub-losses: recon_loss: 72.4625, mut_info_loss: 5.4896, tot_corr_loss: 4.1822, dimwise_kl_loss: 0.8581, anneal_factor: 0.5498, effective_beta_mi_factor: 5.4983, effective_beta_tc_factor: 274.9170, effective_beta_dimKL_factor: 0.2749
Epoch 52 - Valid Loss: 90.4219
Sub-losses: recon_loss: 70.5451, mut_info_loss: 5.3416, tot_corr_loss: 13.9409, dimwise_kl_loss: 0.5943, anneal_factor: 0.5498, effective_beta_mi_factor: 5.4983, effective_beta_tc_factor: 274.9170, effective_beta_dimKL_factor: 0.2749
Epoch 53 - Train Loss: 82.4995
Sub-losses: recon_loss: 71.8610, mut_info_loss: 6.2631, tot_corr_loss: 3.3131, dimwise_kl_loss: 1.0623, anneal_factor: 0.5987, effective_beta_mi_factor: 5.9869, effective_beta_tc_factor: 299.3438, effective_beta_dimKL_factor: 0.2993
Epoch 53 - Valid Loss: 85.8841
Sub-losses: recon_loss: 70.0568, mut_info_loss: 5.6491, tot_corr_loss: 9.3625, dimwise_kl_loss: 0.8157, anneal_factor: 0.5987, effective_beta_mi_factor: 5.9869, effective_beta_tc_factor: 299.3438, effective_beta_dimKL_factor: 0.2993
Epoch 54 - Train Loss: 84.1651
Sub-losses: recon_loss: 72.3524, mut_info_loss: 6.8840, tot_corr_loss: 3.8484, dimwise_kl_loss: 1.0803, anneal_factor: 0.6457, effective_beta_mi_factor: 6.4566, effective_beta_tc_factor: 322.8282, effective_beta_dimKL_factor: 0.3228
Epoch 54 - Valid Loss: 96.6791
Sub-losses: recon_loss: 66.6134, mut_info_loss: 5.9634, tot_corr_loss: 23.3158, dimwise_kl_loss: 0.7864, anneal_factor: 0.6457, effective_beta_mi_factor: 6.4566, effective_beta_tc_factor: 322.8282, effective_beta_dimKL_factor: 0.3228
Epoch 55 - Train Loss: 86.1831
Sub-losses: recon_loss: 71.6036, mut_info_loss: 7.3149, tot_corr_loss: 6.2258, dimwise_kl_loss: 1.0387, anneal_factor: 0.6900, effective_beta_mi_factor: 6.8997, effective_beta_tc_factor: 344.9872, effective_beta_dimKL_factor: 0.3450
Epoch 55 - Valid Loss: 94.0451
Sub-losses: recon_loss: 66.3947, mut_info_loss: 7.2575, tot_corr_loss: 19.3065, dimwise_kl_loss: 1.0865, anneal_factor: 0.6900, effective_beta_mi_factor: 6.8997, effective_beta_tc_factor: 344.9872, effective_beta_dimKL_factor: 0.3450
Epoch 56 - Train Loss: 82.9094
Sub-losses: recon_loss: 72.0950, mut_info_loss: 7.1583, tot_corr_loss: 2.5687, dimwise_kl_loss: 1.0874, anneal_factor: 0.7311, effective_beta_mi_factor: 7.3106, effective_beta_tc_factor: 365.5293, effective_beta_dimKL_factor: 0.3655
Epoch 56 - Valid Loss: 76.3570
Sub-losses: recon_loss: 70.0773, mut_info_loss: 5.0429, tot_corr_loss: 0.0000, dimwise_kl_loss: 1.2367, anneal_factor: 0.7311, effective_beta_mi_factor: 7.3106, effective_beta_tc_factor: 365.5293, effective_beta_dimKL_factor: 0.3655
Epoch 57 - Train Loss: 87.7793
Sub-losses: recon_loss: 72.9668, mut_info_loss: 7.5707, tot_corr_loss: 6.0752, dimwise_kl_loss: 1.1667, anneal_factor: 0.7685, effective_beta_mi_factor: 7.6852, effective_beta_tc_factor: 384.2624, effective_beta_dimKL_factor: 0.3843
Epoch 57 - Valid Loss: 98.5972
Sub-losses: recon_loss: 66.5178, mut_info_loss: 7.7227, tot_corr_loss: 23.1343, dimwise_kl_loss: 1.2223, anneal_factor: 0.7685, effective_beta_mi_factor: 7.6852, effective_beta_tc_factor: 384.2624, effective_beta_dimKL_factor: 0.3843
Epoch 58 - Train Loss: 87.5950
Sub-losses: recon_loss: 71.6008, mut_info_loss: 8.1810, tot_corr_loss: 6.4453, dimwise_kl_loss: 1.3679, anneal_factor: 0.8022, effective_beta_mi_factor: 8.0218, effective_beta_tc_factor: 401.0919, effective_beta_dimKL_factor: 0.4011
Epoch 58 - Valid Loss: 100.6997
Sub-losses: recon_loss: 64.8650, mut_info_loss: 9.1908, tot_corr_loss: 25.2596, dimwise_kl_loss: 1.3843, anneal_factor: 0.8022, effective_beta_mi_factor: 8.0218, effective_beta_tc_factor: 401.0919, effective_beta_dimKL_factor: 0.4011
Epoch 59 - Train Loss: 83.8761
Sub-losses: recon_loss: 71.7905, mut_info_loss: 8.4066, tot_corr_loss: 2.3969, dimwise_kl_loss: 1.2821, anneal_factor: 0.8320, effective_beta_mi_factor: 8.3202, effective_beta_tc_factor: 416.0092, effective_beta_dimKL_factor: 0.4160
Epoch 59 - Valid Loss: 85.1332
Sub-losses: recon_loss: 68.0619, mut_info_loss: 7.8886, tot_corr_loss: 7.8116, dimwise_kl_loss: 1.3711, anneal_factor: 0.8320, effective_beta_mi_factor: 8.3202, effective_beta_tc_factor: 416.0092, effective_beta_dimKL_factor: 0.4160
Epoch 60 - Train Loss: 84.1684
Sub-losses: recon_loss: 74.0520, mut_info_loss: 7.0361, tot_corr_loss: 1.8091, dimwise_kl_loss: 1.2713, anneal_factor: 0.8581, effective_beta_mi_factor: 8.5815, effective_beta_tc_factor: 429.0745, effective_beta_dimKL_factor: 0.4291
Epoch 60 - Valid Loss: 76.7739
Sub-losses: recon_loss: 69.5868, mut_info_loss: 6.0293, tot_corr_loss: 0.0000, dimwise_kl_loss: 1.1578, anneal_factor: 0.8581, effective_beta_mi_factor: 8.5815, effective_beta_tc_factor: 429.0745, effective_beta_dimKL_factor: 0.4291
Epoch 61 - Train Loss: 87.3326
Sub-losses: recon_loss: 71.9350, mut_info_loss: 8.3863, tot_corr_loss: 5.6787, dimwise_kl_loss: 1.3326, anneal_factor: 0.8808, effective_beta_mi_factor: 8.8080, effective_beta_tc_factor: 440.3985, effective_beta_dimKL_factor: 0.4404
Epoch 61 - Valid Loss: 78.2464
Sub-losses: recon_loss: 67.6840, mut_info_loss: 9.3345, tot_corr_loss: 0.0000, dimwise_kl_loss: 1.2279, anneal_factor: 0.8808, effective_beta_mi_factor: 8.8080, effective_beta_tc_factor: 440.3985, effective_beta_dimKL_factor: 0.4404
Epoch 62 - Train Loss: 81.8746
Sub-losses: recon_loss: 72.8115, mut_info_loss: 7.8978, tot_corr_loss: 0.0000, dimwise_kl_loss: 1.1653, anneal_factor: 0.9002, effective_beta_mi_factor: 9.0025, effective_beta_tc_factor: 450.1248, effective_beta_dimKL_factor: 0.4501
Epoch 62 - Valid Loss: 81.3080
Sub-losses: recon_loss: 69.0328, mut_info_loss: 9.8524, tot_corr_loss: 0.8911, dimwise_kl_loss: 1.5316, anneal_factor: 0.9002, effective_beta_mi_factor: 9.0025, effective_beta_tc_factor: 450.1248, effective_beta_dimKL_factor: 0.4501
Epoch 63 - Train Loss: 86.7557
Sub-losses: recon_loss: 72.7168, mut_info_loss: 8.2064, tot_corr_loss: 4.4923, dimwise_kl_loss: 1.3403, anneal_factor: 0.9168, effective_beta_mi_factor: 9.1683, effective_beta_tc_factor: 458.4137, effective_beta_dimKL_factor: 0.4584
Epoch 63 - Valid Loss: 90.3020
Sub-losses: recon_loss: 66.9988, mut_info_loss: 8.8742, tot_corr_loss: 13.2088, dimwise_kl_loss: 1.2202, anneal_factor: 0.9168, effective_beta_mi_factor: 9.1683, effective_beta_tc_factor: 458.4137, effective_beta_dimKL_factor: 0.4584
Epoch 64 - Train Loss: 89.1942
Sub-losses: recon_loss: 72.7667, mut_info_loss: 7.7248, tot_corr_loss: 7.5360, dimwise_kl_loss: 1.1666, anneal_factor: 0.9309, effective_beta_mi_factor: 9.3086, effective_beta_tc_factor: 465.4308, effective_beta_dimKL_factor: 0.4654
Epoch 64 - Valid Loss: 101.7626
Sub-losses: recon_loss: 68.0170, mut_info_loss: 9.2917, tot_corr_loss: 23.3792, dimwise_kl_loss: 1.0747, anneal_factor: 0.9309, effective_beta_mi_factor: 9.3086, effective_beta_tc_factor: 465.4308, effective_beta_dimKL_factor: 0.4654
Epoch 65 - Train Loss: 81.7186
Sub-losses: recon_loss: 73.8119, mut_info_loss: 6.8411, tot_corr_loss: 0.0000, dimwise_kl_loss: 1.0657, anneal_factor: 0.9427, effective_beta_mi_factor: 9.4268, effective_beta_tc_factor: 471.3379, effective_beta_dimKL_factor: 0.4713
Epoch 65 - Valid Loss: 79.3527
Sub-losses: recon_loss: 66.4014, mut_info_loss: 9.2585, tot_corr_loss: 2.4978, dimwise_kl_loss: 1.1950, anneal_factor: 0.9427, effective_beta_mi_factor: 9.4268, effective_beta_tc_factor: 471.3379, effective_beta_dimKL_factor: 0.4713
Epoch 66 - Train Loss: 83.5845
Sub-losses: recon_loss: 71.8700, mut_info_loss: 7.3583, tot_corr_loss: 3.0635, dimwise_kl_loss: 1.2927, anneal_factor: 0.9526, effective_beta_mi_factor: 9.5257, effective_beta_tc_factor: 476.2870, effective_beta_dimKL_factor: 0.4763
Epoch 66 - Valid Loss: 77.0272
Sub-losses: recon_loss: 66.9553, mut_info_loss: 8.8413, tot_corr_loss: 0.0000, dimwise_kl_loss: 1.2306, anneal_factor: 0.9526, effective_beta_mi_factor: 9.5257, effective_beta_tc_factor: 476.2870, effective_beta_dimKL_factor: 0.4763
Epoch 67 - Train Loss: 90.9888
Sub-losses: recon_loss: 72.3022, mut_info_loss: 8.8294, tot_corr_loss: 8.6366, dimwise_kl_loss: 1.2205, anneal_factor: 0.9608, effective_beta_mi_factor: 9.6083, effective_beta_tc_factor: 480.4171, effective_beta_dimKL_factor: 0.4804
Epoch 67 - Valid Loss: 81.1771
Sub-losses: recon_loss: 68.8250, mut_info_loss: 5.6489, tot_corr_loss: 5.1853, dimwise_kl_loss: 1.5179, anneal_factor: 0.9608, effective_beta_mi_factor: 9.6083, effective_beta_tc_factor: 480.4171, effective_beta_dimKL_factor: 0.4804
Epoch 68 - Train Loss: 83.7702
Sub-losses: recon_loss: 73.2418, mut_info_loss: 7.6857, tot_corr_loss: 1.6746, dimwise_kl_loss: 1.1681, anneal_factor: 0.9677, effective_beta_mi_factor: 9.6770, effective_beta_tc_factor: 483.8523, effective_beta_dimKL_factor: 0.4839
Epoch 68 - Valid Loss: 89.9262
Sub-losses: recon_loss: 72.0679, mut_info_loss: 7.5759, tot_corr_loss: 9.3168, dimwise_kl_loss: 0.9656, anneal_factor: 0.9677, effective_beta_mi_factor: 9.6770, effective_beta_tc_factor: 483.8523, effective_beta_dimKL_factor: 0.4839
Epoch 69 - Train Loss: 86.4524
Sub-losses: recon_loss: 72.7061, mut_info_loss: 7.7574, tot_corr_loss: 4.5987, dimwise_kl_loss: 1.3902, anneal_factor: 0.9734, effective_beta_mi_factor: 9.7340, effective_beta_tc_factor: 486.7015, effective_beta_dimKL_factor: 0.4867
Epoch 69 - Valid Loss: 88.5749
Sub-losses: recon_loss: 69.0562, mut_info_loss: 7.4257, tot_corr_loss: 11.0640, dimwise_kl_loss: 1.0290, anneal_factor: 0.9734, effective_beta_mi_factor: 9.7340, effective_beta_tc_factor: 486.7015, effective_beta_dimKL_factor: 0.4867
Epoch 70 - Train Loss: 88.1080
Sub-losses: recon_loss: 72.0139, mut_info_loss: 8.1925, tot_corr_loss: 6.7348, dimwise_kl_loss: 1.1668, anneal_factor: 0.9781, effective_beta_mi_factor: 9.7812, effective_beta_tc_factor: 489.0594, effective_beta_dimKL_factor: 0.4891
Epoch 70 - Valid Loss: 96.4195
Sub-losses: recon_loss: 64.4202, mut_info_loss: 9.1079, tot_corr_loss: 21.2688, dimwise_kl_loss: 1.6226, anneal_factor: 0.9781, effective_beta_mi_factor: 9.7812, effective_beta_tc_factor: 489.0594, effective_beta_dimKL_factor: 0.4891
Epoch 71 - Train Loss: 88.7897
Sub-losses: recon_loss: 72.7602, mut_info_loss: 8.3641, tot_corr_loss: 6.3324, dimwise_kl_loss: 1.3330, anneal_factor: 0.9820, effective_beta_mi_factor: 9.8201, effective_beta_tc_factor: 491.0069, effective_beta_dimKL_factor: 0.4910
Epoch 71 - Valid Loss: 77.1813
Sub-losses: recon_loss: 71.5633, mut_info_loss: 4.5535, tot_corr_loss: 0.0000, dimwise_kl_loss: 1.0645, anneal_factor: 0.9820, effective_beta_mi_factor: 9.8201, effective_beta_tc_factor: 491.0069, effective_beta_dimKL_factor: 0.4910
Epoch 72 - Train Loss: 88.0361
Sub-losses: recon_loss: 72.2816, mut_info_loss: 7.2531, tot_corr_loss: 7.2838, dimwise_kl_loss: 1.2177, anneal_factor: 0.9852, effective_beta_mi_factor: 9.8523, effective_beta_tc_factor: 492.6130, effective_beta_dimKL_factor: 0.4926
Epoch 72 - Valid Loss: 77.0704
Sub-losses: recon_loss: 68.8185, mut_info_loss: 7.0593, tot_corr_loss: 0.0000, dimwise_kl_loss: 1.1927, anneal_factor: 0.9852, effective_beta_mi_factor: 9.8523, effective_beta_tc_factor: 492.6130, effective_beta_dimKL_factor: 0.4926
Epoch 73 - Train Loss: 82.0682
Sub-losses: recon_loss: 73.5960, mut_info_loss: 7.2524, tot_corr_loss: 0.1862, dimwise_kl_loss: 1.0336, anneal_factor: 0.9879, effective_beta_mi_factor: 9.8787, effective_beta_tc_factor: 493.9358, effective_beta_dimKL_factor: 0.4939
Epoch 73 - Valid Loss: 86.4995
Sub-losses: recon_loss: 69.4707, mut_info_loss: 7.2798, tot_corr_loss: 8.3699, dimwise_kl_loss: 1.3791, anneal_factor: 0.9879, effective_beta_mi_factor: 9.8787, effective_beta_tc_factor: 493.9358, effective_beta_dimKL_factor: 0.4939
Epoch 74 - Train Loss: 86.3785
Sub-losses: recon_loss: 72.4691, mut_info_loss: 7.4970, tot_corr_loss: 5.2828, dimwise_kl_loss: 1.1296, anneal_factor: 0.9900, effective_beta_mi_factor: 9.9005, effective_beta_tc_factor: 495.0241, effective_beta_dimKL_factor: 0.4950
Epoch 74 - Valid Loss: 77.9196
Sub-losses: recon_loss: 68.8381, mut_info_loss: 7.9032, tot_corr_loss: 0.0000, dimwise_kl_loss: 1.1783, anneal_factor: 0.9900, effective_beta_mi_factor: 9.9005, effective_beta_tc_factor: 495.0241, effective_beta_dimKL_factor: 0.4950
Epoch 75 - Train Loss: 83.7824
Sub-losses: recon_loss: 73.1181, mut_info_loss: 7.1738, tot_corr_loss: 2.5613, dimwise_kl_loss: 0.9292, anneal_factor: 0.9918, effective_beta_mi_factor: 9.9184, effective_beta_tc_factor: 495.9187, effective_beta_dimKL_factor: 0.4959
Epoch 75 - Valid Loss: 92.0198
Sub-losses: recon_loss: 69.2824, mut_info_loss: 6.6939, tot_corr_loss: 15.0097, dimwise_kl_loss: 1.0338, anneal_factor: 0.9918, effective_beta_mi_factor: 9.9184, effective_beta_tc_factor: 495.9187, effective_beta_dimKL_factor: 0.4959
Epoch 76 - Train Loss: 87.8374
Sub-losses: recon_loss: 72.7451, mut_info_loss: 6.7722, tot_corr_loss: 7.0782, dimwise_kl_loss: 1.2419, anneal_factor: 0.9933, effective_beta_mi_factor: 9.9331, effective_beta_tc_factor: 496.6536, effective_beta_dimKL_factor: 0.4967
Epoch 76 - Valid Loss: 76.2217
Sub-losses: recon_loss: 67.8373, mut_info_loss: 7.5573, tot_corr_loss: 0.0000, dimwise_kl_loss: 0.8271, anneal_factor: 0.9933, effective_beta_mi_factor: 9.9331, effective_beta_tc_factor: 496.6536, effective_beta_dimKL_factor: 0.4967
Epoch 77 - Train Loss: 83.1261
Sub-losses: recon_loss: 73.3307, mut_info_loss: 7.3394, tot_corr_loss: 1.4459, dimwise_kl_loss: 1.0101, anneal_factor: 0.9945, effective_beta_mi_factor: 9.9451, effective_beta_tc_factor: 497.2568, effective_beta_dimKL_factor: 0.4973
Epoch 77 - Valid Loss: 77.5612
Sub-losses: recon_loss: 69.7138, mut_info_loss: 7.0137, tot_corr_loss: 0.0000, dimwise_kl_loss: 0.8337, anneal_factor: 0.9945, effective_beta_mi_factor: 9.9451, effective_beta_tc_factor: 497.2568, effective_beta_dimKL_factor: 0.4973
Epoch 78 - Train Loss: 82.0442
Sub-losses: recon_loss: 72.1073, mut_info_loss: 7.3667, tot_corr_loss: 1.4856, dimwise_kl_loss: 1.0845, anneal_factor: 0.9955, effective_beta_mi_factor: 9.9550, effective_beta_tc_factor: 497.7519, effective_beta_dimKL_factor: 0.4978
Epoch 78 - Valid Loss: 89.6511
Sub-losses: recon_loss: 70.9173, mut_info_loss: 7.7884, tot_corr_loss: 9.8553, dimwise_kl_loss: 1.0902, anneal_factor: 0.9955, effective_beta_mi_factor: 9.9550, effective_beta_tc_factor: 497.7519, effective_beta_dimKL_factor: 0.4978
Epoch 79 - Train Loss: 82.0051
Sub-losses: recon_loss: 73.2434, mut_info_loss: 7.4189, tot_corr_loss: 0.1713, dimwise_kl_loss: 1.1715, anneal_factor: 0.9963, effective_beta_mi_factor: 9.9632, effective_beta_tc_factor: 498.1579, effective_beta_dimKL_factor: 0.4982
Epoch 79 - Valid Loss: 86.9569
Sub-losses: recon_loss: 71.8546, mut_info_loss: 4.5874, tot_corr_loss: 9.5487, dimwise_kl_loss: 0.9662, anneal_factor: 0.9963, effective_beta_mi_factor: 9.9632, effective_beta_tc_factor: 498.1579, effective_beta_dimKL_factor: 0.4982
Epoch 80 - Train Loss: 90.8018
Sub-losses: recon_loss: 73.3497, mut_info_loss: 7.1542, tot_corr_loss: 9.3134, dimwise_kl_loss: 0.9846, anneal_factor: 0.9970, effective_beta_mi_factor: 9.9698, effective_beta_tc_factor: 498.4908, effective_beta_dimKL_factor: 0.4985
Epoch 80 - Valid Loss: 75.5681
Sub-losses: recon_loss: 69.6685, mut_info_loss: 4.8657, tot_corr_loss: 0.0000, dimwise_kl_loss: 1.0338, anneal_factor: 0.9970, effective_beta_mi_factor: 9.9698, effective_beta_tc_factor: 498.4908, effective_beta_dimKL_factor: 0.4985
Epoch 81 - Train Loss: 86.9559
Sub-losses: recon_loss: 73.1558, mut_info_loss: 7.2109, tot_corr_loss: 5.5741, dimwise_kl_loss: 1.0151, anneal_factor: 0.9975, effective_beta_mi_factor: 9.9753, effective_beta_tc_factor: 498.7637, effective_beta_dimKL_factor: 0.4988
Epoch 81 - Valid Loss: 75.7702
Sub-losses: recon_loss: 67.0890, mut_info_loss: 7.4036, tot_corr_loss: 0.0000, dimwise_kl_loss: 1.2776, anneal_factor: 0.9975, effective_beta_mi_factor: 9.9753, effective_beta_tc_factor: 498.7637, effective_beta_dimKL_factor: 0.4988
Epoch 82 - Train Loss: 80.8437
Sub-losses: recon_loss: 73.0845, mut_info_loss: 6.5807, tot_corr_loss: 0.2462, dimwise_kl_loss: 0.9323, anneal_factor: 0.9980, effective_beta_mi_factor: 9.9797, effective_beta_tc_factor: 498.9873, effective_beta_dimKL_factor: 0.4990
Epoch 82 - Valid Loss: 91.3709
Sub-losses: recon_loss: 68.8820, mut_info_loss: 7.0396, tot_corr_loss: 14.1989, dimwise_kl_loss: 1.2504, anneal_factor: 0.9980, effective_beta_mi_factor: 9.9797, effective_beta_tc_factor: 498.9873, effective_beta_dimKL_factor: 0.4990
Epoch 83 - Train Loss: 85.7543
Sub-losses: recon_loss: 73.2366, mut_info_loss: 6.8997, tot_corr_loss: 4.7067, dimwise_kl_loss: 0.9113, anneal_factor: 0.9983, effective_beta_mi_factor: 9.9834, effective_beta_tc_factor: 499.1706, effective_beta_dimKL_factor: 0.4992
Epoch 83 - Valid Loss: 91.2359
Sub-losses: recon_loss: 70.6704, mut_info_loss: 4.8513, tot_corr_loss: 14.9172, dimwise_kl_loss: 0.7970, anneal_factor: 0.9983, effective_beta_mi_factor: 9.9834, effective_beta_tc_factor: 499.1706, effective_beta_dimKL_factor: 0.4992
Epoch 84 - Train Loss: 84.6141
Sub-losses: recon_loss: 72.1935, mut_info_loss: 7.2564, tot_corr_loss: 4.2640, dimwise_kl_loss: 0.9001, anneal_factor: 0.9986, effective_beta_mi_factor: 9.9864, effective_beta_tc_factor: 499.3207, effective_beta_dimKL_factor: 0.4993
Epoch 84 - Valid Loss: 97.8894
Sub-losses: recon_loss: 68.3651, mut_info_loss: 6.3521, tot_corr_loss: 22.1187, dimwise_kl_loss: 1.0535, anneal_factor: 0.9986, effective_beta_mi_factor: 9.9864, effective_beta_tc_factor: 499.3207, effective_beta_dimKL_factor: 0.4993
Epoch 85 - Train Loss: 85.3357
Sub-losses: recon_loss: 73.0181, mut_info_loss: 7.1045, tot_corr_loss: 4.3302, dimwise_kl_loss: 0.8828, anneal_factor: 0.9989, effective_beta_mi_factor: 9.9889, effective_beta_tc_factor: 499.4437, effective_beta_dimKL_factor: 0.4994
Epoch 85 - Valid Loss: 82.4889
Sub-losses: recon_loss: 74.3478, mut_info_loss: 2.9533, tot_corr_loss: 4.3102, dimwise_kl_loss: 0.8777, anneal_factor: 0.9989, effective_beta_mi_factor: 9.9889, effective_beta_tc_factor: 499.4437, effective_beta_dimKL_factor: 0.4994
Epoch 86 - Train Loss: 83.0611
Sub-losses: recon_loss: 72.8042, mut_info_loss: 6.2032, tot_corr_loss: 3.2283, dimwise_kl_loss: 0.8254, anneal_factor: 0.9991, effective_beta_mi_factor: 9.9909, effective_beta_tc_factor: 499.5445, effective_beta_dimKL_factor: 0.4995
Epoch 86 - Valid Loss: 86.8072
Sub-losses: recon_loss: 68.8227, mut_info_loss: 5.9980, tot_corr_loss: 10.7752, dimwise_kl_loss: 1.2114, anneal_factor: 0.9991, effective_beta_mi_factor: 9.9909, effective_beta_tc_factor: 499.5445, effective_beta_dimKL_factor: 0.4995
Epoch 87 - Train Loss: 85.7644
Sub-losses: recon_loss: 73.9089, mut_info_loss: 6.6542, tot_corr_loss: 4.3141, dimwise_kl_loss: 0.8872, anneal_factor: 0.9993, effective_beta_mi_factor: 9.9925, effective_beta_tc_factor: 499.6270, effective_beta_dimKL_factor: 0.4996
Epoch 87 - Valid Loss: 94.6195
Sub-losses: recon_loss: 67.7971, mut_info_loss: 6.4803, tot_corr_loss: 19.6011, dimwise_kl_loss: 0.7411, anneal_factor: 0.9993, effective_beta_mi_factor: 9.9925, effective_beta_tc_factor: 499.6270, effective_beta_dimKL_factor: 0.4996
Epoch 88 - Train Loss: 82.1939
Sub-losses: recon_loss: 73.2199, mut_info_loss: 7.3155, tot_corr_loss: 0.6379, dimwise_kl_loss: 1.0206, anneal_factor: 0.9994, effective_beta_mi_factor: 9.9939, effective_beta_tc_factor: 499.6945, effective_beta_dimKL_factor: 0.4997
Epoch 88 - Valid Loss: 80.3662
Sub-losses: recon_loss: 69.1944, mut_info_loss: 5.6055, tot_corr_loss: 4.4852, dimwise_kl_loss: 1.0811, anneal_factor: 0.9994, effective_beta_mi_factor: 9.9939, effective_beta_tc_factor: 499.6945, effective_beta_dimKL_factor: 0.4997
Epoch 89 - Train Loss: 85.2753
Sub-losses: recon_loss: 73.0402, mut_info_loss: 6.2212, tot_corr_loss: 5.0735, dimwise_kl_loss: 0.9404, anneal_factor: 0.9995, effective_beta_mi_factor: 9.9950, effective_beta_tc_factor: 499.7499, effective_beta_dimKL_factor: 0.4997
Epoch 89 - Valid Loss: 78.8232
Sub-losses: recon_loss: 69.4966, mut_info_loss: 6.5241, tot_corr_loss: 1.6246, dimwise_kl_loss: 1.1779, anneal_factor: 0.9995, effective_beta_mi_factor: 9.9950, effective_beta_tc_factor: 499.7499, effective_beta_dimKL_factor: 0.4997
Epoch 90 - Train Loss: 83.3244
Sub-losses: recon_loss: 74.7916, mut_info_loss: 6.0050, tot_corr_loss: 1.7331, dimwise_kl_loss: 0.7947, anneal_factor: 0.9996, effective_beta_mi_factor: 9.9959, effective_beta_tc_factor: 499.7952, effective_beta_dimKL_factor: 0.4998
Epoch 90 - Valid Loss: 82.4408
Sub-losses: recon_loss: 69.3042, mut_info_loss: 4.9991, tot_corr_loss: 7.3458, dimwise_kl_loss: 0.7918, anneal_factor: 0.9996, effective_beta_mi_factor: 9.9959, effective_beta_tc_factor: 499.7952, effective_beta_dimKL_factor: 0.4998
Epoch 91 - Train Loss: 81.7873
Sub-losses: recon_loss: 73.9576, mut_info_loss: 5.9283, tot_corr_loss: 1.1420, dimwise_kl_loss: 0.7595, anneal_factor: 0.9997, effective_beta_mi_factor: 9.9966, effective_beta_tc_factor: 499.8323, effective_beta_dimKL_factor: 0.4998
Epoch 91 - Valid Loss: 76.3595
Sub-losses: recon_loss: 68.7829, mut_info_loss: 4.7051, tot_corr_loss: 2.2422, dimwise_kl_loss: 0.6293, anneal_factor: 0.9997, effective_beta_mi_factor: 9.9966, effective_beta_tc_factor: 499.8323, effective_beta_dimKL_factor: 0.4998
Epoch 92 - Train Loss: 83.8784
Sub-losses: recon_loss: 73.9651, mut_info_loss: 5.9293, tot_corr_loss: 3.2085, dimwise_kl_loss: 0.7755, anneal_factor: 0.9997, effective_beta_mi_factor: 9.9973, effective_beta_tc_factor: 499.8627, effective_beta_dimKL_factor: 0.4999
Epoch 92 - Valid Loss: 79.4777
Sub-losses: recon_loss: 66.8911, mut_info_loss: 7.9211, tot_corr_loss: 3.5606, dimwise_kl_loss: 1.1050, anneal_factor: 0.9997, effective_beta_mi_factor: 9.9973, effective_beta_tc_factor: 499.8627, effective_beta_dimKL_factor: 0.4999
Epoch 93 - Train Loss: 84.8561
Sub-losses: recon_loss: 73.9394, mut_info_loss: 6.3240, tot_corr_loss: 3.7363, dimwise_kl_loss: 0.8565, anneal_factor: 0.9998, effective_beta_mi_factor: 9.9978, effective_beta_tc_factor: 499.8876, effective_beta_dimKL_factor: 0.4999
Epoch 93 - Valid Loss: 96.8628
Sub-losses: recon_loss: 69.7907, mut_info_loss: 6.3553, tot_corr_loss: 19.8555, dimwise_kl_loss: 0.8613, anneal_factor: 0.9998, effective_beta_mi_factor: 9.9978, effective_beta_tc_factor: 499.8876, effective_beta_dimKL_factor: 0.4999
Epoch 94 - Train Loss: 83.3784
Sub-losses: recon_loss: 73.7555, mut_info_loss: 6.4873, tot_corr_loss: 2.4011, dimwise_kl_loss: 0.7346, anneal_factor: 0.9998, effective_beta_mi_factor: 9.9982, effective_beta_tc_factor: 499.9080, effective_beta_dimKL_factor: 0.4999
Epoch 94 - Valid Loss: 88.4066
Sub-losses: recon_loss: 71.3110, mut_info_loss: 4.8853, tot_corr_loss: 11.4461, dimwise_kl_loss: 0.7642, anneal_factor: 0.9998, effective_beta_mi_factor: 9.9982, effective_beta_tc_factor: 499.9080, effective_beta_dimKL_factor: 0.4999
Epoch 95 - Train Loss: 84.6043
Sub-losses: recon_loss: 73.7275, mut_info_loss: 6.5264, tot_corr_loss: 3.4098, dimwise_kl_loss: 0.9407, anneal_factor: 0.9998, effective_beta_mi_factor: 9.9985, effective_beta_tc_factor: 499.9247, effective_beta_dimKL_factor: 0.4999
Epoch 95 - Valid Loss: 76.0807
Sub-losses: recon_loss: 70.2444, mut_info_loss: 5.0360, tot_corr_loss: 0.0000, dimwise_kl_loss: 0.8003, anneal_factor: 0.9998, effective_beta_mi_factor: 9.9985, effective_beta_tc_factor: 499.9247, effective_beta_dimKL_factor: 0.4999
Epoch 96 - Train Loss: 82.7106
Sub-losses: recon_loss: 73.3685, mut_info_loss: 6.4787, tot_corr_loss: 2.0398, dimwise_kl_loss: 0.8237, anneal_factor: 0.9999, effective_beta_mi_factor: 9.9988, effective_beta_tc_factor: 499.9383, effective_beta_dimKL_factor: 0.4999
Epoch 96 - Valid Loss: 76.8490
Sub-losses: recon_loss: 70.0257, mut_info_loss: 5.1001, tot_corr_loss: 0.8578, dimwise_kl_loss: 0.8654, anneal_factor: 0.9999, effective_beta_mi_factor: 9.9988, effective_beta_tc_factor: 499.9383, effective_beta_dimKL_factor: 0.4999
Epoch 97 - Train Loss: 82.2547
Sub-losses: recon_loss: 74.3908, mut_info_loss: 5.5340, tot_corr_loss: 1.6064, dimwise_kl_loss: 0.7235, anneal_factor: 0.9999, effective_beta_mi_factor: 9.9990, effective_beta_tc_factor: 499.9495, effective_beta_dimKL_factor: 0.4999
Epoch 97 - Valid Loss: 91.1598
Sub-losses: recon_loss: 69.2282, mut_info_loss: 6.1049, tot_corr_loss: 15.0330, dimwise_kl_loss: 0.7937, anneal_factor: 0.9999, effective_beta_mi_factor: 9.9990, effective_beta_tc_factor: 499.9495, effective_beta_dimKL_factor: 0.4999
Epoch 98 - Train Loss: 81.8895
Sub-losses: recon_loss: 74.4782, mut_info_loss: 4.8999, tot_corr_loss: 1.6916, dimwise_kl_loss: 0.8197, anneal_factor: 0.9999, effective_beta_mi_factor: 9.9992, effective_beta_tc_factor: 499.9586, effective_beta_dimKL_factor: 0.5000
Epoch 98 - Valid Loss: 75.7760
Sub-losses: recon_loss: 68.9611, mut_info_loss: 6.0459, tot_corr_loss: 0.0000, dimwise_kl_loss: 0.7689, anneal_factor: 0.9999, effective_beta_mi_factor: 9.9992, effective_beta_tc_factor: 499.9586, effective_beta_dimKL_factor: 0.5000
Epoch 99 - Train Loss: 83.7442
Sub-losses: recon_loss: 73.3577, mut_info_loss: 5.9258, tot_corr_loss: 3.6229, dimwise_kl_loss: 0.8378, anneal_factor: 0.9999, effective_beta_mi_factor: 9.9993, effective_beta_tc_factor: 499.9661, effective_beta_dimKL_factor: 0.5000
Epoch 99 - Valid Loss: 88.0465
Sub-losses: recon_loss: 68.3548, mut_info_loss: 5.4746, tot_corr_loss: 13.2589, dimwise_kl_loss: 0.9582, anneal_factor: 0.9999, effective_beta_mi_factor: 9.9993, effective_beta_tc_factor: 499.9661, effective_beta_dimKL_factor: 0.5000
Epoch 100 - Train Loss: 81.9863
Sub-losses: recon_loss: 74.4416, mut_info_loss: 5.4326, tot_corr_loss: 1.4447, dimwise_kl_loss: 0.6674, anneal_factor: 0.9999, effective_beta_mi_factor: 9.9994, effective_beta_tc_factor: 499.9723, effective_beta_dimKL_factor: 0.5000
Epoch 100 - Valid Loss: 76.2047
Sub-losses: recon_loss: 69.6400, mut_info_loss: 5.8681, tot_corr_loss: 0.0000, dimwise_kl_loss: 0.6966, anneal_factor: 0.9999, effective_beta_mi_factor: 9.9994, effective_beta_tc_factor: 499.9723, effective_beta_dimKL_factor: 0.5000

After conducting the pipeline with .run(), the pipeline function .visualize() is already called and created the plots for model weights and loss curves.

We have now three ways to display them:

• Loss curves, but not model weights, are part of the general show_result() function.
• Specific display functions for loss show_loss() and model weights show_weights()
• accessing figure handles stored in disent.visualizer.plots dictionary

In [2]:

# Using show_result
disent.show_result()

# Using show_result disent.show_result()

Creating plots ...

OMP: Info #276: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.

In [3]:

# Using specific display functions via disent.visualizer
fig_weights = disent.visualizer.show_weights()
fig_loss_abs = disent.visualizer.show_loss(plot_type="absolute") # Displays all loss components and factors over epochs
fig_loss_rel = disent.visualizer.show_loss(plot_type="relative") # Displays contribution of loss components to total loss over epochs

# Using specific display functions via disent.visualizer fig_weights = disent.visualizer.show_weights() fig_loss_abs = disent.visualizer.show_loss(plot_type="absolute") # Displays all loss components and factors over epochs fig_loss_rel = disent.visualizer.show_loss(plot_type="relative") # Displays contribution of loss components to total loss over epochs

In [4]:

# All figure handles are also accessible via disent.visualizer.plots dictionary
disent.visualizer.plots

# All figure handles are also accessible via disent.visualizer.plots dictionary disent.visualizer.plots

Out[4]:

defaultdict(<function autoencodix.utils._utils.nested_dict()>,
            {'ModelWeights': <Figure size 3000x1500 with 8 Axes>,
             'loss_absolute': <Figure size 4500x500 with 9 Axes>,
             'loss_relative': <seaborn._core.plot.Plot at 0x3563ea0b0>,
             'Ridgeline': defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                         {99: defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                                      {'all': defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                                                   {'user_label': <seaborn.axisgrid.FacetGrid at 0x15cb70280>})})}),
             '2D-scatter': defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                         {99: defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                                      {'all': defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                                                   {'user_label': <Figure size 1200x800 with 1 Axes>})})})})

2. Latent space visualization¶

We offer different views on the latent space for different tasks and insights which are all accessible via show_latent_space() and the options plot_type=:

• 2D-scatter - Uses UMAP as dimension reduction if latent_dim>2 to display a 2D scatter (samples) plot representation of the latent space
• Ridgeline - Distributions (per sample class) of latent intensities for each latent dimension
• Clustermap- Heatmap of meant latent intensities per sample class and latent dimension. Classes are clustered by correlation along latent dimensions.
• Coverage-Correlation - Displays average coverage and total correlation across latent dimensions to inspect denseness and disentanglement over epochs.

In [5]:

disent._datasets.test.metadata

disent._datasets.test.metadata

Out[5]:

{'transcriptomics':               condition    batch  quality_score
 sample_103  condition_2  batch_3       0.914741
 sample_110  condition_1  batch_3       0.950450
 sample_111  condition_1  batch_2       0.678535
 sample_115  condition_2  batch_2       0.763138
 sample_120  condition_1  batch_3       0.699364
 ...                 ...      ...            ...
 sample_78   condition_0  batch_1       0.924776
 sample_8    condition_2  batch_1       0.729813
 sample_95   condition_2  batch_3       0.763671
 sample_96   condition_1  batch_2       0.919112
 sample_98   condition_0  batch_1       0.703970
 
 [100 rows x 3 columns],
 'proteomics':               condition    batch  quality_score
 sample_103  condition_2  batch_3       0.914741
 sample_110  condition_1  batch_3       0.950450
 sample_111  condition_1  batch_2       0.678535
 sample_115  condition_2  batch_2       0.763138
 sample_120  condition_1  batch_3       0.699364
 ...                 ...      ...            ...
 sample_78   condition_0  batch_1       0.924776
 sample_8    condition_2  batch_1       0.729813
 sample_95   condition_2  batch_3       0.763671
 sample_96   condition_1  batch_2       0.919112
 sample_98   condition_0  batch_1       0.703970
 
 [100 rows x 3 columns]}

In [6]:

params = ["condition"] # Specify metadata parameters to color/label the plots

fig_latent_2D = disent.visualizer.show_latent_space(result=result, plot_type="2D-scatter", param=params)
fig_latent_ridge = disent.visualizer.show_latent_space(result=result, plot_type="Ridgeline", param=params)
fig_latent_clustermap = disent.visualizer.show_latent_space(result=result, plot_type="Clustermap", param=params)

params = ["condition"] # Specify metadata parameters to color/label the plots fig_latent_2D = disent.visualizer.show_latent_space(result=result, plot_type="2D-scatter", param=params) fig_latent_ridge = disent.visualizer.show_latent_space(result=result, plot_type="Ridgeline", param=params) fig_latent_clustermap = disent.visualizer.show_latent_space(result=result, plot_type="Clustermap", param=params)

In [7]:

# To increase resolution over epochs, set the config parameter 'checkpoint_interval=1' before training
fig_latent_cov_tc = disent.visualizer.show_latent_space(result=result, plot_type="Coverage-Correlation")
# TODO Coverage calculation fails for small valid split

# To increase resolution over epochs, set the config parameter 'checkpoint_interval=1' before training fig_latent_cov_tc = disent.visualizer.show_latent_space(result=result, plot_type="Coverage-Correlation") # TODO Coverage calculation fails for small valid split

/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(
/Users/maximilianjoas/development/autoencodix_package/src/autoencodix/visualize/_general_visualizer.py:788: UserWarning: Coverage calculation fails since combination of sample size and latent dimension results in less than 2 bins.
  warnings.warn(

Advanced options in latent space visualization¶

For a deep analysis of the latent space we offer additional options in show_latent_space():

• param - List of parameters to color plots and already provided as metadata before training. Strings must match column names. If not a list, string "all" is expected for convenient way to make plots for all parameters available.
• labels - List or pd.Series of labels of each sample to color the latent space. If provided as list this must match the order of samples.
• You can use instead the config parameter annotation_columns to specify a default list of parameters to be used for plotting
• split - The data split to visualize. Options are "train", "valid", "test", and "all". Default is "all".
• epoch - The epoch number to visualize. If None, the last epoch is used (default). Warning: this only works for epochs which are checkpointed, see config parameter checkpoint_interval.
• n_downsample - To speed up visualization, visualization is performed only on a random subset. Default is 10000.

Example: we want to display the latent space at the very beginning of the training only for samples in the train split.

Further, we have new labels which we will provide as a pd.Series.

In [8]:

import pandas as pd
disease_labels = ["influenza"] * (len(disent._datasets.train.sample_ids)//2) + ["covid"] * (len(disent._datasets.train.sample_ids)//2) 
train_samples = pd.Series(data=disease_labels, index=disent._datasets.train.sample_ids, name="disease")

fig_latent_epoch0 = disent.visualizer.show_latent_space(
	result=result,
	plot_type="Ridgeline",
	labels=train_samples,
	split="train",
	epoch=0
)

import pandas as pd disease_labels = ["influenza"] * (len(disent._datasets.train.sample_ids)//2) + ["covid"] * (len(disent._datasets.train.sample_ids)//2) train_samples = pd.Series(data=disease_labels, index=disent._datasets.train.sample_ids, name="disease") fig_latent_epoch0 = disent.visualizer.show_latent_space( result=result, plot_type="Ridgeline", labels=train_samples, split="train", epoch=0 )

3. Embedding evaluation¶

Check out the Evaluate Tutorial for a deep dive into how to do the evaluation.

Here, is a quick example on how to plot the results after embedding evaluation.

In [9]:

# Perform embedding evaluation (not part of .run())
params = ["condition", "batch"]
result = disent.evaluate(params=params)

# Perform embedding evaluation (not part of .run()) params = ["condition", "batch"] result = disent.evaluate(params=params)

Perform ML task with feature df: Latent
Latent
Perform ML task for target parameter: condition
Perform ML task for target parameter: batch

In [10]:

from sklearn.linear_model import LogisticRegression

fig_condition_eval = disent.visualizer.show_evaluation(
	param=params[0], # We plot one parameter at a time
	metric="roc_auc_ovo", # This is the default metric for classification tasks
	ml_alg=str(LogisticRegression()) # This is the default ML algorithm for classification tasks
)

from sklearn.linear_model import LogisticRegression fig_condition_eval = disent.visualizer.show_evaluation( param=params[0], # We plot one parameter at a time metric="roc_auc_ovo", # This is the default metric for classification tasks ml_alg=str(LogisticRegression()) # This is the default ML algorithm for classification tasks )

4. Saving and customization¶

Saving¶

All plots in the dictionary in disent.visualizer.plots can be saved via the function save_plots().

In case you only want to save a subset, you can use the option which and provide a list of plots to be saved.

In [11]:

# All available plots
disent.visualizer.plots

# All available plots disent.visualizer.plots

Out[11]:

defaultdict(<function autoencodix.utils._utils.nested_dict()>,
            {'ModelWeights': <Figure size 3000x1500 with 8 Axes>,
             'loss_absolute': <Figure size 4500x500 with 9 Axes>,
             'loss_relative': <seaborn._core.plot.Plot at 0x3563ea0b0>,
             'Ridgeline': defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                         {99: defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                                      {'all': defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                                                   {'user_label': <seaborn.axisgrid.FacetGrid at 0x15cb70280>,
                                                    'condition': <seaborn.axisgrid.FacetGrid at 0x35bdbf8e0>})}),
                          0: defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                                      {'train': defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                                                   {'disease': <seaborn.axisgrid.FacetGrid at 0x35e49cf10>})})}),
             '2D-scatter': defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                         {99: defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                                      {'all': defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                                                   {'user_label': <Figure size 1200x800 with 1 Axes>,
                                                    'condition': <Figure size 1200x800 with 1 Axes>})})}),
             'Clustermap': defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                         {99: defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                                      {'all': defaultdict(<function autoencodix.utils._utils.nested_dict()>,
                                                   {'condition': <Figure size 700x550 with 4 Axes>})})}),
             'Coverage-Correlation': <Figure size 1200x500 with 2 Axes>,
             'ML_Evaluation': {'condition': {'roc_auc_ovo': {'LogisticRegression()': <seaborn.axisgrid.FacetGrid at 0x35959bca0>}},
              'batch': {'roc_auc_ovo': {'LogisticRegression()': <seaborn.axisgrid.FacetGrid at 0x15f9e4040>}}}})

In [12]:

my_selection = ["loss_relative", "Ridgeline", "ML_Evaluation"]

disent.visualizer.save_plots(
	path="./myplots/",
	which=my_selection, # Options are "all" or a list of plot names as in disent.visualizer.plots.keys()
	format="png" # Options are "png", "pdf, "svg"
)

my_selection = ["loss_relative", "Ridgeline", "ML_Evaluation"] disent.visualizer.save_plots( path="./myplots/", which=my_selection, # Options are "all" or a list of plot names as in disent.visualizer.plots.keys() format="png" # Options are "png", "pdf, "svg" )

Customization¶

The stored figure handles can be recalled to customize labels and styles to your liking.

In [13]:

import matplotlib.pyplot as plt

fig_weights = disent.visualizer.plots["ModelWeights"]

# change the title
fig_weights.suptitle("My custom title for model weights")

fig_weights.figure

import matplotlib.pyplot as plt fig_weights = disent.visualizer.plots["ModelWeights"] # change the title fig_weights.suptitle("My custom title for model weights") fig_weights.figure

Out[13]:

5. XModalix and Imagix specialities¶

XModalix and Imagix behave differently for some visualizations. This is related to the fact they are composed of multiple autoencoders (XModalix) and support images as data modality.

Differences in standard visualizations¶

• We do not provive a model weight visualization for more complex autoencoders (...yet...)
• Similarly, coverage and total correlation Coverage-Correlation is not supported (...yet...)
• Supported latent space visualizations are Ridgeline and 2D-scatter and show each VAE as subplot
• Loss curves behave as before and are expanded to each loss term for complex autoencoders

Additional plots for XModalix¶

At the core XModalix tries learn a joint latent space to enable translation across modalities. To check the capabilities of translation, we provide additional visualizations:

1. xmodalix.visualizer.show_2D_translation() - This plot compares on the test-split the translated_modality reconstruction by the XModalix with the original input in joint 2D repesentation calculated used a specified reducer method (default UMAP). A well trained XModalix and good translated modality (right plot) should as similar as possible to the original input (left plot).
2. xmodalix.visualizer.show_image_translation() - For images as target modality of translation, translation capability can be checked by visual comparison of test samples, possibly across classes specified under param, from a) original b) translated from_key modality to to_key modality and c) reference reconstruction of the image VAE inside the XModalix.

For an example checkout the XModalix Tutorial

Additional plot for Imagix¶

Analogously to XModalix, imagix.visualizer.show_image_recon_grid() provides a grid of test sample images showing a comparison of original images and reconstructed images by the Imagix.