Skip to content

Module/Function Name: MMLayerNorm

# Usage example:
import torch

from zeta.nn import MMLayerNorm

mm_ln = MMLayerNorm(num_modalities=2, dim=64)
modality1 = torch.randn(32, 10, 64)
modality2 = torch.randn(32, 10, 64)
mm_ln([modality1, modality2])
print(mm_ln)

Explanation:

The MMLayerNorm class represents a Multi-Modality Layer Normalization module that fuses and normalizes input tensors from different modalities. It helps in combining and normalizing information extracted from different sources, like images, text, etc.

The parameters are as follows: - num_modalities (int): The number of modalities to be fused. - dim (int): The dimension of the input tensors. - epsilon (float): A small value added to the denominator for numerical stability. Default value is 1e-5.

The MMLayerNorm class contains a method called forward that takes a list of input tensors representing different modalities and returns the output tensor after fusing and normalizing the modalities.

The usage example demonstrates how to instantiate the MMLayerNorm class and pass input tensors to obtain the fused and normalized output tensor.

Note: Ensure that the shapes of all the input modalities are identical. All modalities must have the same shape in order to perform fusion and normalization.

This code snippet can be used to create and use a Multi-Modality Layer Normalization module in neural network architectures that require combining input tensors from different modalities for processing. The class structure ensures that submodules are registered and their parameters are converted as expected.

For advanced usage and additional options, or to explore further, refer to the example provided above and the official PyTorch documentation.

Example References: - PyTorch nn.Module documentation: https://pytorch.org/docs/stable/generated/torch.nn.Module.html - PyTorch Layer Normalization: https://pytorch.org/docs/stable/generated/torch.nn.LayerNorm.html

These references provide further details and background information on how the MMLayerNorm class and other PyTorch modules can be utilized or extended, enabling developers to explore their full potential in designing and implementing machine learning models.