Skip to content

DualPathBlock

Table of Contents

  1. Introduction
  2. Key Features
  3. Class Definition
  4. Example Usage
  5. Practical Tips
  6. Reference and Other Resources

Introduction

The DualPathBlock class is a PyTorch-based module or grammar that represents a basic computational unit in dual path networks. This class combines the output of two submodules by element-wise addition. The core idea behind this method is to efficiently use the information from both paths in a balanced way.

Key Features

  • Efficient combination of data: The DualPathBlock method combines data from two submodules in an effective way by using element-wise addition.

  • Flexibility in submodule choice: Users have the flexibility to choose the submodules, provided they are torch.nn.Module instances.

  • Simplicity and readability of code: Due to its modular design, the code is easy to understand, thereby making it easier for users to implement and modify.

  • Easy integration with other torch.nn.Module instances: The DualPathBlock can be easily integrated within other pipelines as a subnet.

Class Definition

The class design for DualPathBlock is very straightforward. It is initialized with two submodules that are instances of nn.Module. Then, during the forward pass, the inputs are passed through each submodule and the result of these computations is then computed by element-wise addition.

Parameters:

Parameter Type Description
submodule1 nn.Module First submodule through which input tensor x is passed.
submodule2 nn.Module Second submodule through which input tensor x is passed.

Methods:

Method Parameters Description
forward x: torch.Tensor Performs forward pass through the model. Calculates output tensor obtained by adding outputs of submodule1 and submodule2. Returns the computed tensor

Input / Output Type:

  • Input: Receives a tensor of any shape.
  • Output: Produces a tensor of the same shape as the inputs after the forward computation is done.

Example Usage 

# Import the necessary libraries
import torch
import torch.nn as nn

from zeta.nn import DualPathBlock

# Define two simple submodule
submodule1 = nn.Linear(20, 20)
submodule2 = nn.Linear(20, 20)

# Create an instance of DualPathBlock
dual_path_block = DualPathBlock(submodule1, submodule2)

# Define an input tensor
input_tensor = torch.randn(10, 20)

# Perform forward operation
output = dual_path_block(input_tensor)

# Print the output tensor
print(output)

Practical Tips

  • While DualPathBlock design allows for the use of any submodules, please make sure the outputs of both submodules can be summed up i.e., they are of the same shape.

  • DualPathBlock is particularly useful in constructing networks with parallel paths where the outputs are combined.

References and Other Resources

Pytorch Documentation

Dual Path Networks <-- If relevant