torch.nn

torch.nn

torch.nn enables ueres to quickly instantiate NN architectures by defining some of these high-level aspects as opposed to having to specify all the details manually.

 

import math
import torch

# assume a 256-dimensional input and a 4-dimensional output for this 1-layer NN
# hence, initialize a 256x4 dimensional matrix filled with random values
weights = torch.randn(256, 4) / math.sqrt(256)
# then ensure that the parameters of this NN are trainable
weights.requires_grad_()
# finally also add the bias for the 4-dimensional output, and make these trainable too
bias = torch.zeros(4, requires_grad=True)



######################################################################
# we can instead use torch.nn.Linear(256, 4) to represent the same thing
######################################################################

 

torch.nn.functional

within the torch.nn module, there is a submodule called torch.nn.functional. This submodel consists of all the functions within the torch.nn module whereas all the other submodules are classes. 

Example

import torch.nn.functional as F
loss_func = F.cross_entropy
loss = loss_func(model(x), y)