PyTorch Networks within a Model

Question

I would like to define a network that comprises many templates. Below under Network Definitions is a simplified example where the first network definition is used as a template in the second one. This doesn't work - when I initialise my optimiser is says that the network parameters are empty! How should I do this properly? The network that I ultimately want is very complicated.

Main Function

if __name__ == "__main__":

myNet       = Network().cuda().train()
optimizer   = optim.SGD(myNet.parameters(), lr=0.01, momentum=0.9)

Network definitions:

class NetworkTemplate(nn.Module):


def __init__(self):
    super(NetworkTemplate, self).__init__()
    self.conv1 = nn.Conv2d(1, 3, kernel_size=1, bias=False)
    self.bn1 = nn.BatchNorm2d(3)

def forward(self, x):
    x = self.conv1(x)
    x = self.bn1(x)

    return x

class Network(nn.Module):


def __init__(self, nNets):
    super(Network, self).__init__()

    self.nets = []
    for curNet in range(nNets):
        self.nets.append(NetworkTemplate())

    def forward(self, x):

        for curNet in self.nets:
            x = curNet(x)

        return x

Jan · Accepted Answer · 2020-06-16 11:58:45Z

2

Just use torch.nn.Sequential? Like self.nets=torch.nn.Sequential(*self.nets) after you populated self.nets and then call return self.nets(x) in your forward function?

If you want to do something more complicated, you can put all networks into torch.nn.ModuleList, however you'll need to manually take care of calling them in your forward method in that case (but it can be more complicated than just sequential).

edited Jun 16, 2020 at 11:58

answered Jun 16, 2020 at 11:51

Jan

1,26014 silver badges29 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

keyserSoze Over a year ago

Changing it with nn.Sequential now gives the error list is not a module subclass

Jan Over a year ago

My bad, I'd forgotten the * to unwrap self.nets when passing it to nn.Sequential. I've corrected the answer.

keyserSoze Over a year ago

This works! Thank you very much indeed. I will up-vote your answer. Can you please explain the * operator. Is this a Python thing, or is this a PyTorch thing?

Jan Over a year ago

No, that's python. Say you have function my_function(a,b,c) with three arguments. As it happens, your arguments are in a list my_list=['hello', 'world', '!']. You could now pass those arguments as my_function(my_list[0], my_list[1], my_list[2]), basically unwrapping the list yourself and passing each individually. Or you use *: my_function(*my_list), that is much easier!

Collectives™ on Stack Overflow

PyTorch Networks within a Model

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related