How to learn two functions simultaneously in using python (either pytorch or tensorflow)?

Question

I have three series of observations, namely Y, T, and X. I would like to study the differences between the predicted values of the two models. The first model is to learn g such that Y=g(T, X). The second model is to learn L and f such that Y=L(T)f(X). I have no problem in learning the first model using the PyTorch package or the Tensorflow package. However, I am not sure how to learn L and f. In using the PyTorch package, I can set up two feedforward MLPs with different hidden layers and inputs. For simplicity, I define a Feedforward MLP class as follows:

class Feedforward(t.nn.Module): # the definition of a feedforward neural network
    # Basic definition
    def __init__(self, input_size, hidden_size):
        super(Feedforward, self).__init__()
        self.input_size = input_size
        self.hidden_size  = hidden_size
        self.fc1 = t.nn.Linear(self.input_size, self.hidden_size)
        self.relu = t.nn.ReLU()
        self.fc2 = t.nn.Linear(self.hidden_size, 1)
        self.sigmoid = t.nn.Sigmoid()
    # Advance definition
    def forward(self, x):
        hidden = self.fc1(x)
        relu = self.relu(hidden)
        output = self.fc2(relu)
        output = self.sigmoid(output)
        return output

Suppose L=Feedforward(2,10) and L=Feedforward(3,9). From my understanding, I can only learn either L or f, but not both simultaneously. Is it possible to learn L and f simultaneously using Y, T, and X?

trialNerror · Accepted Answer · 2020-10-27 13:59:54Z

1

I may be missing something, but I think you can :

L = Feedforward(2,10)
f = Feedforward(3,9)
L_opt = Adam(L.parameters(), lr=...)
f_opt = Adam(f.parameters(), lr=...)
for (x,t,y) in dataset:
    L.zero_grad()
    f.zero_grad()
    y_pred = L(t)*f(x)
    loss = (y-y_pred)**2
    loss.backward()
    L_opt.step()
    f_opt.step()

You can also fuse them together in one single model :

class ProductModel(t.nn.Module):
    def __init__(self, L, f):
        self.L = L
        self.f = f
    def forward(self, x,t):
        return self.L(t)*self.f(x)

and then train this model like you trained g

answered Oct 27, 2020 at 13:59

trialNerror

3,59311 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

will_cheuk Over a year ago

However, if we fuse the two functions together in one single model, I am not sure how the model recognizes the inputs for L and inputs for f?

trialNerror Over a year ago

well it's up to you to do it in the forward function. In the one suggested above, there is no confusion between x and t, is it ?

Collectives™ on Stack Overflow

How to learn two functions simultaneously in using python (either pytorch or tensorflow)?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related