Linear regression with pytorch

Question

I tried to run linear regression on ForestFires dataset. Dataset is available on Kaggle and gist of my attempt is here: https://gist.github.com/Chandrak1907/747b1a6045bb64898d5f9140f4cf9a37

I am facing two problems:

Output from prediction is of shape 32x1 and target data shape is 32.

input and target shapes do not match: input [32 x 1], target [32]¶

Using view I reshaped predictions tensor.

y_pred = y_pred.view(inputs.shape[0])

Why there is a mismatch in shapes of predicted tensor and actual tensor?

SGD in pytorch never converges. I tried to compute MSE manually using

print(torch.mean((y_pred - labels)**2))

This value does not match

loss = criterion(y_pred,labels)

Can someone highlight where is the mistake in my code?

Thank you.

Viet Phan · Accepted Answer · 2018-07-30 17:00:06Z

2

Problem 1

This is reference about MSELoss from Pytorch docs: https://pytorch.org/docs/stable/nn.html#torch.nn.MSELoss

Shape:
 - Input: (N,∗) where * means, any number of additional dimensions
 - Target: (N,∗), same shape as the input

So, you need to expand dims of labels: (32) -> (32,1), by using: torch.unsqueeze(labels, 1) or labels.view(-1,1)

https://pytorch.org/docs/stable/torch.html#torch.unsqueeze

torch.unsqueeze(input, dim, out=None) → Tensor

Returns a new tensor with a dimension of size one inserted at the specified position.

The returned tensor shares the same underlying data with this tensor.

Problem 2

After reviewing your code, I realized that you have added size_average param to MSELoss:

criterion = torch.nn.MSELoss(size_average=False)

size_average (bool, optional) – Deprecated (see reduction). By default, the losses are averaged over each loss element in the batch. Note that for some losses, there multiple elements per sample. If the field size_average is set to False, the losses are instead summed for each minibatch. Ignored when reduce is False. Default: True

That's why 2 computed values not matched. This is sample code:

import torch
import torch.nn as nn

loss1 = nn.MSELoss()
loss2 = nn.MSELoss(size_average=False)
inputs = torch.randn(32, 1, requires_grad=True)
targets = torch.randn(32, 1)

output1 = loss1(inputs, targets)
output2 = loss2(inputs, targets)
output3 = torch.mean((inputs - targets) ** 2)

print(output1)  # tensor(1.0907)
print(output2)  # tensor(34.9021)
print(output3)  # tensor(1.0907)

edited Jul 30, 2018 at 17:00

answered Jul 30, 2018 at 4:32

Viet Phan

2,1293 gold badges25 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Chandra Over a year ago

Great. This resolved problem 1. However, loss computed using torch.mean((y_pred - labels)**2) are not matching loss = criterion(y_pred,labels). Can you pls let me know, how they are different?

Viet Phan Over a year ago

@Chandra I've realized that your code's using size_average param. I've updated my answer for detail

Chandra Over a year ago

Yes. Thank you.

Collectives™ on Stack Overflow

Linear regression with pytorch

1 Answer 1

Problem 1

Problem 2

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Problem 1

Problem 2

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related