ValueError: could not broadcast input array from shape (1,3) into shape (3,1)

Question

Im working on a linear regression coding problem and i get this error on trying to code the feature matrix part. can you please help me correct this ?

Traceback (most recent call last): File "C:\Users\visha\AppData\Local\Continuum\anaconda3\lib\site-packages\nose\case.py", line 197, in runTest self.test(*self.arg) File "C:\Users\visha\machinelearning\test.py", line 22, in test_compute_Phi Phi = compute_Phi(x,2) File "C:\Users\visha\machinelearning\linear_regression.py", line 30, in compute_Phi Phi[:,i] = np.power(x,i).reshape(x.shape[0],) ValueError: could not broadcast input array from shape (1,3) into shape (3,1)

[code]

def compute_Phi(x,p):
    x = np.asmatrix(x)
    Phi = np.zeros(shape = (x.shape[0],p))
    for i in range(0,p):
        Phi[:,i] = np.power(x,i).reshape(x.shape[0],)
        Phi = np.asmatrix(Phi)
return Phi

Hi @Dev, x is np.mat('1.;2.;3') and p is 2 in the test case. — VishwaV
– VishwaV, Commented Sep 20, 2019 at 6:00
Hi @hpaulj, yeah I changed it but still gives me the same error. — VishwaV
– VishwaV, Commented Sep 20, 2019 at 6:00
You created Phi as 2d ndarray. Why did you pass it through np.matrix in the loop? np.matrix behave differently when indexed. That produces confusion, which is part of why their use in new code is discouraged. — hpaulj
– hpaulj, Commented Sep 20, 2019 at 6:18

hpaulj · Accepted Answer · 2019-09-20 06:16:25Z

Your x, without the np.mat:

In [225]: x = np.array([1,2,3])[:,None]                                                
In [226]: x                                                                            
Out[226]: 
array([[1],
       [2],
       [3]])
In [227]: p = 2                                                                        
In [228]: Phi = np.zeros((3,2))                                                        
In [229]: Phi[:,0] = np.power(x,0)                                                     
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-229-f8ff29de133c> in <module>
----> 1 Phi[:,0] = np.power(x,0)

ValueError: could not broadcast input array from shape (3,1) into shape (3)

Why? x has shape (3,1) (as defined). But Phi[:,0] has shape (3,). By broadcasting rules the (3,1) can't be put in a (3,) space. A (1,3) could. A (3,) could.

Let's make x (3,):

In [230]: x = np.array([1,2,3])                                                        
In [231]: Phi[:,0] = np.power(x,0)                                                     
In [232]: Phi[:,1] = np.power(x,1)                                                     
In [233]: Phi                                                                          
Out[233]: 
array([[1., 1.],
       [1., 2.],
       [1., 3.]])

Now we can assign the columns.

Now the (3,1) shape x can work with multiple powers at once:

In [234]: np.power(x[:,None],[0,1,2,3])                                                
Out[234]: 
array([[ 1,  1,  1,  1],
       [ 1,  2,  4,  8],
       [ 1,  3,  9, 27]])

Here the (3,1) x broadcasts with a (4,) p to produce a (3,4) result.

The broadcasting steps are: (3,1), (4,) => (3,1), (1,4) => (3,4), (3,4)

The keys are - size 1 dimensions can be added automatically in the lead position. And size 1 dimensions are scaled to match the others.

Dev Khadka · Accepted Answer · 2019-09-20 07:08:01Z

0

Actually the error is because in 1st iteration of for loop you Phi is np.array and in second iteration it is changed to matrix. It is working if you moved the line Phi = np.asmatrix(Phi) outside of the loop

def compute_Phi(x,p):
    x = np.asmatrix(x)
    Phi = np.zeros(shape = (x.shape[0],p))
    Phi = np.asmatrix(Phi)
    for i in range(0,p):
        print(Phi[:,i])
        print(np.power(x,i))
        Phi[:,i] = np.power(x,i)

    return Phi 

compute_Phi(np.mat('1.;2.;3'), 2)

Output

matrix([[1., 1.],
        [1., 2.],
        [1., 3.]])

edited Sep 20, 2019 at 7:08

answered Sep 20, 2019 at 6:45

Dev Khadka

5,5415 gold badges23 silver badges36 bronze badges

7 Comments

VishwaV Over a year ago

Hi @Dev, the above code you gave gives me a type error now instead of the broadcast error. It says that, assert type(Phi) ==np.matrixlib.defmatrix.matrix

Dev Khadka Over a year ago

@VishwaV at what line, the code is working for me, in python 3

Dev Khadka Over a year ago

@VishwaV may be your function caller expecting matrix type, change the return statement to "return np.asmatrix(Phi)"

VishwaV Over a year ago

That works, but it shouldn't be done as the given return statement should not be changed in the assignment. Can you tell me another way to implement your above idea?

Dev Khadka Over a year ago

I edited the answer above plz check if that works for your

|

Collectives™ on Stack Overflow

ValueError: could not broadcast input array from shape (1,3) into shape (3,1)

2 Answers 2

Comments

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related