Adding columns to a matrix in loop

Question

I am trying to add selective columns from one matrix to another matrix in loop when condition becomes true. I have tried various combinations but it adds in rows with np.append etc commands, Any help is appreciated. Thanks

I have tried np.append, np.concatenation, np.hstack commands but not getting the desired soltuion.

n,m=np.shape(K)
Z=np.array([])
for number in range(m):
    A=function
    if A<0.05:
        Z = np.append(Z,np.vstack(K[:,number]))

I want Z matrix with columns of K that satisfy the condition A. Like Z=[K[:,3] K[:,8] K[:,10]]

Hello and welcome to Stack Overflow. That's a good start, but what would be best for us to solve your problem is a minimal reproducible example. We should be able to copy and paste your code in our favorite IDE and correct it accordingly to get the output you should also provide. :) — IMCoins
– IMCoins, Commented Jan 24, 2019 at 7:27

hpaulj · Accepted Answer · 2019-01-24 07:52:27Z

2

In [21]: Z=np.array([])
In [22]: Z.shape
Out[22]: (0,)

How do you expect to add something to an array with this shape?

np.append does work, but only because if first makes sure Z is atleast 1d, that is it ravels it.

In [23]: np.append(Z,np.arange(10))
Out[23]: array([0., 1., 2., 3., 4., 5., 6., 7., 8., 9.])

We need some sort of big bold letters in numpy, saying DO NOT TRY TO APPEND (or concatenate or stack) ARRAYS IN A LOOP.

Did you try building a list of columns, and then using just one concatenate?

alist = []
for i in [3,8,10]:
    alist.append(K[:,[i]])
print(alist)
Z = np.concatenate(alist, axis=1)
# Z=[K[:,3] K[:,8] K[:,10]]

Or

alist = []
alist.append(3); alist.append(8); alist.append(10)
Z = K[:,alist]

Repeatedly concatenating arrays has two big problems - it is slow, and it is hard to get started. That (0,) shape array is hot a valid substitute for the empty list.

If you need to do things iteratively, think lists. list append is relatively fast, and simple to use.

edited Jan 24, 2019 at 7:52

answered Jan 24, 2019 at 7:30

hpaulj

233k14 gold badges260 silver badges392 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Rayyan Khan Over a year ago

I am new to python and didn't know much about it, however Thanks for your kind response.

hpaulj Over a year ago

But evidently you know enough Python to use the list append, and to attempt to imitate it in numpy. np.append really needs some big bold warning notes. Too many people misuse it. At least you realized that it didn't operate in-place.

Mohanavel T · Accepted Answer · 2019-01-24 07:50:12Z

0

Numpy is actually meant for fast/robust matrix operations, such as multiplication, addition etc.

Since, you are dealing with sort rows/columns tables structure and manipulations. I strongly recommend you to use Pandas Dataframe

You can easily add columns or even merge two matrices based on condition (yes, like sql joins).

Look at df.assign

df = pd.Dataframe(npArray)
df.assign('newcolumn', yourNewColumnValuesAsList)

answered Jan 24, 2019 at 7:50

Mohanavel T

3914 silver badges18 bronze badges

Collectives™ on Stack Overflow

Adding columns to a matrix in loop

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related