Creating column in pandas using vectorization

Question

I am trying to use vectorization on a pandas dataframe to create a new column. The dataframe is fairly huge(millions of records). I am showing a dummy example here. I am showing a non vecotorised version which works but is not very efficient. I am trying to implement the vectorised version while using the function(the actual function is fairly complicated than the one shown here).

import pandas as pd
import numpy as np

df = pd.DataFrame({'Type':list('ABBC'), 'Set':list('ZZXY')})
df['color'] = np.where(df['Set']=='Z', 'green', 'red')


def test(row):
    if row['color'] =='green':
        value='Green'
    elif row['color'] =='red':
        value=row['Type']
    else: 
        value=row['Set']
    return value

def test1(s,t,c):
    if c =='green':
        value='Green'
    elif c =='red':
        value=t
    else: 
        value=s
    return value

df['new_color']=df.apply(test,axis=1)
#df['new_color']=test1(df.Set,df.Type,df.color)
print(df)

   Set Type color  new_color
0   Z    A  green     Green
1   Z    B  green     Green
2   X    B    red         B
3   Y    C    red         C

Any help would be appreciated.

BENY · Accepted Answer · 2018-08-31 02:21:02Z

1

You can do with np.where

df['NC']=np.where(df.color=='green','Green',df.Type)

df
Out[1234]: 
  Set Type  color     NC
0   Z    A  green  Green
1   Z    B  green  Green
2   X    B    red      B
3   Y    C    red      C

answered Aug 31, 2018 at 2:21

BENY

324k22 gold badges176 silver badges250 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

itthrill Over a year ago

that's good suggestion. However, I need to retain the function because the actual conditions are a bit more complicated than the one shown in dummy example above. Its easier for me to wrap all conditions in a function and use it. I am trying to avoid apply which is slow.

Collectives™ on Stack Overflow

Creating column in pandas using vectorization

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related