Add multiple columns at once to multiindex Pandas dataframe

Question

I have a dataframe multiindex pandas dataframe df

First    Foo     Bar
Second   Begin   Begin
1        5       1
2        4       4
3        6       6

And I want to add two columns of the same name

First    Foo             Bar
Second   Begin   End     Begin   End
1        5       1       1       2       
2        4       5       4       4       
3        6       7       6       7

From this source (new):

First    Foo    Bar
1        1      2
2        5      4
3        7      7

I tried things like df[:] = new[:] but this returned only NaN

An alternative would be to use something like a for-loop but that's not the Pandas approach. Searching the web did not give me any insights as to solving this problem.

How can I add new columns with the same name and shape to every first level of a multiindex Pandas dataframe?

Edit:

This approach df[('Foo', 'End')] = new['Foo'] df[('Bar', 'End')] = new['Bar'] is not an option because in my actual problem there is not two columns to be added, but hundreds of columns.

why are you using a multi-index? There may be an easier way to achieve what you want — anon01
– anon01, Commented Aug 24, 2020 at 20:04
Does this answer your question? Selecting columns from pandas MultiIndex — Trenton McKinney
– Trenton McKinney, Commented Aug 24, 2020 at 20:25
@TrentonMcKinney no my question is not about selecting but about inserting columns — Duco
– Duco, Commented Aug 24, 2020 at 21:00

Trenton McKinney · Accepted Answer · 2020-08-24 21:10:22Z

1

Multi-column names are passed as Tuples, like df[('Foo', 'End')].

import pamadas as pd

# test data
col = pd.MultiIndex.from_arrays([['Foo', 'Bar'], ['Begin', 'Begin']], names=['First', 'Second'])
df = pd.DataFrame([[5, 1], [4, 4], [6, 6]], columns=col)
new = pd.DataFrame({'Foo': [1, 5, 7], 'Bar': [2, 4, 7]})

# write new columns
df[('Foo', 'End')] = new['Foo']
df[('Bar', 'End')] = new['Bar']

# display(df)
First    Foo   Bar Foo Bar
Second Begin Begin End End
0          5     1   1   2
1          4     4   5   4
2          6     6   7   7

For many columns

col, column name in new, must correspond to the top level column name in df.

for col in new.columns:
    df[(col, 'new col name')] = new[col]

edited Aug 24, 2020 at 21:10

answered Aug 24, 2020 at 20:32

Trenton McKinney

63.2k41 gold badges170 silver badges213 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Add multiple columns at once to multiindex Pandas dataframe

1 Answer 1

For many columns

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

For many columns

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related