Pandas add additional empty columns during DF creation

Question

I'm getting:

AssertionError: 14 columns passed, passed data had 12 
columns.

Error is self-explanatory, I'm creating a DataFrame from a list of lists, each list in fact contains 12 fields. How can I force pandas.DataFrame to add extra columns with NaN in one operation. This is how Im creating the df.

pandas.DataFrame(results, columns=FIELDS)

Is there a way to simplify this in DataFrame creation, without having to do this:

pandas.DataFrame(results, columns=FIELDS_12)
df["ExtraField13"] = np.nan
df["ExtraField14"] = np.nan

Will the extra columns always be at the end? Otherwise what you want is ambiguous. — SethMMorton
– SethMMorton, Commented Feb 14, 2017 at 23:23
Some cases, yes, others not, I would ask if there is a way to force always 14 columns regardless the size of the list in the list of lists. Another workaround would be normalize the list of lists, but wondering if this is possible with Pandas. — gogasca
– gogasca, Commented Feb 14, 2017 at 23:29
If it's not at the end, how do you know where to insert the empty column? — SethMMorton
– SethMMorton, Commented Feb 14, 2017 at 23:35
Consider assign: pandas.DataFrame(results, columns=FIELDS_12).assign(ExtraField13=np.nan, ExtraField14=np.nan) — Parfait
– Parfait, Commented Feb 15, 2017 at 0:41

boot-scootin · Accepted Answer · 2017-02-14 23:37:19Z

1

Well, you don't really have to hard-code things like you have above with individual column assignments, line-by-line. This does a similar thing but uses a loop instead to pad columns:

>>> import pandas as pd
>>> df = pd.DataFrame({'A': [1,2,3], 'B': [4,5,6]})
>>> df
   A  B
0  1  4
1  2  5
2  3  6
>>> def pad_cols(df, n_cols):
...     while len(df.columns) < n_cols:
...         df['padded' + str(len(df.columns)+1)] = None
...     return df
... 
>>> pad_cols(df, 4)
   A  B padded3 padded4
0  1  4    None    None
1  2  5    None    None
2  3  6    None    None

answered Feb 14, 2017 at 23:37

boot-scootin

12.6k10 gold badges73 silver badges118 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Pandas add additional empty columns during DF creation

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related