Getting an error ' No objects to concatenate' while trying a create pandas dataframe?

Question

I am extracting values from API and trying to create an dataframe. My code fails when I dont get any data back from API i.e if API returns zero values.The below code works perfectly when API returns values.

I want create a dataframe structure with null values even if API returns zero values . How can this be done.

My code

data = json.loads(response.text)
    if 'items' in data:
        if len(data['items']) > 0:
            df = pd.json_normalize(data, 'items', 'totalItems') 
            #NaN in column, so failed - replace NaN to empty list
            f = lambda x: x if isinstance(x, list) else []
            df['relatedEntities'] = df['relatedEntities'].apply(f)
            df['raw.identifiers'] = df['raw.identifiers'].apply(f)
            df['raw.relationships'] = df['raw.relationships'].apply(f)
            df1 = pd.concat([pd.DataFrame(x) for x in df.pop('relatedEntities')], keys=df.index).add_prefix('relatedEntities.')
            df2 = pd.concat([pd.DataFrame(x) for x in df.pop('raw.identifiers')], keys=df.index).add_prefix('raw.identifiers.')
            df3 = pd.concat([pd.DataFrame(x) for x in df.pop('raw.relationships')], keys=df.index).add_prefix('raw.relationships.'         
            df4 = df.join(df1.join(df2).join(df3).reset_index(level=1, drop=True))
            dfs.append(df4)

df = pd.concat(dfs, ignore_index=True)

Error That I get when API returns zero values

ValueError                                Traceback (most recent call last)
<ipython-input-31-ee7500e589f7> in <module>
     36             dfs.append(df4)
     37 
---> 38 df = pd.concat(dfs, ignore_index=True)


    330         if len(objs) == 0:
--> 331             raise ValueError("No objects to concatenate")
    332 
    333         if keys is None:

ValueError: No objects to concatenate

I started getting this error with pandas 1.2.x, while it worked with pandas 1.1.x. Which version(s) are you using? — rudolfbyker
– rudolfbyker, Commented Jun 1, 2021 at 11:10

jezrael · Accepted Answer · 2021-05-10 05:51:02Z

3

One idea is test if empty list of DataFrames, if empty list is returned empty DataFrame:

cols = ['start','end','source','title','body','severity','category']
df = pd.concat(dfs, ignore_index=True) if len(dfs) > 0 else pd.DataFrame(columns=cols)

edited May 10, 2021 at 5:51

answered May 10, 2021 at 5:42

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

aeapen Over a year ago

I tried option 1 . But I need to create empty dataframe with fixed column names as wellcolumns=['start','end','source','title','body','severity','category']. What should be done

Collectives™ on Stack Overflow

Getting an error ' No objects to concatenate' while trying a create pandas dataframe?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related