How to group by hierarchical data in pandas/sql?

Question

I have a problem on hierarchy.i have data like this.

   id       performance_rating     parent_id     level
   111           8                   null         0 
   122           3                   null         0
   123           9                   null         0
   254           5                   111          1
   265           8                   111          1
   298           7                   122          1
   220           6                   123          1
   305           5                   298          2
   395           8                   220          2
   ...           ...                 ...         ...
   654           4                   562          5

the id is person unique identity. performance_rating is his rating out of 10 parent_id is the id of the person who is working above the corresponding id.

I need to find out the average rating of an individual tree(111,122,123).

what I tried is separate data frame according to levels. Then merge it and groupby. But it is quite long.

What do you mean by roll up ?

yatu
– yatu

2019-05-02 09:35:47 +00:00
Commented May 2, 2019 at 9:35 — yatu
– yatu, Commented May 2, 2019 at 9:35
the average for all the child under that id

Aamir Siddiqui
– Aamir Siddiqui

2019-05-02 10:16:27 +00:00
Commented May 2, 2019 at 10:16 — Aamir Siddiqui
– Aamir Siddiqui, Commented May 2, 2019 at 10:16

Jim Eisenberg · Accepted Answer · 2019-05-02 15:48:43Z

2

there will be a few different ways to do this - here's an ugly solution.

We use a while and for loop over a function to "back-level" each column of the dataframe: This requires that we first set 'id' as index and sort by 'level', descending. It also requires no duplicate IDs. Here goes:

df = df.set_index('id')
df = df.sort_values(by='level', ascending=False)

for i in df.index:
    while df.loc[i, 'level'] > 1:
        old_pid = df.loc[i, 'parent_id']
        df.loc[i, 'parent_id'] = df.loc[old_pid, 'parent_id']
        old_level = df.loc[i,'level']
        df.loc[i, 'level'] = old_level - 1

This way, no matter how many levels there are, we are left with everything at level 1 of hierarchy and can then do:

grouped = df.groupby('parent_id').mean()

(or whatever variation of that you need) I hope that helps!

answered May 2, 2019 at 15:48

Jim Eisenberg

1,5001 gold badge11 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to group by hierarchical data in pandas/sql?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related