Average two rows in Python while ignoring NaN

Question

I have a two-dimensional array of data. I need to average every two rows, and return the average with an array half of the height. I also need to ignore all NaN values for averaging purposes. For example:

>>> x = numpy.array([[ 1,  nan,  3,  4,  5],
... [ 6,  7,  8,  9, nan],
... [11, 12, 13, 14, nan],
... [16, nan, 18, 19, nan]])

And the function would need to return:

>>> x
array([[3.5,  7,  5.5,  6.5,  5],
[13.5, 12, 15.5, 16.5, nan]])

numpy has masked array, and i'd think you can specify the np.nan being the mask, then apply the averaging operation. — yosukesabai
– yosukesabai, Commented Sep 11, 2012 at 4:26
+1: question is kind of localized, but at least it's clear and concise with expected input and output. — Mu Mind
– Mu Mind, Commented Sep 11, 2012 at 5:00

Mu Mind · Accepted Answer · 2012-09-11 04:56:31Z

3

This should do the trick:

numpy.ma.average(numpy.ma.masked_invalid(x).reshape(-1, 2, x.shape[-1]), 1)

For me it returns

masked_array(data =
 [[3.5 7.0 5.5 6.5 5.0]
 [13.5 12.0 15.5 16.5 --]],
             mask =
 [[False False False False False]
 [False False False False  True]],
       fill_value = 1e+20)

edited Sep 11, 2012 at 4:56

answered Sep 11, 2012 at 4:33

Mu Mind

11.3k4 gold badges43 silver badges70 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Mu Mind Over a year ago

Props to yosukesabai for suggesting masked_array

Josiah Over a year ago

This is exactly what I was looking for, thanks so much. Just started learning Python this summer, still a long way to go!

yosukesabai Over a year ago

hmmm, i am getting array([[ 3.5, nan, 5.5, 6.5, nan], [ 13.5, nan, 15.5, 16.5, nan]]), and not sure what's wrong

Mu Mind Over a year ago

Added my output to the answer. I don't know why you would be getting different output. I'm using python 2.6 with numpy 1.3.0.

Mu Mind Over a year ago

numpy.savetxt doesn't seem to work on masked arrays. Convert back to a normal array with explicit NaN values with x.filled(numpy.NaN), then pass it to numpy.savetxt.

|

Collectives™ on Stack Overflow

Average two rows in Python while ignoring NaN

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related