numpy array filter and replace

Question

I have an array

a = np.array([1,2,3,4,np.nan])

I would like to replace anything which is less than 1.5 by np.nan, i.e. I would like

a = np.array([np.nan,2,3,4,np.nan])

how do I do that?

I did

 a[a<1.5] = np.nan

I got the following run time warning error in IPython (Py3.4) RuntimeWarning: invalid value encountered in less. Is this because my list had np.nan? Is there anything I can do to prevent this?

Also is there a way doing this inline without assigning? Instead of doing

a[a<1.5]=np.nan 
return a

I can just do

 return a...

where that .... is something that need filling in.

@BradSolomon i meant, I am writing this in the middle of a program. is there an inplace replacement function? — Lost1
– Lost1, Commented Sep 6, 2017 at 21:02

wim · Accepted Answer · 2017-09-06 21:34:30Z

4

Is this [RuntimeWarning] because my list had np.nan?

Yes.

Is there anything I can do to prevent this?

In your case, this warning can be safely ignored. So that you don't accidentally suppress unrelated warnings, please don't put anything else inside the context manager besides the one line shown.

>>> import numpy as np
>>> a = np.array([1,2,3,4,np.nan])
>>> with np.errstate(invalid='ignore'):
...     a[a<1.5] = np.nan
...     
>>> a
array([ nan,   2.,   3.,   4.,  nan])

This operates in-place, a copy is not created here. To return a copy, with the original a unmodified, prefer the masked array approach.

edited Sep 6, 2017 at 21:34

answered Sep 6, 2017 at 21:04

wim

368k114 gold badges682 silver badges819 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

MSeifert Over a year ago

I don't think it's actually about the in-place operation (well, a[a<1.5] = np.nan is also in-place) but about avoiding assignment so that it can be done in one-line.

Brad Solomon · Accepted Answer · 2017-09-06 21:17:42Z

1

Another option that gets you to your return statement as desired:

mask = ~np.isnan(a)
mask[mask] &= a[mask] < 1.5
return np.where(mask, np.nan, a)

Example:

def ma_lessthan(arr, num):
    mask = ~np.isnan(arr)
    mask[mask] &= arr[mask] < num
    return np.where(mask, np.nan, arr)

print(ma_lessthan(a, 1.5))
[ nan   2.   3.   4.  nan]

mask credit to: @Jaime.

answered Sep 6, 2017 at 21:17

Brad Solomon

41.2k39 gold badges167 silver badges261 bronze badges

2 Comments

wim Over a year ago

It should be mentioned: this creates a copy.

wim Over a year ago

It is avoidable, and I showed how. Using a masked array approach is good too, mind you, I just thought this was worth mentioning because we often care about details of performance and memory usage when using numpy.

Collectives™ on Stack Overflow

numpy array filter and replace

2 Answers 2

1 Comment

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related