matplotlib-users Mailing List for matplotlib

On 02/04/2011 02:03 PM, Christoph Gohlke wrote:
[...]
>
> How about these changes to color.py (attached). This avoids copies, uses
> in-place operations, and calculates single precision when normalizing
> small integer and float32 arrays. Similar could be done for LogNorm. Do
> masked arrays support in-place operations?
>
> Christoph

Christoph,

Thank you.

Done (with slight modifications) in 8946 (trunk).

I was surprised by the speedup in normalizing large arrays when using 
float32 versus float64.  A factor of 10 on my machine with (1000,1000), 
timed with ipython %timeit.  Because of the way %timeit does multiple 
tests, I suspect it may exaggerate cache effects.

Eric

On Fri, Feb 4, 2011 at 6:41 PM, Benjamin Root <ben...@ou...> wrote:

> On Friday, February 4, 2011, Massimo Di Stefano
> <mas...@gm...> wrote:
> > Hello All,
> >
> >
> > i'm plotting a 3d colored surface using a 4D array that comes from a .mat
> file
> > using this code :
> >
> >
> > import scipy.io as sio
> > import pylab as p
> > import mpl_toolkits.mplot3d.axes3d as p3
> >
> > def loadmatfile(matfile):
> >    matdata = sio.loadmat(matfile)
> >    return matdata
> >
> >
> > def plot3dcolor(matfile):
> >    data = loadmatfile(matfile)
> >    x = data['X_depth']
> >    y = data['Y_depth']
> >    z = -data['Z_depth']
> >    c = data['Z_compl']
> >    fig=p.figure()
> >    ax = p3.Axes3D(fig)
> >    cmap = p.get_cmap('jet')
> >    norm = p.Normalize(c.min(), c.max())
> >    colors = cmap(norm(c))
> >    ax.plot_surface(x, y, z, rstride=10, cstride=10, facecolors=colors)
> >    ax.set_xlabel('X')
> >    ax.set_ylabel('Y')
> >    ax.set_zlabel('Z')
> >    print x,y
> >    p.show()
> >
> >
> > matfile = '/Users/epy/Desktop/complexity_depth_grid1.mat'
> > plot3dcolor(matfile)
> >
> >
> >
> > the results is nice :
> >
> > http://img831.imageshack.us/f/schermata20110204a14542.png/
> >
> >
> > but as you can see, the mouse cursor shows me the x,y values (they are
> longitude and latitude)
> > but on the axis i have them starting from 0 ...
> >
> > how can i change the axis to display the lon-lat coordinates ?
> >
> >
> > thanks a lot for any help!
> >
> > Massimo.
> >
> >
>
> I suspect what is happening is that the axes label numbers are right,
> but is not showing the offset information.  The display of offset data
> in a 3d plot is a new feature that exists only in the development
> branch.
>
> To confirm this, could you send me your may file (if it is small) so
> that I can try out your script?
>
> Ben Root
>

Ok, so I can confirm that there are offset labels displaying when using the
development version of mpl, which means that for some reason, the
auto-ticker is deciding to use the offset display instead of just the
regular display.  Usually, this behavior doesn't trigger until 1e4 or
larger, I believe?

I will have to take a peek at what ticker is being used by default and
seeing if it is different from the regular plots.  In the meantime, if you
need something to work right away, and you feel ambitious, you can fiddle
with  the tickers through the ax.w_xaxis and ax.w_yaxis objects (not the
ax.xaxis and ax.yaxis objects):

http://matplotlib.sourceforge.net/api/ticker_api.html?highlight=ticker#matplotlib-ticker

Ben Root

Jorge Scandaliaris <jorgesmbox-ml@...> writes:

< snip >
> 
> The symptoms are that
> the interaction (keys and mouse clicks) works OK as long as I don't resize the
> window. Once I do this, the interaction is gone. What happens when the window
> is
> resized, that could be related to what I experience.
> 
> OK, lets see if I can come up with a simple code reproducing this...
> 

I couldn't come up with a simple example showing the problem. My attempts so far
ended in working examples. I did find a change in my code that triggers the
problem: My code uses a modified version of the lasso_demo example, contains two
axes, where I draw an image with imshow and a collection of circles with
scatter. Originally scatter was called with a fixed size for all points, and I
just changed it so each points gets a different size. 

Just don't ask me why this seemingly innocent change affects (breaks) the event
handling. I hope somebody can give me some hints. I'll keep checking the code to
see if I find anything suspicious.

jorges

Dear all,

I want to generate a pdf with some text and graphics. Since I use
matplotlib to make plot through the graphical user interface, I want to
embed these graphs directly into a cairo (or pangocairo) instance.

But I cannot manage.

I send a small example. If you can help me....

import cairo
import pango
import pangocairo
import matplotlib
matplotlib.use('Cairo')
from matplotlib import pyplot

surface = cairo.PDFSurface('test_report.pdf', 595, 795)
context = pangocairo.CairoContext(cairo.Context(surface))

context.translate(0, 50)
layout = context.create_layout()
layout.set_text('Hello, this is a long text to see if the lines are
automatically broken.')
layout.set_font_description(pango.FontDescription("Sans Bold 12"))
layout.set_width(pango.SCALE * 570)
# Put the text in the context.
context.set_source_rgb(0, 0, 0)
context.show_layout(layout)

figure = pyplot.figure()
axe = figure.add_subplot(111)
toto = axe.plot(range(20),[i**2 for i in range(20)])
context.set_source(figure.canvas)

Thanks a lot.

On Friday, February 4, 2011, Massimo Di Stefano
<mas...@gm...> wrote:
> Hello All,
>
>
> i'm plotting a 3d colored surface using a 4D array that comes from a .mat file
> using this code :
>
>
> import scipy.io as sio
> import pylab as p
> import mpl_toolkits.mplot3d.axes3d as p3
>
> def loadmatfile(matfile):
>    matdata = sio.loadmat(matfile)
>    return matdata
>
>
> def plot3dcolor(matfile):
>    data = loadmatfile(matfile)
>    x = data['X_depth']
>    y = data['Y_depth']
>    z = -data['Z_depth']
>    c = data['Z_compl']
>    fig=p.figure()
>    ax = p3.Axes3D(fig)
>    cmap = p.get_cmap('jet')
>    norm = p.Normalize(c.min(), c.max())
>    colors = cmap(norm(c))
>    ax.plot_surface(x, y, z, rstride=10, cstride=10, facecolors=colors)
>    ax.set_xlabel('X')
>    ax.set_ylabel('Y')
>    ax.set_zlabel('Z')
>    print x,y
>    p.show()
>
>
> matfile = '/Users/epy/Desktop/complexity_depth_grid1.mat'
> plot3dcolor(matfile)
>
>
>
> the results is nice :
>
> http://img831.imageshack.us/f/schermata20110204a14542.png/
>
>
> but as you can see, the mouse cursor shows me the x,y values (they are longitude and latitude)
> but on the axis i have them starting from 0 ...
>
> how can i change the axis to display the lon-lat coordinates ?
>
>
> thanks a lot for any help!
>
> Massimo.
>
>

I suspect what is happening is that the axes label numbers are right,
but is not showing the offset information.  The display of offset data
in a 3d plot is a new feature that exists only in the development
branch.

To confirm this, could you send me your may file (if it is small) so
that I can try out your script?

Ben Root

Hi,
I have some code that's been working for a long time, but it's behaving strange
with current (svn) matplotlib. It's too large to post here, so I will try to
come up with a minimal version reproducing the problem. Meanwhile, I wanted to
ask if anyone would have a clue of what could be going on. The symptoms are that
the interaction (keys and mouse clicks) works OK as long as I don't resize the
window. Once I do this, the interaction is gone. What happens when the window is
resized, that could be related to what I experience.

OK, lets see if I can come up with a simple code reproducing this...

jorges

Hello All,


i'm plotting a 3d colored surface using a 4D array that comes from a .mat file
using this code :


import scipy.io as sio
import pylab as p
import mpl_toolkits.mplot3d.axes3d as p3

def loadmatfile(matfile):
   matdata = sio.loadmat(matfile)
   return matdata


def plot3dcolor(matfile):
   data = loadmatfile(matfile)
   x = data['X_depth']
   y = data['Y_depth']
   z = -data['Z_depth']
   c = data['Z_compl']
   fig=p.figure()
   ax = p3.Axes3D(fig)
   cmap = p.get_cmap('jet')
   norm = p.Normalize(c.min(), c.max())
   colors = cmap(norm(c))
   ax.plot_surface(x, y, z, rstride=10, cstride=10, facecolors=colors)
   ax.set_xlabel('X')
   ax.set_ylabel('Y')
   ax.set_zlabel('Z')
   print x,y
   p.show()


matfile = '/Users/epy/Desktop/complexity_depth_grid1.mat'
plot3dcolor(matfile)



the results is nice :

http://img831.imageshack.us/f/schermata20110204a14542.png/


but as you can see, the mouse cursor shows me the x,y values (they are longitude and latitude)
but on the axis i have them starting from 0 ...

how can i change the axis to display the lon-lat coordinates ?


thanks a lot for any help!

Massimo.

On 2/4/2011 3:29 PM, Eric Firing wrote:
> On 02/04/2011 12:33 PM, Christoph Gohlke wrote:
>>
>>
>> On 2/4/2011 2:14 PM, Eric Firing wrote:
>>> On 02/04/2011 11:33 AM, Eric Firing wrote:
>>>> On 02/04/2011 10:28 AM, Christoph Gohlke wrote:
>>>>>
>>>>>
>>>>> On 2/4/2011 11:54 AM, Eric Firing wrote:
>>>>>> On 02/03/2011 05:35 PM, Christoph Gohlke wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 2/3/2011 6:50 PM, Eric Firing wrote:
>>>>>>>> On 02/03/2011 03:04 PM, Benjamin Root wrote:
>>>>>>>>
>>>>>>>>> Also, not to sound too annoying, but has anyone considered the idea of
>>>>>>>>> using compressed arrays for holding those rgba values?
>>>>>>>>
>>>>>>>> I don't see how that really helps; as far as I know, a full rgba array
>>>>>>>> has to be passed into agg. What *does* help is using uint8 from start
>>>>>>>> to finish. It might also be possible to use some smart downsampling
>>>>>>>> before generating the rgba array, but the uint8 route seems to me the
>>>>>>>> first thing to attack.
>>>>>>>>
>>>>>>>> Eric
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ben Root
>>>>>>>>
>>>>>>>
>>>>>>> Please review the attached patch. It avoids generating and storing
>>>>>>> float64 rgba arrays and uses uint8 rgba instead. That's a huge memory
>>>>>>> saving and also faster. I can't see any side effects as
>>>>>>> _image.fromarray() converts the float64 input to uint8 anyway.
>>>>>>
>>>>>> Christoph,
>>>>>>
>>>>>> Thank you!  I haven't found anything wrong with that delightfully simple
>>>>>> patch, so I have committed it to the trunk.  Back in 2007 I added the
>>>>>> ability of colormapping to generate uint8 directly, precisely to enable
>>>>>> this sort of optimization.  Why it was not already being used in imshow,
>>>>>> I don't know--maybe I was going to do it, got sidetracked, and never
>>>>>> finished.
>>>>>>
>>>>>> I suspect it won't be as simple as for the plain image, but there may be
>>>>>> opportunities for optimizing with uint8 in other image-like operations.
>>>>>>
>>>>>>>
>>>>>>> So far other attempts to optimize memory usage were thwarted by
>>>>>>> matplotlib's internal use of masked arrays. As mentioned before, users
>>>>>>> can provide their own normalized rgba arrays to avoid all this processing.
>>>>>>>
>>>>>>
>>>>>> Did you see other potential low-hanging fruit that might be harvested
>>>>>> with some changes to the code associated with masked arrays?
>>>>>>
>>>>>> Eric
>>>>>>
>>>>>
>>>>> The norm function currently converts the data to double precision
>>>>> floating point and also creates temporary arrays that can be avoided.
>>>>> For float32 and low precision integer images this seems overkill and one
>>>>> could use float32. It might be possible to replace the norm function
>>>>> with numpy.digitize if that works with masked arrays. Last, the
>>>>> _image.frombyte function does a copy of 'strided arrays' (only relevant
>>>>> when zooming/panning large images). I try to provide a patch for each.
>>>>
>>>> masked arrays can be filled to create an ndarray before passing to
>>>> digitize; whether that will be faster, remains to be seen.  I've never
>>>> used digitize.
>>>
>>> I didn't say that ("can be filled...") right.  I think one would need to
>>> use the mask to put in the i_bad index where appropriate.  np.ma does
>>> not have a digitize function.  I suspect it won't help much if at all in
>>> Normalize, but it would be a natural for use in BoundaryNorm.
>>>
>>> It looks easy to allow Normalize.__call__ to use float32 if that is what
>>> it receives.
>>>
>>> I don't see any unnecessary temporary array creation apart from the
>>> conversion to float64, except for the generation of a masked array
>>> regardless of input.  I don't think this costs much; if it gets an
>>> ndarray it does not copy it, and it does not generate a full mask array.
>>>      Still, the function probably could be sped up a bit by handling
>>> masking more explicitly instead of letting ma do the work.
>>>
>>
>> In class Normalize:
>>        result = 0.0 * val
>> and
>>        result = (val-vmin) / (vmax-vmin)
>>
>>> Eric
>>>
>>>>
>>>> Regarding frombyte, I suspect you can't avoid the copy; the data
>>>> structure being passed to agg is just a string of bytes, as far as I can
>>>> see, so everything is based on having a simple contiguous array.
>>>>
>>
>> The PyArray_ContiguousFromObject call will return a copy if the input
>> array is not already contiguous.
>
> Exactly.  I thought you were suggesting that this was not needed, but
> maybe I misunderstood.
>
> Eric

In fact I am suggesting that this is not needed. The copy-to-agg routine 
could be made 'stride aware' and use PyArray_FromObject. Not a very low 
hanging fruit but it seems fromarray() does this already. I'm not sure 
it's worth.

How about these changes to color.py (attached). This avoids copies, uses 
in-place operations, and calculates single precision when normalizing 
small integer and float32 arrays. Similar could be done for LogNorm. Do 
masked arrays support in-place operations?

Christoph