[GRASS-dev] numeric-numpy-scipy for graphs?

GRASS GIS GRASS GIS developer
1

We need to create a graph/plot for at least 2 functions in the wxgrass GUI—profile plotting and histogramming. I did the TclTk profile module with just standard graphic calls. However, wxPython has a very nice plot control:PyPlot. It has lots of nice options, including ways to save the plot easily.

BUT, it requires numeric or numarray — which have been superceded by numpy or scipy. These are as available the main python package and provide a rich scientific computing library for Python and wxPython. It looks like these are available for all platforms either as binary or source files. However, it would be another dependency.

So what do you think? Should I install numpy (or scipy) and try the PyPlot or just create these modules with simple graphics?

Michael

Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

2

Based on your previous experience with TclTk:

using pyplot, do you forecast less pain/less time for coding the same and
better quality of GUI interface because of mentioned advantages?

if yes, +1 to the added dependency.

yann

On Sunday 22 April 2007 13:45, Michael Barton wrote:

We need to create a graph/plot for at least 2 functions in the wxgrass
GUI‹profile plotting and histogramming. I did the TclTk profile module with
just standard graphic calls. However, wxPython has a very nice plot
control:PyPlot. It has lots of nice options, including ways to save the
plot easily.

BUT, it requires numeric or numarray ‹ which have been superceded by numpy
or scipy. These are as available the main python package and provide a rich
scientific computing library for Python and wxPython. It looks like these
are available for all platforms either as binary or source files. However,
it would be another dependency.

So what do you think? Should I install numpy (or scipy) and try the PyPlot
or just create these modules with simple graphics?

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

--
Yann Chemin
Sainte-Anne d'Auray, France

3

Hi,

have a look at matplotlib http://matplotlib.sourceforge.net/ . It depends on numeric.

Wolfgang

Michael Barton wrote:

We need to create a graph/plot for at least 2 functions in the wxgrass GUI—profile plotting and histogramming. I did the TclTk profile module with just standard graphic calls. However, wxPython has a very nice plot control:PyPlot. It has lots of nice options, including ways to save the plot easily.

BUT, it requires numeric or numarray — which have been superceded by numpy or scipy. These are as available the main python package and provide a rich scientific computing library for Python and wxPython. It looks like these are available for all platforms either as binary or source files. However, it would be another dependency.

So what do you think? Should I install numpy (or scipy) and try the PyPlot or just create these modules with simple graphics?

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

------------------------------------------------------------------------

_______________________________________________
grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

4

Michael Barton wrote:

We need to create a graph/plot for at least 2 functions in the wxgrass
GUI‹profile plotting and histogramming. I did the TclTk profile module
with just standard graphic calls. However, wxPython has a very nice
plot control:PyPlot. It has lots of nice options, including ways to
save the plot easily.

maybe you are looking for something more advanced, but can it be done
with d.graph and/or d.linegraph? I've never actually used d.linegraph,
but would be willing to help bring it up to speed if it is useful but
for a few small rendering issues.

Hamish

5

I wouldn't mind. As a bit of perspective, the GDAL Python interface uses Numeric for it's old bindings and Numpy for its new bindings, so either will be something many GRASS users will probably have.

On Apr 22, 2007, at 1:45 AM, Michael Barton wrote:

We need to create a graph/plot for at least 2 functions in the wxgrass GUI—profile plotting and histogramming. I did the TclTk profile module with just standard graphic calls. However, wxPython has a very nice plot control:PyPlot. It has lots of nice options, including ways to save the plot easily.

BUT, it requires numeric or numarray — which have been superceded by numpy or scipy. These are as available the main python package and provide a rich scientific computing library for Python and wxPython. It looks like these are available for all platforms either as binary or source files. However, it would be another dependency.

So what do you think? Should I install numpy (or scipy) and try the PyPlot or just create these modules with simple graphics?

-----
William Kyngesburye <kyngchaos*at*kyngchaos*dot*com>
http://www.kyngchaos.com/

"Oh, look, I seem to have fallen down a deep, dark hole. Now what does that remind me of? Ah, yes - life."

- Marvin

6

On 4/22/07 12:03 AM, "Yann" <yann.chemin@gmail.com> wrote:

Based on your previous experience with TclTk:

using pyplot, do you forecast less pain/less time for coding the same and
better quality of GUI interface because of mentioned advantages?

I don't know yet. I haven't installed scipy and so haven't seen what pyplot
does. I wanted to make sure that it was worth the effort on my part to test
it out before doing so.

There have been enough positive replies that I'll go ahead and work on
installing scipy (the successor to numeric/numarray->numpy) to try this out.
If this becomes a general GRASS dependency,

I could see some very positive results in having a sophisticated scientific
computation toolkit available for scripting.

Michael

if yes, +1 to the added dependency.

yann

__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

7

Really impressive!

Michael

On 4/22/07 1:25 AM, "wolle" <wollez@gmx.net> wrote:

Hi,

have a look at matplotlib http://matplotlib.sourceforge.net/ . It
depends on numeric.

Wolfgang

Michael Barton wrote:

We need to create a graph/plot for at least 2 functions in the wxgrass
GUI‹profile plotting and histogramming. I did the TclTk profile module
with just standard graphic calls. However, wxPython has a very nice plot
control:PyPlot. It has lots of nice options, including ways to save the
plot easily.

BUT, it requires numeric or numarray ‹ which have been superceded by
numpy or scipy. These are as available the main python package and
provide a rich scientific computing library for Python and wxPython. It
looks like these are available for all platforms either as binary or
source files. However, it would be another dependency.

So what do you think? Should I install numpy (or scipy) and try the
PyPlot or just create these modules with simple graphics?

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

------------------------------------------------------------------------

_______________________________________________
grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

8

Thanks Hamish,

In fact, we can do considerably more now with the wxPython (and TclTk)
drawing commands to than is available in d.graph or d.linegraph, even with
your nice enhancements.

In the case of histogramming and profiling, it seems a better display to
have them come up in a separate window (like is now the case with the TclTk
profiling module) than have them rendered like a map in the main display
(like we do now for d.histogram in the current TclTk GUI). Rendering maps is
complex and it doesn't seem efficient to have to go through the same complex
procedures to draw a histogram or profile that doesn't need to be drawn
georeferenced.

For this reason, it seems better to do these kinds of analyses with the gui
tools, using underlying GRASS modules like r.profile and r.stats.

Michael

On 4/22/07 2:54 AM, "Hamish" <hamish_nospam@yahoo.com> wrote:

Michael Barton wrote:

We need to create a graph/plot for at least 2 functions in the wxgrass
GUI‹profile plotting and histogramming. I did the TclTk profile module
with just standard graphic calls. However, wxPython has a very nice
plot control:PyPlot. It has lots of nice options, including ways to
save the plot easily.

maybe you are looking for something more advanced, but can it be done
with d.graph and/or d.linegraph? I've never actually used d.linegraph,
but would be willing to help bring it up to speed if it is useful but
for a few small rendering issues.

Hamish

__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

9

Michael Barton wrote:

In fact, we can do considerably more now with the wxPython (and TclTk)
drawing commands to than is available in d.graph or d.linegraph, even with
your nice enhancements.

In the case of histogramming and profiling, it seems a better display to
have them come up in a separate window (like is now the case with the TclTk
profiling module) than have them rendered like a map in the main display
(like we do now for d.histogram in the current TclTk GUI). Rendering maps is
complex and it doesn't seem efficient to have to go through the same complex
procedures to draw a histogram or profile that doesn't need to be drawn
georeferenced.

For this reason, it seems better to do these kinds of analyses with the gui
tools, using underlying GRASS modules like r.profile and r.stats.

Even if you decide not to use the existing graphics API, I would still
suggest that any functionality which could reasonably be used
non-interactively can be.

--
Glynn Clements <glynn@gclements.plus.com>

10

I agree that these functions should be maintained and even enhanced,
especially for scripting.

Michael

On 4/22/07 11:41 AM, "Glynn Clements" <glynn@gclements.plus.com> wrote:

Michael Barton wrote:

In fact, we can do considerably more now with the wxPython (and TclTk)
drawing commands to than is available in d.graph or d.linegraph, even with
your nice enhancements.

In the case of histogramming and profiling, it seems a better display to
have them come up in a separate window (like is now the case with the TclTk
profiling module) than have them rendered like a map in the main display
(like we do now for d.histogram in the current TclTk GUI). Rendering maps is
complex and it doesn't seem efficient to have to go through the same complex
procedures to draw a histogram or profile that doesn't need to be drawn
georeferenced.

For this reason, it seems better to do these kinds of analyses with the gui
tools, using underlying GRASS modules like r.profile and r.stats.

Even if you decide not to use the existing graphics API, I would still
suggest that any functionality which could reasonably be used
non-interactively can be.

__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

11

http://matplotlib.sourceforge.net/screenshots/plotmap_large.png

Really nice !

On Sunday 22 April 2007 15:25, wolle wrote:

Hi,

have a look at matplotlib http://matplotlib.sourceforge.net/ . It
depends on numeric.

Wolfgang

Michael Barton wrote:
> We need to create a graph/plot for at least 2 functions in the wxgrass
> GUI—profile plotting and histogramming. I did the TclTk profile module
> with just standard graphic calls. However, wxPython has a very nice plot
> control:PyPlot. It has lots of nice options, including ways to save the
> plot easily.
>
> BUT, it requires numeric or numarray — which have been superceded by
> numpy or scipy. These are as available the main python package and
> provide a rich scientific computing library for Python and wxPython. It
> looks like these are available for all platforms either as binary or
> source files. However, it would be another dependency.
>
> So what do you think? Should I install numpy (or scipy) and try the
> PyPlot or just create these modules with simple graphics?
>
> Michael
> __________________________________________
> Michael Barton, Professor of Anthropology
> School of Human Evolution & Social Change
> Center for Social Dynamics & Complexity
> Arizona State University
>
> phone: 480-965-6213
> fax: 480-965-7671
> www: http://www.public.asu.edu/~cmbarton
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> grass-dev mailing list
> grass-dev@grass.itc.it
> http://grass.itc.it/mailman/listinfo/grass-dev

_______________________________________________
grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

--
Yann Chemin
Sainte-Anne d'Auray, France

12

On Sun, 22 Apr 2007 11:44:34 -0700
Michael Barton wrote:

I agree that these functions should be maintained and even enhanced,
especially for scripting.

As discussed a long time ago on the list, there is no reason why a
wrapper function couldn't be created in python to make calls to the GUI
from the command line, to allow for scripted control of the GUI using
sockets (not Unix specific ones). One socket could be listened to by
the GUI and the command line wrapper would simply pass calls to that
socket thus allowing CLI access. I had planned to develop this myself,
but being up to my ass in alligators (figuratively speaking) I've not
had the time.

If there is good functionality available through wxGRASS, why maintain
another version of a similar function. It could be argued that some
users would prefer to avoid the overhead of a GUI, but for them, there
are older versions of GRASS. AFAIC duplication is a bad thing. In
the similar vein to Python, there should be one obvious way to do
things. GRASS is already so big, we don't need more duplication.

So I would vote for inclusion of scipy for graphing and planning the
obsolescence of the older means.

T
--
Trevor Wiens
twiens@interbaun.com

The significant problems that we face cannot be solved at the same
level of thinking we were at when we created them.
(Albert Einstein)

13

Trevor Wiens wrote:

> I agree that these functions should be maintained and even enhanced,
> especially for scripting.

As discussed a long time ago on the list, there is no reason why a
wrapper function couldn't be created in python to make calls to the GUI
from the command line, to allow for scripted control of the GUI using
sockets (not Unix specific ones). One socket could be listened to by
the GUI and the command line wrapper would simply pass calls to that
socket thus allowing CLI access. I had planned to develop this myself,
but being up to my ass in alligators (figuratively speaking) I've not
had the time.

Being able to remote-control the GUI is a separate issue.

Functionality (including creating graphics) should be available in an
environment where you cannot create a window (no X server, no X libs).

--
Glynn Clements <glynn@gclements.plus.com>

14

On Sun, 22 Apr 2007 21:31:49 +0100
Glynn Clements wrote:

Trevor Wiens wrote:

> > I agree that these functions should be maintained and even enhanced,
> > especially for scripting.
>
> As discussed a long time ago on the list, there is no reason why a
> wrapper function couldn't be created in python to make calls to the GUI
> from the command line, to allow for scripted control of the GUI using
> sockets (not Unix specific ones). One socket could be listened to by
> the GUI and the command line wrapper would simply pass calls to that
> socket thus allowing CLI access. I had planned to develop this myself,
> but being up to my ass in alligators (figuratively speaking) I've not
> had the time.

Being able to remote-control the GUI is a separate issue.

Functionality (including creating graphics) should be available in an
environment where you cannot create a window (no X server, no X libs).

In that case why not allow users to export their data to R and use that
or pipe to another graph producing package like ploticus (which I've
used extensively and found to be excellent as it includes CMYK
eps output options, and much better than any other of the products
suggested in this discussion so far IMHO). Neither of these solutions
would require as much work as maintaining duplicate modules which
produce lower quality results. Yes, it is an additional dependency, but
a general management rule states one should manage for the majority and
treat special cases individually. I would expect that the majority of
users use GRASS in a windowing environment, so we are talking about a
very small portion of the user base, no?

Another option would be to use a single solution that can be used in
all cases. If this were to be the case I would vote for serious
consideration being given to ploticus. FYI, Steve Grubb also has a
small postscript library which supports CMYK, which we might want to
look at using to enhance the ps output options with the d.* modules.

I'm not trying to make things complicated, but I'm trying to make sure
we don't waste effort recreating things that already exist and good.

One of my current projects is producing a book on birds for which I
produced almost 600 graphs as CMYK eps files. Matplotlib couldn't do it.
GnuPlot sucks IMHO. PyX was as user friendly as a poke in the eye,
although the list people were very nice. I was able to do what I needed
with ploticus within half a day of looking at it and could continue to
customize it as needed. It may appear a bit old fashioned compared to
some of the Python plotting products, but it is easily understood,
provides fine control when needed, and produces output in a variety of
formats. No GUI required. If we wrote wrapper to use it, the output
could be displayed in a separate window or written to file.

T
--
Trevor Wiens
twiens@interbaun.com

The significant problems that we face cannot be solved at the same
level of thinking we were at when we created them.
(Albert Einstein)

15

Michael Barton wrote:

We need to create a graph/plot for at least 2 functions in the wxgrass
GUI‹profile plotting and histogramming. I did the TclTk profile module with
just standard graphic calls. However, wxPython has a very nice plot
control:PyPlot. It has lots of nice options, including ways to save the plot
easily.

Can you provide a URL; the first thing that comes up for "PyPlot" on
Google uses Python/Tk for the interface and gnuplot for the output:

http://www.wadsworth.org/spider_doc/spider/docs/spire/guitools/pyplot.html

The remainder of the first couple of pages are posts to mailing lists.

BUT, it requires numeric or numarray ‹ which have been superceded by numpy
or scipy. These are as available the main python package and provide a rich
scientific computing library for Python and wxPython. It looks like these
are available for all platforms either as binary or source files. However,
it would be another dependency.

AFAICT, both NumPy and SciPy deal are primarily numerical packages; I
can't see anything about graphics.

--
Glynn Clements <glynn@gclements.plus.com>

16

"wolle" wrote:
> have a look at matplotlib http://matplotlib.sourceforge.net/ . It
> depends on numeric.

Michael Barton wrote:

Really impressive!

oh, and by the way, IMO translucency is the neat way to go for bubble
plots, not sorting by size before rendering and drawing the little ones
on top (can lead to coin toss overlap results; you lose any order/time
info in the data).

e.g. study these two:

http://matplotlib.sourceforge.net/screenshots/scatter_demo2_large.png

and attached graphic (clipped from the NY Times online edition a few
days ago) which demonstrates it quite well.

Hamish

(attachments)

nyt_USpres_fundraising.gif

17

I agree about the translucency. Looks nice AND is more informative.

Michael

On 4/22/07 11:15 PM, "Hamish" <hamish_nospam@yahoo.com> wrote:

"wolle" wrote:

have a look at matplotlib http://matplotlib.sourceforge.net/ . It
depends on numeric.

Michael Barton wrote:

Really impressive!

oh, and by the way, IMO translucency is the neat way to go for bubble
plots, not sorting by size before rendering and drawing the little ones
on top (can lead to coin toss overlap results; you lose any order/time
info in the data).

e.g. study these two:

http://matplotlib.sourceforge.net/screenshots/scatter_demo2_large.png

and attached graphic (clipped from the NY Times online edition a few
days ago) which demonstrates it quite well.

Hamish

__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

18

On Sun, April 22, 2007 22:31, Glynn Clements wrote:

Trevor Wiens wrote:

> I agree that these functions should be maintained and even enhanced,
> especially for scripting.

As discussed a long time ago on the list, there is no reason why a
wrapper function couldn't be created in python to make calls to the GUI
from the command line, to allow for scripted control of the GUI using
sockets (not Unix specific ones). One socket could be listened to by
the GUI and the command line wrapper would simply pass calls to that
socket thus allowing CLI access. I had planned to develop this myself,
but being up to my ass in alligators (figuratively speaking) I've not
had the time.

Being able to remote-control the GUI is a separate issue.

Functionality (including creating graphics) should be available in an
environment where you cannot create a window (no X server, no X libs).

+1

Moritz

19

On Sun, 22 Apr 2007, Michael Barton wrote:

Thanks Hamish,

In fact, we can do considerably more now with the wxPython (and TclTk)
drawing commands to than is available in d.graph or d.linegraph, even with
your nice enhancements.

In the case of histogramming and profiling, it seems a better display to
have them come up in a separate window (like is now the case with the TclTk
profiling module) than have them rendered like a map in the main display
(like we do now for d.histogram in the current TclTk GUI). Rendering maps is
complex and it doesn't seem efficient to have to go through the same complex
procedures to draw a histogram or profile that doesn't need to be drawn
georeferenced.

Another thought - with the enhancements to the display architecture in progress/idea stage, it might actually be possible to produce better quality graphs for inclusion in reports/papers etc. using the existing display architecture. I'm thinking of the move towards PostScript as a display output format. I imagine with pop-up Windows in the GUI with histograms etc. you would be limited to printing the thing as a raster graphic at the same resolution as the screen? Not so good. Unless those new python modules you're looking at perhaps have an option to save as postscript (I mean proper line-graphics postscript) for printing. And of course if you use GRASS modules to produce the graphs then as Glynn says it is also possible to produce them from scripts, which is rather important. (Rather than having all this functionality tied up in the GUI and having to re-implement it if we change the GUI).

For this reason, it seems better to do these kinds of analyses with the gui
tools, using underlying GRASS modules like r.profile and r.stats.

I agree it "seemed" better/tidier/neater to me at first, but if we consider the display architecture is being enhanced beyond it's old clunky state it may not be as logical is it first seemed?

Paul

20

Paul Kelly wrote:

> In fact, we can do considerably more now with the wxPython (and TclTk)
> drawing commands to than is available in d.graph or d.linegraph, even with
> your nice enhancements.
>
> In the case of histogramming and profiling, it seems a better display to
> have them come up in a separate window (like is now the case with the TclTk
> profiling module) than have them rendered like a map in the main display
> (like we do now for d.histogram in the current TclTk GUI). Rendering maps is
> complex and it doesn't seem efficient to have to go through the same complex
> procedures to draw a histogram or profile that doesn't need to be drawn
> georeferenced.

Another thought - with the enhancements to the display architecture in
progress/idea stage

Idea, currently. I need to introduce non-trivial API breakage in order
to go much further. Mainly, switching to floating-point coordinates
and adding begin/end operations to the move/cont and raster
primitives.

it might actually be possible to produce better
quality graphs for inclusion in reports/papers etc. using the existing
display architecture. I'm thinking of the move towards PostScript as a
display output format.

Certainly, I want (decent) PostScript output to at least be an option.

The main question is whether we can abandon other formats, i.e.
whether modules can rely upon being able to use the full feature set
(e.g. arbitrary clip paths, rotation/scaling of all output, pattern
fills etc), or whether they will be limited to a common-denominator
subset.

I imagine with pop-up Windows in the GUI with
histograms etc. you would be limited to printing the thing as a raster
graphic at the same resolution as the screen?

Not quite. I'm assuming that lines, polygons etc wouldn't be
rasterised when using a print DC, but you're still stuck with
coordinates on integer boundaries (and a fairly primitive API).

This is one of the main distinctions between an "ideal" graphics API
(PostScript, OpenGL) and an "adequate for GUI use" API (X11, Windows
GDI).

[FWIW, the wx API is heavily based upon Windows' GDI, which hasn't
really changed much since Windows 1.0.]

Not so good. Unless those
new python modules you're looking at perhaps have an option to save as
postscript (I mean proper line-graphics postscript) for printing.

wx has PostScriptDC and PrinterDC, which allow you to use the same
code as for drawing on a screen DC. Not all operations will work on
such contexts, e.g. you can't use raster ops (XOR-plotting etc) in
PostScript. And you're still limited to the standard DC methods, e.g.
no rotated images (NT/2K/XP has this, 95/98/ME doesn't, nor does X).

And of
course if you use GRASS modules to produce the graphs then as Glynn says
it is also possible to produce them from scripts, which is rather
important. (Rather than having all this functionality tied up in the GUI
and having to re-implement it if we change the GUI).

> For this reason, it seems better to do these kinds of analyses with the gui
> tools, using underlying GRASS modules like r.profile and r.stats.

I agree it "seemed" better/tidier/neater to me at first, but if we
consider the display architecture is being enhanced beyond it's old
clunky state it may not be as logical is it first seemed?

+1 :wink:

--
Glynn Clements <glynn@gclements.plus.com>