[GRASSLIST:5897] Working with grass in clusteres enviroments.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi.

I was wondering if, when and how Grass will ever support clustering.
Now days, setting up a openMosix cluster is cheap and easy, giving on
the other hand pure computing power. As far as I can understand, the
fact that Grass is a collection of unix programs that can run
indipendently on the same machine makes Grass the optimum playground for
testing mosix clusters.
Do you know of any Grass-openmosix experiments on the net?
BTW. I had pointed out in previous months that some Grass apps would
benefit enourmously from being parallized. I think of r.mapcalc,
r.surf.contour, etc.
Are there plans of parallelizing such apps?

Greetings

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: Per informazioni si veda http://www.gnupg.org

iD8DBQE+hM5cIRaQQEznKxURAu5LAJ9/hpwgdX6VbustnlTGBrWZboThHQCfd2we
4BlNDCrXn4QJk1NPxnfIMlY=
=0YsE
-----END PGP SIGNATURE-----

Antonio G. - Geotronix wrote:

I was wondering if, when and how Grass will ever support clustering.
Now days, setting up a openMosix cluster is cheap and easy, giving on
the other hand pure computing power. As far as I can understand, the
fact that Grass is a collection of unix programs that can run
indipendently on the same machine makes Grass the optimum playground for
testing mosix clusters.
Do you know of any Grass-openmosix experiments on the net?
BTW. I had pointed out in previous months that some Grass apps would
benefit enourmously from being parallized. I think of r.mapcalc,
r.surf.contour, etc.
Are there plans of parallelizing such apps?

No.

Or, to put it another way, if anyone is planning on adding parallelism
to GRASS modules, they are likely to be disappointed. The core GRASS
libraries aren't remotely threadsafe, so any parallelism would have to
be restricted to pure memory-to-memory operations; making multiple
concurrent calls to GRASS library functions won't work, and can't be
made to work without a fundamental re-write (i.e. discarding GRASS and
re-implementing equivalent functionality from scratch).

For the foreseeable future, the only feasible way to exploit
parallelism would be to run multiple sessions.

--
Glynn Clements <glynn.clements@virgin.net>

> I was wondering if, when and how Grass will ever support clustering.
> Now days, setting up a openMosix cluster is cheap and easy,

No.

Or, to put it another way, if anyone is planning on adding parallelism
to GRASS modules, they are likely to be disappointed. The core GRASS
libraries aren't remotely threadsafe, so any parallelism would have to
be restricted to pure memory-to-memory operations; making multiple
concurrent calls to GRASS library functions won't work, and can't be
made to work without a fundamental re-write (i.e. discarding GRASS and
re-implementing equivalent functionality from scratch).

For the foreseeable future, the only feasible way to exploit
parallelism would be to run multiple sessions.

Note that (as far as I understand) mosix clusters will not split up
running processes, just shift the job to the most available/fastest
processor in the cluster. It is a lot like having an asymmetric
multi-processor system.. mosix is a load balancing tool mainly.

The time your job takes to run is proportional to the fastest single
processor available to it, NOT the sum total of all the available
processing power in the cluster. You'd need a Beowulf Cluster and
specially written threaded code if you wanted that.

For example, your job will take the same time to run on a mosix cluster
of two or two thousand 2.4GHz computers. You can just run a lot more
jobs on the two thousand computer cluster without a slow down.

I don't think running GRASS on a mosix cluster would be any different
from backgrounding a job, e.g. "s.surf.rst {...} &", and then continuing
on doing other stuff. As long as you are very careful not to interfere
with any of the input/output files belonging to another process, of
course.

I had two concurrent s.surf.rst calculations running the other day
testing different tension= parameters, and everything seemed to work
correctly. It is in this type of usage where a dual-processor computer
(or mosix cluster) would be most useful to you, I would think.

I think I've got that right anyway.

Hamish

I need to create shore normal transects from established sampling points at Salton Sea, Ca.

How do I automate this in GRASS ?
The tangent point of a radius around the sample would work for this relatively simple coastline.

In ArcView a "join" calculation records a tangent distance between features. Is there a similar calculation available in GRASS. or do I need to make a looping calculation that increments a radius until a tangent solution is reached ?

A more complex coastline would require "smoothing". The normal vector including the sample point would describe the shore normal transect. This approach would require selection and creation of seg,ments of the "smoothed" coastline. How would I automate the selection process and store the locally smoothed segments.

On Sun, Mar 30, 2003 at 07:57:42AM -0800, John Chesnut wrote:

I need to create shore normal transects from established sampling
points at Salton Sea, Ca.

How do I automate this in GRASS ?
The tangent point of a radius around the sample would work for this
relatively simple coastline.

In ArcView a "join" calculation records a tangent distance between
features. Is there a similar calculation available in GRASS. or do I
need to make a looping calculation that increments a radius until a
tangent solution is reached ?

v.distance almost does what you need, but not quite, since it doesn't
return the point on the shoreline. With some effort, it could be
modified, though the search routines used by v.distance would have to be
recreated ;-(.

Presuming the sample locations aren't actually on the shoreline, then
there is a function in the diglib that will calculate the distance of
a point to a line, returning the distance and the point of intersection.
The distance line segment will be normal to the coastline unless the
point is outside the "segment space" (in which case, it is the shortest
distance to an end point). If the point is actually on the shoreline,
then the distance will be "zero" and the end point will "equal" the
start point (quotes for floating point tolerances of precision). The
function only works in planar coordinate systems. In a brute force
iteration, the shortest distance/point will be the desired end point
and distance. The "status" flag (below) will tell you if it is
perpendicular to the shoreline segment or the distance to a vertex.

src/libes/vect32/diglib/line_dist.c

/*
** dig_xy_distance3_point_to_line ()
** same as dig_distance2_point_to_line () except returns value in status
** telling if point is w/in segment space, or past ends
**
** status = 0 if ok, -1 if t < 0 and 1 if t > 1
**
*/

double dig_xy_distance3_point_to_line (
    double *px,double *py, /* point */
    double x1,double y1,double x2,double y2, /* line segment */
    int *status)

A more complex coastline would require "smoothing". The normal
vector including the sample point would describe the shore normal
transect. This approach would require selection and creation of
seg,ments of the "smoothed" coastline. How would I automate the
selection process and store the locally smoothed segments.

Maybe v.prune? I don't know if it really will create the desired
smoothing though...

--
echo ">gra.fcw@2ztr< eryyvZ .T pveR" | rot13 | reverse

On Fri, Mar 28, 2003 at 11:36:12PM +0100, Antonio G. - Geotronix wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi.

I was wondering if, when and how Grass will ever support clustering.
Now days, setting up a openMosix cluster is cheap and easy, giving on
the other hand pure computing power. As far as I can understand, the
fact that Grass is a collection of unix programs that can run
indipendently on the same machine makes Grass the optimum playground for
testing mosix clusters.
Do you know of any Grass-openmosix experiments on the net?

Not on the net, but I have done that recently.

BTW. I had pointed out in previous months that some Grass apps would
benefit enourmously from being parallized. I think of r.mapcalc,
r.surf.contour, etc.
Are there plans of parallelizing such apps?

No plans at time. Of course (new) developers are welcome to pick up
such task.

In general GRASS can only be run pseudo-parallel, i.e. multiple-sequentially
on an openMosix cluster. I have written a startup-script for launching
several GRASS sessions in parallel. My task was to reclass some hundred
aerial images (i.smap). They were processed individually in multiple sessions.
The main trick is to run such multiple GRASS sessions in own mapsets
within the same location. Only then you can avoid problems that the
WIND file is not overwritten by concurring sessions.

If there is any interest in the launch script, I can make it available.

Markus Neteler