On Mon, 6 Dec 2004, Markus Neteler wrote:
Helena,
(cc grass5)
On Sun, Dec 05, 2004 at 04:07:22PM -0500, Helena wrote:
> Markus,
>
> as I said already all we need for now is to prevent the code to go into
> swapping and
> freeze the users machines when their file is too big for their memory (this
> seems to
> be needed for both v.in.sites and v.in.ascii run without flags).
I don't think that we can prevent GRASS from swapping as it is
done by the operating system (however, better to discuss this in 'grass5' with
the other experts).
I tried v.in.sites on
http://mpa.itc.it/grasstutor/data_menu2nd.phtml
nclidar-utm.tar.gz: Ready to use GRASS LOCATION in UTM (39.5MB) for North Carolina
and it swaps too much for me as well (1GB RAM). There seems to be a memory
leak either in v.in.sites or the underlying DBMI engine.
Radim is out of town currently, and I have zero experience to track
down memory leaks.
Recently Professor Brian Ripley applied valgrind to contributed packages
in the R project with positive results, valgrind is not invasive, and may
point to memory leaks: http://valgrind.kde.org/, a copy of his posting
https://stat.ethz.ch/pipermail/r-devel/2004-November/031264.html
"Valgrind is easy to use. Valgrind uses dynamic binary translation, so you
don't need to modify, recompile or relink your applications. Just prefix
your command line with valgrind and everything works."
It sounds a bit as though trying this here might bring rewards?
Roger
Maybe someone skilled to replicate n times a local sites file and
use v.in.sites? No need to download the large nclidar-utm.tar.gz for that.
> If there is
> enough memory, it runs just fine. The problem is that I cannot import the
> files
> created in 5.3 into 5.7 on the same machine - I am not trying bigger files,
> I am just asking
> for 5.7 being able to read the same size site file as 5.3 without buying a
> new computer
> or installing more memory and if that is not possible, have the program say
> that.
I clearly understand what you wrote.
But if v.in.sites fails, maybe better v.in.ascii?
cat `g.gisenv GISDBASE`/`g.gisenv LOCATION_NAME`/`g.gisenv MAPSET`/site_lists/lidar99 |\
grep -v '^#' | sed 's+#++g' | sed 's+ %+|+g' | sed 's+ $++g' |\
v.in.ascii -z out=lidar99 xcol=1 ycol=2 catcol=3 zcol=4 \
columns='cat int, x double, y double, height double' fs='|'
If this approach works for you, we could even retire v.in.sites and make
above code a script with identical name.
> We will look at the vector code with Jaro and maybe
> others that I work with to see what we can do to get around this issue both
> for importing the site file (skipping the table? do I need it for sites in a
> way that
> it is done right now?) and reading the points in v.surf.rst. But this has
> been an unexpected issue and it is delaying the updates to v.surf.rst
>
> As for trying s.out.ascii | v.in.ascii combination - yes I tried it, as you
> might have noticed
> in my previous emails - it reads it without problems if I use -t flag.
It should also work without -t flag.
> I will try -z, this may help because there will be no attributes in case I
> don't have
> variable smoothing parameter. But I really don't want to run any of the
> v.in.* programs
> until there is an error message and the program ends in some decent way
> when it runs out of memory and starts swapping.
I assume that there is a (small, but accumulated with large data sets)
memory leak somewhere in the vector engine/DBMI. But as not being real
programmer I don't have the slightest idea how to track this down.
Markus
_______________________________________________
grass5 mailing list
grass5@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass5
--
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Breiviksveien 40, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 93 93
e-mail: Roger.Bivand@nhh.no