- memory usage: can anything be done to keep this under control? it
chews through my gigabyte of memory pretty quickly.. I suppose if the
above target mask is implemented, this would not be such an issue?
Hamish,
memory usage is r.sun's one of the weakest points. Thomas Huld who helped me with the latest improvements is currently working on this as well.
However, there is another problem with memory associated with proj library. If you have latitude defined via region settings (i.e. you don't define lat and latin options) then memory usage is unbelievably high. I guess it is some memory allocation/freeing problem in proj library. I have used proj library functions to get latitude/longitute for each ratser cell using examples from grass programmers manual. I hadn't time to figure out the problem in detail. I would appreciate if could look at it - perhaps you could check if such problems with proj library functions are also in other grass programs or check the source code of these functions. Thanks.
However, there is another problem with memory associated with proj
library. If you have latitude defined via region settings (i.e. you
don't define lat and latin options) then memory usage is unbelievably
high. I guess it is some memory allocation/freeing problem in proj
library. I have used proj library functions to get latitude/longitute
for each ratser cell using examples from grass programmers manual. I
I didn't realise r.sun calls the proj functions so often. Maybe it needs
to free the memory used for the key/value pairs read from the PROJ_INFO
file. Does the patch below help? I don't use r.sun so can't really test
it. When I have time I mean to update the example in the programmer's manual.
However, there is another problem with memory associated with proj
library. If you have latitude defined via region settings (i.e. you
don't define lat and latin options) then memory usage is unbelievably
high. I guess it is some memory allocation/freeing problem in proj
library. I have used proj library functions to get latitude/longitute
for each ratser cell using examples from grass programmers manual. I
I didn't realise r.sun calls the proj functions so often. Maybe it needs
to free the memory used for the key/value pairs read from the PROJ_INFO
file. Does the patch below help? I don't use r.sun so can't really test
it. When I have time I mean to update the example in the programmer's manual.
> However, there is another problem with memory associated with proj
> library. If you have latitude defined via region settings (i.e. you
> don't define lat and latin options) then memory usage is
> unbelievably high. I guess it is some memory allocation/freeing
> problem in proj library.
[...]
I didn't realise r.sun calls the proj functions so often. Maybe it
needs to free the memory used for the key/value pairs read from the
PROJ_INFO file. Does the patch below help?
Brilliant! Memory usage is now 2.5 times less than it was. It still
grows into the hundreds of megabytes as it runs which makes me suspect
there are about 2-3 more of them in there to find.
Some numbers:
517x416 cells (100m res)
w/o patch: grew to 300mb RAM
with patch: grew to 117mb
with lat=c: steady 6mb
--
time to run on a 1.6gHzP4: 1min 2sec w/o patch
1034x832 cells (50m res)
w/o patch: I killed it at 940mb 69% complete when it went to swap
with patch: grew to 485mb
with lat=c: steady 21mb
--
time to run: 4min 6 sec w/o, 3min 36 sec lat=c
while we're at it, s.out.ascii's memory use is pretty terrible, and
grows out of control as well. The program is only a few lines long, so
maybe someone who knows the libraries better than I could spot it.. I
just had it die only a little bit into a 3million point site file,
before I remembered I could just use the wonders of vi to "%s/|/\t/g" ..
I tried swapping fprintf(stdout, to a file but the memory problem
persisted, for whatever that's worth.
while we're at it, s.out.ascii's memory use is pretty terrible, and
grows out of control as well. The program is only a few lines long, so
maybe someone who knows the libraries better than I could spot it.
while (G_site_get (fd, site) == 0)
{
...
if (!full)
...
else
fprintf (stdout,"%s\n", G_site_format (site, fs, strip));
}
G_site_format() uses malloc() to allocate the buffer for the returned
string, so it needs to be free()d afterwards.
> while we're at it, s.out.ascii's memory use is pretty terrible, and
> grows out of control as well. The program is only a few lines long, so
> maybe someone who knows the libraries better than I could spot it.