[GRASS-user] help with r.hants

GRASS GRASS user
1

Dear all,

I’m using the following command to run r.hants on a series of 506 maps of chlorophyll. It seems to start thinking and a few seconds after i get an error:

r.hants file=lista_hants nf=6 fet=0.1 dod=5 delta=0.1 base_period=46

ERROR: no temp files available

As stated before, the total number of images is 506 (11 years) but the base period is 1 year, hence base_period=46 images, and i’m passing a plain text file with the map names (one per line) as stated here: http://grass.osgeo.org/grass70/manuals/addons/r.hants.html

The files are registered in a spatio-temporal raster data set, if that means something, dunno.

Is the command correct? What am I missing? Do you know what that error mean?
Please, help! :slight_smile:

Thanks much in advance! And Happy Labor day for everybody!

Vero

2

hi Veronica,

On Thu, May 1, 2014 at 3:17 PM, Veronica Andreo <veroandreo@gmail.com> wrote:

Dear all,

I’m using the following command to run r.hants on a series of 506 maps of
chlorophyll. It seems to start thinking and a few seconds after i get an
error:

r.hants file=lista_hants nf=6 fet=0.1 dod=5 delta=0.1 base_period=46

ERROR: no temp files available

probably it runs out of memory? How many pixels does one map have?

As stated before, the total number of images is 506 (11 years) but the base
period is 1 year, hence base_period=46 images, and i’m passing a plain text
file with the map names (one per line) as stated here:
http://grass.osgeo.org/grass70/manuals/addons/r.hants.html

You can simply use g.mlist for that, much easier (see below).

The files are registered in a spatio-temporal raster data set, if that means
something, dunno.

I guess that this will be simply ignored, so no problem.

I have tried myself, using the new dataset prepared by Anna:

Climate dataset for North Carolina from publicly available data, with
monthly temperature average and monthly precipitation from 2000 to 2012:
http://courses.ncsu.edu/mea592/common/media/02/nc_climate_spm_2000_2012.zip

#########

GRASS 7.1.svn (nc_climate_spm_2000_2012):~ >

set computational region

g.region rast=2004_03_tempmean -p

check for the maps, especially how many

g.mlist rast pattern=20??_??_tempmean | wc -l
raster map(s) available in mapset <climate_1970_2012>:
156

try out at lower resolution for quick assessment:

g.region -p res=5000

HANTS

just wildly guessing the parameters for a test run:

r.hants input=g.mlist rast pattern=20??_??_tempmean sep=comma
nf=6 fet=0.1 dod=5 delta=0.1 base_period=12
Harmonic analysis of 156 input maps…
100%

assign reasonable color tables

for map in g.mlist rast pattern="*tempmean_hants" ; do
r.colors $map color=celsius
done

verify with one of the 156 results:

remember that we had reduced the resolution

by factor 10 to this quick assessment

r.mapcalc “diff = 2000_06_tempmean - 2000_06_tempmean_hants”

r.colors diff color=differences
d.mon wx0
d.rast diff

r.univar diff -g
n=5066
null_cells=5040
cells=10106
min=-0.0899336115228095
max=0.359362050140941
range=0.449295661663751
mean=0.188579838052468
…

see original as animation (still at 5km resolution)

note: color table is different from standard “celsius”

g.gui.animation rast=g.mlist rast pattern="*tempmean" sep=comma

see HANTS time series as animation

g.gui.animation rast=g.mlist rast pattern="*tempmean_hants" sep=comma

Looks all fine!

Best
markusN

3

Hi Markus, all

Thanks much for answering! and for the example!

···

I’m using the following command to run r.hants on a series of 506 maps of
chlorophyll. It seems to start thinking and a few seconds after i get an
error:

r.hants file=lista_hants nf=6 fet=0.1 dod=5 delta=0.1 base_period=46

ERROR: no temp files available

probably it runs out of memory? How many pixels does one map have?

I have 506 maps, 146880 pixels each… maybe that’s too much for my laptop? :stuck_out_tongue:

I’ve tried the example and works fine, also tried without changing resolution and it starts loading fine, too (i stopped it before it finished)… but when I try with my maps using g.mlist instead of file:

GRASS 7.1.svn (latlong_wgs84):~ > r.hants input=g.mlist rast pat=*_arg sep=comma nf=6 fet=0.1 dod=5 delta=0.1 base_period=46
raster map(s) available in mapset :
ERROR: no temp files available

same error… seems it doesn’t read the maps… g.mlist works just fine though… So, dunno… not sure about memory limit since it seems to work (slowly, of course) with Anna’s data set at original resolution… is there a number of characters for map names? They have an awfully long name: A20133612013365.L3m_8D_CHL_chlor_a_4km_arg

well, no idea, this data is bewitched :stuck_out_tongue:

Thanks again!

Best,
Vero

4

On Fri, May 2, 2014 at 6:20 PM, Veronica Andreo <veroandreo@gmail.com> wrote:

Hi Markus, all

Thanks much for answering! and for the example!

> I'm using the following command to run r.hants on a series of 506 maps
> of
> chlorophyll. It seems to start thinking and a few seconds after i get an
> error:
>
> r.hants file=lista_hants nf=6 fet=0.1 dod=5 delta=0.1 base_period=46
>
> ERROR: no temp files available

probably it runs out of memory? How many pixels does one map have?

I have 506 maps, 146880 pixels each... maybe that's too much for my laptop?
:stuck_out_tongue:

Please run your calculation in DEBUG mode:

g.gisenv set=DEBUG=1
r.hants ...

The relevant part for us is around this error message (it will print a
lot more of course).
You may dump the interesting part here
http://pastebin.com/
and send the resulting URL only.

We'll figure out! :slight_smile:

ciao
Markus

5

2014-05-02 14:06 GMT-03:00 Markus Neteler <neteler@osgeo.org>:

On Fri, May 2, 2014 at 6:20 PM, Veronica Andreo <veroandreo@gmail.com>
wrote:
> Hi Markus, all
>
> Thanks much for answering! and for the example!
>
>> > I'm using the following command to run r.hants on a series of 506 maps
>> > of
>> > chlorophyll. It seems to start thinking and a few seconds after i get
an
>> > error:
>> >
>> > r.hants file=lista_hants nf=6 fet=0.1 dod=5 delta=0.1 base_period=46
>> >
>> > ERROR: no temp files available
>>
>> probably it runs out of memory? How many pixels does one map have?
>
> I have 506 maps, 146880 pixels each... maybe that's too much for my
laptop?
> :stuck_out_tongue:

Please run your calculation in DEBUG mode:

g.gisenv set=DEBUG=1
r.hants ...

done

The relevant part for us is around this error message (it will print a

lot more of course).
You may dump the interesting part here
http://pastebin.com/
and send the resulting URL only.

http://pastebin.com/ySYAwNJJ

We'll figure out! :slight_smile:

:slight_smile: thanks much, again!

Vero

6

Here’s also the last part of the output when

g.gisenv set=DEBUG=3
r.hants …

···

http://pastebin.com/cp139prq

Thanks!
Vero

2014-05-02 15:27 GMT-03:00 Veronica Andreo <veroandreo@gmail.com>:

2014-05-02 14:06 GMT-03:00 Markus Neteler <neteler@osgeo.org>:

On Fri, May 2, 2014 at 6:20 PM, Veronica Andreo <veroandreo@gmail.com> wrote:

Hi Markus, all

Thanks much for answering! and for the example!

I’m using the following command to run r.hants on a series of 506 maps
of
chlorophyll. It seems to start thinking and a few seconds after i get an
error:

r.hants file=lista_hants nf=6 fet=0.1 dod=5 delta=0.1 base_period=46

ERROR: no temp files available

probably it runs out of memory? How many pixels does one map have?

I have 506 maps, 146880 pixels each… maybe that’s too much for my laptop?
:stuck_out_tongue:

Please run your calculation in DEBUG mode:

g.gisenv set=DEBUG=1
r.hants …

done

The relevant part for us is around this error message (it will print a
lot more of course).
You may dump the interesting part here
http://pastebin.com/
and send the resulting URL only.

http://pastebin.com/ySYAwNJJ

We’ll figure out! :slight_smile:

:slight_smile: thanks much, again!

Vero

7

On Sat, May 3, 2014 at 12:51 AM, Veronica Andreo <veroandreo@gmail.com> wrote:

Here's also the last part of the output when

g.gisenv set=DEBUG=3
r.hants ...

http://pastebin.com/cp139prq

OK I think I know where a modification is needed (and I think that too
many files are open for your current settings, see "ulimit" stuff in
the manual page of r.series):

...
D2/3: G_file_name(): path =
/home/veroandreo/grassdata//latlong_wgs84/clorofila/GDAL
D2/3: G_file_name(): path =
/home/veroandreo/grassdata//latlong_wgs84/clorofila/.tmp/localhost.localdomain/12772.6
D2/3: G_file_name(): path =
/home/veroandreo/grassdata//latlong_wgs84/clorofila/.tmp/localhost.localdomain/12772.7
D2/3: G_file_name(): path =
/home/veroandreo/grassdata//latlong_wgs84/clorofila/GDAL
D2/3: G_file_name(): path =
/home/veroandreo/grassdata//latlong_wgs84/clorofila/.tmp/localhost.localdomain/12772.8
D2/3: G_file_name(): path =
/home/veroandreo/grassdata//latlong_wgs84/clorofila/.tmp/localhost.localdomain/12772.9
ERROR: no temp files available
GRASS 7.1.svn (latlong_wgs84):~ >

in lib/gis/tempfile.c

char *G__tempfile(int pid)
{

[...]
    do {
    int uniq = G_counter_next(&unique);
    sprintf(name, "%d.%d", pid, uniq);
    G_file_name(path, element, name, G_mapset());
    }
    while (access(path, F_OK) == 0);
[...]

We need to decipher the error with perror() as done in

in lib/gis/mapset.c

    if (access(mapset, F_OK) != 0) {
    perror("access");
    G_fatal_error(_("MAPSET <%s> not available"), mapset);
    }

Anyone able to modify in lib/gis/tempfile.c properly?

thanks
Markus

8

On Sat, May 3, 2014 at 3:09 PM, Markus Neteler <neteler@osgeo.org> wrote:

On Sat, May 3, 2014 at 12:51 AM, Veronica Andreo <veroandreo@gmail.com> wrote:

Here's also the last part of the output when

g.gisenv set=DEBUG=3
r.hants ...

http://pastebin.com/cp139prq

...

Anyone able to modify in lib/gis/tempfile.c properly?

I have opened a ticket:
http://trac.osgeo.org/grass/ticket/2278

Markus

9

Hi Markus and all

Thanks much for all the help! Hope someone is willing to address the task in the ticket. It would be helpful to have a more informative error message.

I’ve changed the limit of open files with ulimit to 2000 but it wasn’t working anyway… so, i tried with -z flag, and it is doing the job now, though (as expected) it is taking quite some time… I should have tried that from the beginning :stuck_out_tongue:

All the best! And thanks again!
Vero

···

2014-05-04 5:33 GMT-03:00 Markus Neteler <neteler@osgeo.org>:

On Sat, May 3, 2014 at 3:09 PM, Markus Neteler <neteler@osgeo.org> wrote:

On Sat, May 3, 2014 at 12:51 AM, Veronica Andreo <veroandreo@gmail.com> wrote:

Here’s also the last part of the output when

g.gisenv set=DEBUG=3
r.hants …

http://pastebin.com/cp139prq

…

Anyone able to modify in lib/gis/tempfile.c properly?

I have opened a ticket:
http://trac.osgeo.org/grass/ticket/2278

Markus