[GRASS-dev] [GRASS GIS] #2278: G__tempfile() lacks perror support

GRASS GIS GRASS GIS developer
,
1

#2278: G__tempfile() lacks perror support
-------------------------+--------------------------------------------------
Reporter: neteler | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: LibGIS | Version: svn-releasebranch70
Keywords: r.hants | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------
(The problem arises from r.hants Addon use, reported on grass-user)

Using r.hants (derived from r.series) on almost 600 raster
maps generates an "ERROR: no temp files available" on
Linux 64bit. The input maps have been registered with
r.external:

{{{
D2/3: G_file_name(): path = /home/user/grassdata/latlong/cloro/GDAL
D2/3: G_file_name(): path =
/home/user/grassdata/latlong/cloro/.tmp/localhost.localdomain/12772.6
D2/3: G_file_name(): path =
/home/user/grassdata/latlong/cloro/.tmp/localhost.localdomain/12772.7
D2/3: G_file_name(): path = /home/user/grassdata/latlong/cloro/GDAL
D2/3: G_file_name(): path =
/home/user/grassdata/latlong/cloro/.tmp/localhost.localdomain/12772.8
D2/3: G_file_name(): path =
/home/user/grassdata/latlong/cloro/.tmp/localhost.localdomain/12772.9
ERROR: no temp files available
GRASS 7.1.svn (latlong_wgs84):~ >
}}}

For a better error message, perror() should be used:

in lib/gis/tempfile.c
{{{
char *G__tempfile(int pid)
{

[...]
     do {
     int uniq = G_counter_next(&unique);
     sprintf(name, "%d.%d", pid, uniq);
     G_file_name(path, element, name, G_mapset());
     }
     while (access(path, F_OK) == 0);
[...]
}}}

We need to explain the error with perror() as done in

in lib/gis/mapset.c
{{{
     if (access(mapset, F_OK) != 0) {
     perror("access");
     G_fatal_error(_("MAPSET <%s> not available"), mapset);
}}}

Anyone able to modify the while() loop in lib/gis/tempfile.c
accordingly?

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2278&gt;
GRASS GIS <http://grass.osgeo.org>

2

#2278: G__tempfile() lacks perror support
-------------------------+--------------------------------------------------
Reporter: neteler | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: LibGIS | Version: svn-releasebranch70
Keywords: r.hants | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by glynn):

Replying to [ticket:2278 neteler]:

> For a better error message, perror() should be used:

Ideally, strerror() (or similar) should be used, and the string included
in the error message. Otherwise, the additional details will be omitted
from any logfile ($GIS_ERROR_LOG) or email message ($GRASS_ERROR_MAIL).

Except, strerror() isn't thread-safe (it returns a pointer to a static
buffer). POSIX.1-2001 specifies strerror_r(), which requires the caller to
supply a buffer. Windows doesn't have that, but it has strerror_s (same
functionality, different argument order). So we'd need configure checks.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2278#comment:1&gt;
GRASS GIS <http://grass.osgeo.org>

3

#2278: G__tempfile() lacks perror support
-------------------------+--------------------------------------------------
Reporter: neteler | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: LibGIS | Version: svn-releasebranch70
Keywords: r.hants | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by mmetz):

Replying to [ticket:2278 neteler]:
> (The problem arises from r.hants Addon use, reported on grass-user)
>
> Using r.hants (derived from r.series) on almost 600 raster
> maps generates an "ERROR: no temp files available" on
> Linux 64bit.

r.series with the same number of input maps should fail with the same
error. r.hants has been derived from r.series, and both modules have the
-z flag (Don't keep files open) which should avoid this problem.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2278#comment:2&gt;
GRASS GIS <http://grass.osgeo.org>

4

#2278: G__tempfile() lacks perror support
-------------------------+--------------------------------------------------
Reporter: neteler | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: LibGIS | Version: svn-releasebranch70
Keywords: r.hants | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by mmetz):

Replying to [comment:2 mmetz]:
> Replying to [ticket:2278 neteler]:
> > (The problem arises from r.hants Addon use, reported on grass-user)
> >
> > Using r.hants (derived from r.series) on almost 600 raster
> > maps generates an "ERROR: no temp files available" on
> > Linux 64bit.
>
> r.series with the same number of input maps should fail with the same
error. r.hants has been derived from r.series, and both modules have the
-z flag (Don't keep files open) which should avoid this problem.

Sorry, the -z flag does not help here because r.hants creates a new output
map for each input map, and these need to be kept open. For 600 input
maps, the hard limit of the number of open files is usually exceeded.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2278#comment:3&gt;
GRASS GIS <http://grass.osgeo.org>

5

#2278: G__tempfile() lacks perror support
-------------------------+--------------------------------------------------
Reporter: neteler | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: LibGIS | Version: svn-releasebranch70
Keywords: r.hants | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by neteler):

Replying to [comment:1 glynn]:
> Replying to [ticket:2278 neteler]:
>
> > For a better error message, perror() should be used:
>
> Ideally, strerror() (or similar) should be used, and the string included
in
> the error message.

Done by mmetz in r60148 and related.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2278#comment:4&gt;
GRASS GIS <http://grass.osgeo.org>

6

#2278: G__tempfile() lacks perror support
-------------------------+--------------------------------------------------
Reporter: neteler | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: LibGIS | Version: svn-releasebranch70
Keywords: r.hants | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by glynn):

Replying to [comment:4 neteler]:

> > Ideally, strerror() (or similar) should be used, and the string
included in
> > the error message.
>
> Done by mmetz in r60148 and related.

For robustness, errno should be copied to a local variable immediately
after the failed call, e.g.:

{{{
     fcb->null_fd = creat(tempname, 0666);
     if (fcb->null_fd < 0) {
         int err = errno;
         ...
         G_fatal_error(_("No temp files available: %s"), strerror(err));
     }
}}}

Otherwise, it may be modified by an intervening call (specifically,
close() can modify errno).

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2278#comment:5&gt;
GRASS GIS <http://grass.osgeo.org>