[GRASS-dev] opencell opening temp null file: no temp files available

Andi_Jochem · November 19, 2008, 3:48pm

Hello!

I found out that it has nothing to do with my open and closed cell files

I must be something in my reproject function.

I read in some points (2000) and each point is transformed to lat/lang using following function

void transfform_lat(double longitude, double latitude){

    struct Key_Value *in_proj_info, *in_unit_info;
    struct pj_info iproj;
    struct pj_info oproj;

...
...
...

  if ((in_proj_info = G_get_projinfo()) == NULL)
    G_fatal_error("Can't get projection info of current location");

  if ((in_unit_info = G_get_projunits()) == NULL)
    G_fatal_error("Can't get projection units of current location");

  if (pj_get_kv(&iproj, in_proj_info, in_unit_info) < 0)
    G_fatal_error("Can't get projection key values of current location");

  G_free_key_value( in_proj_info );
  G_free_key_value( in_unit_info );

  /* set output projection to lat/long for solpos*/
  oproj.zone = 0;
  oproj.meters = 1.;
  sprintf(oproj.proj, "ll");
  if ((oproj.pj = pj_latlong_from_proj(iproj.pj)) == NULL)
    G_fatal_error("Unable to set up lat/long projection parameters");

  if(pj_do_proj(&longitude, &latitude, &iproj, &oproj) < 0)
  {
  fprintf(stderr,"Error in pj_do_proj (projection of input coordinate pair)\n");
  exit(0);
  }

  ......

}

After 1024 projections the programm stops.

When I have a look in /proc/<pid>/fd/

I can see the following:
...
...
...
lr-x------ 1 andi andi 64 2008-11-19 16:21 66 -> /usr/local/grass-6.3.0/etc/ellipse.table
lr-x------ 1 andi andi 64 2008-11-19 16:21 67 -> /usr/local/grass-6.3.0/etc/ellipse.table
lr-x------ 1 andi andi 64 2008-11-19 16:21 68 -> /usr/local/grass-6.3.0/etc/ellipse.table
lr-x------ 1 andi andi 64 2008-11-19 16:21 69 -> /usr/local/grass-6.3.0/etc/ellipse.table
lr-x------ 1 andi andi 64 2008-11-19 16:19 7 -> /usr/local/grass-6.3.0/etc/ellipse.table
lr-x------ 1 andi andi 64 2008-11-19 16:21 70 -> /usr/local/grass-6.3.0/etc/ellipse.table
lr-x------ 1 andi andi 64 2008-11-19 16:21 71 -> /usr/local/grass-6.3.0/etc/ellipse.table
lr-x------ 1 andi andi 64 2008-11-19 16:21 72 -> /usr/local/grass-6.3.0/etc/ellipse.table
lr-x------ 1 andi andi 64 2008-11-19 16:21 73 -> /usr/local/grass-6.3.0/etc/ellipse.table
lr-x------ 1 andi andi 64 2008-11-19 16:21 74 -> /usr/local/grass-6.3.0/etc/ellipse.table
lr-x------ 1 andi andi 64 2008-11-19 16:21 75 -> /usr/local/grass-6.3.0/etc/ellipse.table
lr-x------ 1 andi andi 64 2008-11-19 16:21 76 -> /usr/local/grass-6.3.0/etc/ellipse.table
...

Does anybody know how to solve the problem???

Thanks.

Andi

Glynn Clements wrote:

> Andruit@gmx.de wrote:
>

>> I am writing a Grass programm which creates a raster, reads the values
>> of the raster map at certain points and creates a new rastermap having
>> the same name as the old one.
>>
>> I am using the function
>>
>> G_open_cell_new() to overwrite the existing raster map.
>>
>> and the function
>>
>> G_close_cell()
>>
>>
>> It works fine.
>>
>> But after about 1000 loops I get the following messsage.
>>
>> WARNING: opencell opening temp null file: no temp files available
>>
>> Does anyone know what it means. I have enough space left in my ./tmp
>> folder. And I also have write permissions on it.

>
> It probably means that files aren't getting closed, so you are
> exceeding the limit on the number of open files per process (by
> default 1024, check with ulimit -n).
>
> On Linux, you can check which files a process has open by looking in
> the /proc/<pid>/fd directory, where <pid> is the PID of the process.
>
> If you're modifying a map "in-place", you'll need to close and re-open
> the input map in order to see the changes. Opening a map for write
> creates a temporary file, which is renamed over the original when
> closed. If the original cell/fcell file is still open for read at that
> point, it will still exist and still be open (it will show up as
> "(deleted)" in /proc/<pid>/fd).
>

-- Ist Ihr Browser Vista-kompatibel? Jetzt die neuesten Browser-Versionen downloaden: http://www.gmx.net/de/go/browser
-- "Feel free" - 10 GB Mailbox, 100 FreeSMS/Monat ... Jetzt GMX TopMail testen: http://www.gmx.net/de/go/topmail _______________________________________________ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
--
Sensationsangebot nur bis 30.11: GMX FreeDSL - Telefonanschluss + DSL
für nur 16,37 Euro/mtl.!* http://dsl.gmx.de/?ac=OM.AD.PD003K11308T4569a

Glynn_Clements1 · November 19, 2008, 5:13pm

Andi Jochem wrote:

I found out that it has nothing to do with my open and closed cell files

I must be something in my reproject function.

I read in some points (2000) and each point is transformed to lat/lang using following function

void transfform_lat(double longitude, double latitude){

    struct Key_Value *in_proj_info, *in_unit_info;
    struct pj_info iproj;
    struct pj_info oproj;

...
...
...

  if ((in_proj_info = G_get_projinfo()) == NULL)
    G_fatal_error("Can't get projection info of current location");

  if ((in_unit_info = G_get_projunits()) == NULL)
    G_fatal_error("Can't get projection units of current location");

  if (pj_get_kv(&iproj, in_proj_info, in_unit_info) < 0)
    G_fatal_error("Can't get projection key values of current location");

  G_free_key_value( in_proj_info );
  G_free_key_value( in_unit_info );

  /* set output projection to lat/long for solpos*/
  oproj.zone = 0;
  oproj.meters = 1.;
  sprintf(oproj.proj, "ll");
  if ((oproj.pj = pj_latlong_from_proj(iproj.pj)) == NULL)
    G_fatal_error("Unable to set up lat/long projection parameters");

You shouldn't be doing this for each point. Do it once, and keep iproj
and oproj around.

After 1024 projections the programm stops.

When I have a look in /proc/<pid>/fd/

I can see the following:
...
...
...
lr-x------ 1 andi andi 64 2008-11-19 16:21 66 -> /usr/local/grass-6.3.0/etc/ellipse.table
lr-x------ 1 andi andi 64 2008-11-19 16:21 67 -> /usr/local/grass-6.3.0/etc/ellipse.table

[snip]

This looks like a GRASS bug.

read_ellipsoid_table() in lib/proj/ellipse.c never closes the file.
You can fix it with:

--- lib/proj/ellipse.c (revision 31109)
+++ lib/proj/ellipse.c (working copy)
@@ -264,6 +264,9 @@
       continue;
   }
     }
+
+ fclose(fd);
+
     if (!err)
   return outputlist;

This has been fixed in 6.4/7.0.

read_ellipsoid_table() in lib/gis/get_ellipse.c has the same issue
(which I've now fixed). However, that function checks whether the
table has already been read, and returns immediately if it has, so it
never opens the file more than once.

I suspect the lib/proj version should do something similar. In fact,
it should probably use the code in lib/gis.

--
Glynn Clements <glynn@gclements.plus.com>

neteler · November 20, 2008, 6:58pm

On Wed, Nov 19, 2008 at 6:13 PM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Andi Jochem wrote:

...

After 1024 projections the programm stops.

When I have a look in /proc/<pid>/fd/

I can see the following:
...
...
...
lr-x------ 1 andi andi 64 2008-11-19 16:21 66 -> /usr/local/grass-6.3.0/etc/ellipse.table
lr-x------ 1 andi andi 64 2008-11-19 16:21 67 -> /usr/local/grass-6.3.0/etc/ellipse.table

[snip]

This looks like a GRASS bug.

read_ellipsoid_table() in lib/proj/ellipse.c never closes the file.
You can fix it with:

--- lib/proj/ellipse.c (revision 31109)
+++ lib/proj/ellipse.c (working copy)
@@ -264,6 +264,9 @@
           continue;
       }
    }
+
+ fclose(fd);
+
    if (!err)
       return outputlist;

This has been fixed in 6.4/7.0.

read_ellipsoid_table() in lib/gis/get_ellipse.c has the same issue
(which I've now fixed). However, that function checks whether the
table has already been read, and returns immediately if it has, so it
never opens the file more than once.

I suspect the lib/proj version should do something similar. In fact,
it should probably use the code in lib/gis.

this probably solves
http://trac.osgeo.org/gdal/ticket/2537
?

Markus

Glynn_Clements1 · November 20, 2008, 9:08pm

Markus Neteler wrote:

this probably solves
http://trac.osgeo.org/gdal/ticket/2537

Part of it. r34411 and r34412 eliminate the leaks regarding "file" and
"errbuf".

--
Glynn Clements <glynn@gclements.plus.com>