[GRASS-dev] [GRASS GIS] #3685: pygrass GridModule: ERROR: Unable to open element file <windows>

GDAL · October 19, 2018, 9:23am

#3685: pygrass GridModule: ERROR: Unable to open element file <windows>
-------------------------+-------------------------
Reporter: mlennert | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.6.0
Component: PyGRASS | Version: svn-trunk
Keywords: GridModule | CPU: Unspecified
Platform: Unspecified |
-------------------------+-------------------------
I've been using a script with code resembling the attached python script
for months (notably through the i.cutlines addon), without any error. It
uses the GridModule class from pygrass to run i.zc on tiles of an input
map (as i.zc reads everything into memory).

I suddenly get the following errors, both with trunk and release76:

{{{
GRASS 7.7.svn (nc_spm_08):~ > ./grid_test.py
Creating edge map
ERROR: ERROR: ERROR: UnableUnableUnable toto openopen elementelement
filefile to open<windows> <windows>for elementfor file

        <tmp.grid_test.py.498@izc_000_002>
  <tmp.grid_test.py.498@izc_000_003>
<windows> for
        <tmp.grid_test.py.498@izc_000_001>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_000_000>
ERROR: ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_000_003>
Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_000_001>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_000_000>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_000_002>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_001_001>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_001_002>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_001_000>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_001_003>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_001_001>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_001_002>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_001_000>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_001_003>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_002_000>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_002_001>
ERROR: ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_002_003>
Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_002_002>
ERROR: ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_002_001>
Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_002_000>
ERROR: ERROR: UnableUnable toto openopen elementelement filefile
<windows><windows> forfor

<tmp.grid_test.py.498@izc_002_002><tmp.grid_test.py.498@izc_002_003>

ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_003_001>
ERROR: ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_003_000>
Unable to open element file <windows> for
        ERROR: <tmp.grid_test.py.498@izc_003_002>
Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_003_003>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_003_001>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_003_003>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_003_000>
ERROR: Unable to open element file <windows> for
        <tmp.grid_test.py.498@izc_003_002>
Traceback (most recent call last):
   File "./grid_test.py", line 33, in <module>
     grd.run()
   File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-
gnu/etc/python/grass/pygrass/modules/grid/grid.py", line 568, in run
     self.patch()
   File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-
gnu/etc/python/grass/pygrass/modules/grid/grid.py", line 609, in patch
     self.start_row, self.start_col, self.out_prefix)
   File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-
gnu/etc/python/grass/pygrass/modules/grid/patch.py", line 79, in
rpatch_map
     rtype.open('r')
   File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-
gnu/etc/python/grass/pygrass/raster/__init__.py", line 210, in open
     raise OpenError(str_err)
grass.exceptions.OpenError: The map does not exist, I can't open in 'r'
mode
}}}

Does anyone have any idea what might be going wrong ? When I look into one
of these izc_* directories, I see the following:

{{{
GRASS 7.6.svn (nc_spm_08):~ > ls -l GRASSDATA/nc_spm_08/izc_002_002/
total 16
-rw-r--r-- 1 mlennert mlennert 14 oct 19 11:20 CURGROUP
-rw-r--r-- 1 mlennert mlennert 291 oct 19 11:20 SEARCH_PATH
-rw-r--r-- 1 mlennert mlennert 81 oct 19 11:20 VAR
-rw-r--r-- 1 mlennert mlennert 317 oct 19 11:20 WIND
}}}

Does anyone have any idea about what might be going on ? This keeps me
from launching quite long-running processes, so it would be great if I
could solve this rapidly. I have to admit I'm a bit lost even just trying
to understand what might be wrong and how I should debug this. Any hints
are more than welcome.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3685>
GRASS GIS <https://grass.osgeo.org>

mlennert · October 19, 2018, 11:58am

Hi everyone,

As this is quite urgent in the context of my work, I would love some help on this issue. Does anyone have any hint as to where I should start looking about this ?

Moritz

-------- Forwarded Message --------
Subject: [GRASS-dev] [GRASS GIS] #3685: pygrass GridModule: ERROR: Unable to open element file <windows>
Date: Fri, 19 Oct 2018 09:23:26 -0000
From: GRASS GIS <trac@osgeo.org>
Reply-To: grass-dev@lists.osgeo.org
CC: grass-dev@lists.osgeo.org

#3685: pygrass GridModule: ERROR: Unable to open element file <windows>
-------------------------+-------------------------
  Reporter: mlennert | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.6.0
Component: PyGRASS | Version: svn-trunk
  Keywords: GridModule | CPU: Unspecified
  Platform: Unspecified |
-------------------------+-------------------------
  I've been using a script with code resembling the attached python script
  for months (notably through the i.cutlines addon), without any error. It
  uses the GridModule class from pygrass to run i.zc on tiles of an input
  map (as i.zc reads everything into memory).

I suddenly get the following errors, both with trunk and release76:

  {{{
  GRASS 7.7.svn (nc_spm_08):~ > ./grid_test.py
  Creating edge map
  ERROR: ERROR: ERROR: UnableUnableUnable toto openopen elementelement
  filefile to open<windows> <windows>for elementfor file

         <tmp.grid_test.py.498@izc_000_002>
   <tmp.grid_test.py.498@izc_000_003>
  <windows> for
         <tmp.grid_test.py.498@izc_000_001>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_000_000>
  ERROR: ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_000_003>
  Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_000_001>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_000_000>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_000_002>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_001_001>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_001_002>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_001_000>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_001_003>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_001_001>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_001_002>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_001_000>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_001_003>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_002_000>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_002_001>
  ERROR: ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_002_003>
  Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_002_002>
  ERROR: ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_002_001>
  Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_002_000>
  ERROR: ERROR: UnableUnable toto openopen elementelement filefile
  <windows><windows> forfor

<tmp.grid_test.py.498@izc_002_002><tmp.grid_test.py.498@izc_002_003>

  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_003_001>
  ERROR: ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_003_000>
  Unable to open element file <windows> for
         ERROR: <tmp.grid_test.py.498@izc_003_002>
  Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_003_003>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_003_001>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_003_003>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_003_000>
  ERROR: Unable to open element file <windows> for
         <tmp.grid_test.py.498@izc_003_002>
  Traceback (most recent call last):
    File "./grid_test.py", line 33, in <module>
      grd.run()
    File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-
  gnu/etc/python/grass/pygrass/modules/grid/grid.py", line 568, in run
      self.patch()
    File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-
  gnu/etc/python/grass/pygrass/modules/grid/grid.py", line 609, in patch
      self.start_row, self.start_col, self.out_prefix)
    File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-
  gnu/etc/python/grass/pygrass/modules/grid/patch.py", line 79, in
  rpatch_map
      rtype.open('r')
    File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-
  gnu/etc/python/grass/pygrass/raster/__init__.py", line 210, in open
      raise OpenError(str_err)
  grass.exceptions.OpenError: The map does not exist, I can't open in 'r'
  mode
  }}}

Does anyone have any idea what might be going wrong ? When I look into one
of these izc_* directories, I see the following:

  {{{
  GRASS 7.6.svn (nc_spm_08):~ > ls -l GRASSDATA/nc_spm_08/izc_002_002/
  total 16
  -rw-r--r-- 1 mlennert mlennert 14 oct 19 11:20 CURGROUP
  -rw-r--r-- 1 mlennert mlennert 291 oct 19 11:20 SEARCH_PATH
  -rw-r--r-- 1 mlennert mlennert 81 oct 19 11:20 VAR
  -rw-r--r-- 1 mlennert mlennert 317 oct 19 11:20 WIND
  }}}

  Does anyone have any idea about what might be going on ? This keeps me
  from launching quite long-running processes, so it would be great if I
  could solve this rapidly. I have to admit I'm a bit lost even just trying
  to understand what might be wrong and how I should debug this. Any hints
  are more than welcome.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3685>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 19, 2018, 12:37pm

#3685: pygrass GridModule: ERROR: Unable to open element file <windows>
--------------------------+-------------------------
  Reporter: mlennert | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.6.0
Component: PyGRASS | Version: svn-trunk
Resolution: | Keywords: GridModule
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------
Changes (by mlennert):

* Attachment "grid_test.py" added.

script to reproduce error

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3685>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 19, 2018, 2:02pm

#3685: pygrass GridModule: ERROR: Unable to open element file <windows>
--------------------------+-------------------------
  Reporter: mlennert | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.6.0
Component: PyGRASS | Version: svn-trunk
Resolution: | Keywords: GridModule
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------

Comment (by Nikos Alexandris):

Same errors here. What is the content of the WIND file(s).
Here, all of the WIND files, inside each izc_* directory, are identical
and set to

{{{
proj: 99
zone: 0
north: 1000
south: 0
east: 1000
west: 0
cols: 1000
rows: 1000
e-w resol: 1
n-s resol: 1
top: 1.000000000000000
bottom: 0.000000000000000
cols3: 1000
rows3: 1000
depths: 1
e-w resol3: 1
n-s resol3: 1
t-b resol: 1
}}}

which is the "default" region in the Location 'nc_spm_08_grass7', Mapset
'PERMANENT':

{{{
g.region -p
projection: 99 (Lambert Conformal Conic)
zone: 0
datum: nad83
ellipsoid: a=6378137 es=0.006694380022900787
north: 1000
south: 0
west: 0
east: 1000
nsres: 1
ewres: 1
rows: 1000
cols: 1000
cells: 1000000
}}}

Yet the raster map lies elsewhere:
{{{
r.info ortho_2001_t792_1m -g
north=222504
south=219456
east=640081
west=637033
nsres=1
ewres=1
rows=3048
cols=3048
cells=9290304
datatype=CELL
ncats=188
}}}

Or is it irrelevant, and they are just set back to the default after the
script's completion, since the script uses `gscript.use_temp_region()`?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3685#comment:1>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 19, 2018, 9:34pm

#3685: pygrass GridModule: ERROR: Unable to open element file <windows>
--------------------------+-------------------------
  Reporter: mlennert | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.6.0
Component: PyGRASS | Version: svn-trunk
Resolution: | Keywords: GridModule
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------

Comment (by mmetz):

Replying to [ticket:3685 mlennert]:
> I've been using a script with code resembling the attached python script
for months (notably through the i.cutlines addon), without any error. It
uses the GridModule class from pygrass to run i.zc on tiles of an input
map (as i.zc reads everything into memory).
>
> I suddenly get the following errors, both with trunk and release76:
>
>
> {{{
> GRASS 7.7.svn (nc_spm_08):~ > ./grid_test.py
> Creating edge map
> ERROR: ERROR: ERROR: UnableUnableUnable toto openopen elementelement
filefile to open<windows> <windows>for elementfor file
>
> <tmp.grid_test.py.498@izc_000_002>
> <tmp.grid_test.py.498@izc_000_003>
> <windows> for
> <tmp.grid_test.py.498@izc_000_001>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_000_000>
> ERROR: ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_000_003>
> Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_000_001>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_000_000>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_000_002>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_001_001>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_001_002>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_001_000>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_001_003>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_001_001>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_001_002>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_001_000>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_001_003>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_002_000>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_002_001>
> ERROR: ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_002_003>
> Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_002_002>
> ERROR: ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_002_001>
> Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_002_000>
> ERROR: ERROR: UnableUnable toto openopen elementelement filefile
<windows><windows> forfor
>
>
<tmp.grid_test.py.498@izc_002_002><tmp.grid_test.py.498@izc_002_003>
>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_003_001>
> ERROR: ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_003_000>
> Unable to open element file <windows> for
> ERROR: <tmp.grid_test.py.498@izc_003_002>
> Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_003_003>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_003_001>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_003_003>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_003_000>
> ERROR: Unable to open element file <windows> for
> <tmp.grid_test.py.498@izc_003_002>
> Traceback (most recent call last):
> File "./grid_test.py", line 33, in <module>
> grd.run()
> File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-
gnu/etc/python/grass/pygrass/modules/grid/grid.py", line 568, in run
> self.patch()
> File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-
gnu/etc/python/grass/pygrass/modules/grid/grid.py", line 609, in patch
> self.start_row, self.start_col, self.out_prefix)
> File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-
gnu/etc/python/grass/pygrass/modules/grid/patch.py", line 79, in
rpatch_map
> rtype.open('r')
> File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-
gnu/etc/python/grass/pygrass/raster/__init__.py", line 210, in open
> raise OpenError(str_err)
> grass.exceptions.OpenError: The map does not exist, I can't open in 'r'
mode
> }}}
>
> Does anyone have any idea what might be going wrong ? When I look into
one of these izc_* directories, I see the following:
>
>
> {{{
> GRASS 7.6.svn (nc_spm_08):~ > ls -l GRASSDATA/nc_spm_08/izc_002_002/
> total 16
> -rw-r--r-- 1 mlennert mlennert 14 oct 19 11:20 CURGROUP
> -rw-r--r-- 1 mlennert mlennert 291 oct 19 11:20 SEARCH_PATH
> -rw-r--r-- 1 mlennert mlennert 81 oct 19 11:20 VAR
> -rw-r--r-- 1 mlennert mlennert 317 oct 19 11:20 WIND
> }}}
>
> Does anyone have any idea about what might be going on ? This keeps me
from launching quite long-running processes, so it would be great if I
could solve this rapidly. I have to admit I'm a bit lost even just trying
to understand what might be wrong and how I should debug this. Any hints
are more than welcome.

I think this is related to `gscript.use_temp_region()`. The error messages
are all related to a region named `tmp.grid_test.py.498`, but this region
does not exist in the subsequently created temporary mapset. This
temporary region is defined in the current mapset and only valid in the
current mapset. Other temporary mapsets created later on in the script can
not access this temporary region definition, thus the ERRORs searching for
`tmp.grid_test.py.498` in different mapsets.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3685#comment:2>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 20, 2018, 9:18am

#3685: pygrass GridModule: ERROR: Unable to open element file <windows>
--------------------------+-------------------------
  Reporter: mlennert | Owner: grass-dev@…
      Type: defect | Status: closed
  Priority: normal | Milestone: 7.6.0
Component: PyGRASS | Version: svn-trunk
Resolution: fixed | Keywords: GridModule
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------
Changes (by mlennert):

* status: new => closed
* resolution: => fixed

Comment:

Replying to [comment:2 mmetz]:
> I think this is related to `gscript.use_temp_region()`. The error
messages are all related to a region named `tmp.grid_test.py.498`, but
this region does not exist in the subsequently created temporary mapset.
This temporary region is defined in the current mapset and only valid in
the current mapset. Other temporary mapsets created later on in the script
can not access this temporary region definition, thus the ERRORs searching
for `tmp.grid_test.py.498` in different mapsets.

Yes, that's it. Thanks to both of you and sorry for the noise. I'm busy
with too many things in parallel and forgot that I had only added
use_temp_region() recently into this script.

Don't know if it is worth trying to fit GridModule with use_temp_region(),
so closing.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3685#comment:3>
GRASS GIS <https://grass.osgeo.org>