[GRASS-dev] [GRASS GIS] #3593: Error in v.stream.order add-on, segmentation fault

#3593: Error in v.stream.order add-on, segmentation fault
-------------------------+---------------------------------
Reporter: jradinger | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone:
Component: Addons | Version: svn-releasebranch74
Keywords: | CPU: Unspecified
Platform: Unspecified |
-------------------------+---------------------------------
The `v.stream.order` add-on in GRASS 7.4.1 (freshly installed, bundled
version from [http://grassmac.wikidot.com/downloads\]) on MacOS 10.13.5
does not work, but causes an error: segmentation fault 11. The last time I
used this add-on was, I think with GRASS 7.3 (where it was working).

My current setup:
GRASS version: 7.4.1
GRASS SVN revision: r72807M
Build date: 2018-06-13
Build platform: x86_64-apple-darwin17.6.0
GDAL: 2.0.0
PROJ.4: 5.1.0
GEOS: 3.6.2
SQLite: 3.19.3
Python: 2.7.15
wxPython: 4.0.0
Platform: Darwin-17.6.0-x86_64-i386-64bit

I tried the example from the manual of v.stream.order that uses the North
Carolina example dataset
([https://grass.osgeo.org/grass74/manuals/addons/v.stream.order.html\]):

{{{
# We need to generate 4 outlet points that will lead to
# 4 stream networks and therefore 4 output vector maps
cat > points.csv << EOF
640781.56098|214897.033189
642228.347134|214979.370612
638470.926725|214984.99142
645247.580918|223346.644849
EOF

v.in.ascii output=streams_outlets input=points.csv x=1 y=2

v.stream.order input=streams@PERMANENT points=streams_outlets \
     output=streams_order threshold=25 order=strahler,shreve
}}}

and I receive following error (with debug=3):
{{{
<s7):~ > v.stream.order input=streams@PERMANENT points=streams_outlets \
> output=streams_order threshold=25 order=strahler,shreve
D2/3: filename =
/Users/Johannes_Radinger/Library/GRASS/7.4/Modules/scripts/v.stream.order
D1/3: G_set_program_name(): v.stream.order
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order/vector/streams_order
D1/3: G_find_vector2(): name=streams mapset=PERMANENT
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams
D1/3: G_find_vector2(): name=streams mapset=PERMANENT
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams
D1/3: Vect__open_old(): name = streams, mapset = PERMANENT, layer = 1,
update = 0, head_only = 0, is_tmp = 0
D1/3: Vect_set_thresh(): thresh = 0.000000
D3/3: dig_init_plus()
D1/3: dig_spidx_init(), 2 dims
D3/3: dig_cidx_init()
D1/3: G_find_vector2(): name=streams mapset=PERMANENT
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams
D1/3: Map: name = streams, mapset = PERMANENT, temporary = 0
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/frmt
D1/3: Vector format: 0 (native)
D1/3: Vect_set_thresh(): thresh = 0.000000
D1/3: Vect__read_head(): vector = streams@PERMANENT
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/head
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/head
D2/3: file open: read (mode = r)
D1/3: Vect_set_thresh(): thresh = 0.000000
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order/WIND
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order/WIND
D2/3: file open: read (mode = r)
D2/3: G__read_Cell_head
D2/3: G__read_Cell_head_array
D3/3: region item: proj: 99
D3/3: region item: zone: 0
D3/3: region item: north: 221230
D3/3: region item: south: 219580
D3/3: region item: east: 639530
D3/3: region item: west: 637740
D3/3: region item: cols: 179
D3/3: region item: rows: 165
D3/3: region item: e-w resol: 10
D3/3: region item: n-s resol: 10
D3/3: region item: top: 1
D3/3: region item: bottom: 0
D3/3: region item: cols3: 179
D3/3: region item: rows3: 165
D3/3: region item: depths: 1
D3/3: region item: e-w resol3: 10
D3/3: region item: n-s resol3: 10
D3/3: region item: t-b resol: 1
D1/3: Level request = 2
D1/3: Vect_open_topo(): name = streams mapset = PERMANENT
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/topo
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/topo
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/topo
D2/3: file open: read (mode = r)
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/coor
D1/3: get coor info: /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/coor
D1/3: Vect_coor_info(): Info->size = 2747992, Info->mtime = 1198040776
D2/3: Topo header: file version 5.1 , supported from GRASS version 5.1
D2/3: byte order 0
D2/3: header size 142
D2/3: topo off_t size = 4
D2/3: with_z 0
D2/3: coor size 2747992
D1/3: Topo head: coor size = 2747992, coor mtime = 0
D1/3: dig_load_plus()
D2/3: dig_free_plus()
D2/3: dig_free_plus_nodes()
D2/3: dig_free_plus_lines()
D2/3: dig_free_plus_areas()
D2/3: dig_free_plus_isles()
D2/3: dig_cidx_free()
D3/3: dig_init_plus()
D1/3: dig_spidx_init(), 2 dims
D3/3: dig_cidx_init()
D2/3: Topo header: file version 5.1 , supported from GRASS version 5.1
D2/3: byte order 0
D2/3: header size 142
D2/3: topo off_t size = 4
D2/3: with_z 0
D2/3: coor size 2747992
D1/3: Vect_open_sidx(): name = streams mapset= PERMANENT mode = old
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/sidx
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/sidx
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/sidx
D2/3: file open: read (mode = r)
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/coor
D1/3: get coor info: /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/coor
D1/3: Vect_coor_info(): Info->size = 2747992, Info->mtime = 1198040776
D1/3: dig_spidx_init(), 2 dims
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order
D2/3: G__temp_element(): .tmp/eco4.udg.edu (tmp=0)
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order/.tmp/eco4.udg.edu/76467.0
D2/3: G_tempfile_pid(): /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order/.tmp/eco4.udg.edu/76467.0
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order
D2/3: G__temp_element(): .tmp/eco4.udg.edu (tmp=0)
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order/.tmp/eco4.udg.edu/76467.1
D2/3: G_tempfile_pid(): /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order/.tmp/eco4.udg.edu/76467.1
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order
D2/3: G__temp_element(): .tmp/eco4.udg.edu (tmp=0)
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order/.tmp/eco4.udg.edu/76467.2
D2/3: G_tempfile_pid(): /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order/.tmp/eco4.udg.edu/76467.2
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order
D2/3: G__temp_element(): .tmp/eco4.udg.edu (tmp=0)
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order/.tmp/eco4.udg.edu/76467.3
D2/3: G_tempfile_pid(): /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/test_stream_order/.tmp/eco4.udg.edu/76467.3
D2/3: Spidx header: file version 5.1 , supported from GRASS version 5.1
D2/3: byte order 0
D2/3: header size 113
D2/3: with_z 0
D2/3: coor size 2747992
D1/3: Sidx head: coor size = 2747992, coor mtime = 0
D2/3: Vect_cidx_open(): name = streams mapset= PERMANENT
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/cidx
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/cidx
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/cidx
D2/3: file open: read (mode = r)
D3/3: dig_read_cidx()
D2/3: dig_cidx_free()
D3/3: dig_cidx_init()
D3/3: Cidx header: file version 5.0 , supported from GRASS version 5.0
D3/3: byte order 0
D3/3: header size 9
D1/3: V1_open_old_nat(): name = streams mapset = PERMANENT
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/coor
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/coor
D2/3: file open: read (mode = r)
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/coor
D1/3: get coor info: /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/coor
D1/3: Vect_coor_info(): Info->size = 2747992, Info->mtime = 1198040776
D2/3: dig__read_head(): name = 'streams'
D2/3: Coor header: file version 5.1 , supported from GRASS version 5.1
D2/3: byte order 0
D2/3: header size 14
D2/3: with_z 0
D2/3: coor size 2747992
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/coor
D1/3: get coor info: /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/coor
D1/3: Vect_coor_info(): Info->size = 2747992, Info->mtime = 1198040776
D1/3: coor size in head = 2747992, real coor file size= 2747992
D2/3: dig_file_load ()
D2/3: requested mode = 2
D2/3: size = 2747992
D2/3: file was not loaded to the memory
D1/3: Vect__open_old(): vector opened on level 2
D1/3: Vect_read_dblinks(): map = streams, mapset = PERMANENT
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/dbln
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/dbln
D2/3: file open: read (mode = r)
D1/3: dbln:
1/streams|streams|cat|$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite/sqlite.db|sqlite
D3/3: Field number <1>, name <streams>
D3/3: Vect_check_dblink: field 1, name streams
D1/3: field = 1 name = streams, table = streams, key = cat, database =
$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite/sqlite.db, driver = sqlite
D1/3: Dblinks read
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/hist
D2/3: G_file_name(): path = /Users/Johannes_Radinger/Documents/GRASS-
GIS/nc_spm_08_grass7/PERMANENT/vector/streams/hist
D2/3: file open: read (mode = r)
D1/3: Vect_Rewind(): name = streams level = 2
Segmentation fault: 11
}}}

Any ideas?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3593&gt;
GRASS GIS <https://grass.osgeo.org>

#3593: Error in v.stream.order add-on, segmentation fault
--------------------------+---------------------------------
  Reporter: jradinger | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Addons | Version: svn-releasebranch74
Resolution: | Keywords:
       CPU: Unspecified | Platform: Unspecified
--------------------------+---------------------------------

Comment (by jradinger):

This issue still persists. I've tested now also v.stream.order with GRASS
7.5 (bundled version for MacOS) (SVN r73007M, built 26.07.18) and also get
this error related to 'segmentation fault 11'. Any suggestions how to
proceed?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3593#comment:1&gt;
GRASS GIS <https://grass.osgeo.org>

#3593: Error in v.stream.order add-on, segmentation fault
--------------------------+---------------------------------
  Reporter: jradinger | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Addons | Version: svn-releasebranch74
Resolution: | Keywords:
       CPU: Unspecified | Platform: Unspecified
--------------------------+---------------------------------

Comment (by neteler):

Since it is a Python script: which step in it leads to sig11?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3593#comment:2&gt;
GRASS GIS <https://grass.osgeo.org>

#3593: Error in v.stream.order add-on, segmentation fault
--------------------------+---------------------------------
  Reporter: jradinger | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Addons | Version: svn-releasebranch74
Resolution: | Keywords:
       CPU: Unspecified | Platform: Unspecified
--------------------------+---------------------------------

Comment (by jradinger):

How can I find out that step/line? Do I need to run the script manually or
can that be obtained by a specific debug level?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3593#comment:3&gt;
GRASS GIS <https://grass.osgeo.org>

#3593: Error in v.stream.order add-on, segmentation fault
--------------------------+---------------------------------
  Reporter: jradinger | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Addons | Version: svn-releasebranch74
Resolution: | Keywords:
       CPU: Unspecified | Platform: Unspecified
--------------------------+---------------------------------

Comment (by neteler):

Replying to [comment:3 jradinger]:
> How can I find out that step/line? Do I need to run the script manually
or can that be obtained by a specific debug level?

The easiest way is to use pdb, see

https://grasswiki.osgeo.org/wiki/GRASS_Debugging#Python_debugging_with_pdb

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3593#comment:4&gt;
GRASS GIS <https://grass.osgeo.org>

#3593: Error in v.stream.order add-on, segmentation fault
--------------------------+---------------------------------
  Reporter: jradinger | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Addons | Version: svn-releasebranch74
Resolution: | Keywords:
       CPU: Unspecified | Platform: Unspecified
--------------------------+---------------------------------

Comment (by mmetz):

Replying to [comment:4 neteler]:
> Replying to [comment:3 jradinger]:
> > How can I find out that step/line? Do I need to run the script
manually or can that be obtained by a specific debug level?
>
> The easiest way is to use pdb, see
>
>
https://grasswiki.osgeo.org/wiki/GRASS_Debugging#Python_debugging_with_pdb

You can also try valgrind:
{{{
CMD="v.stream.order input=streams points=streams_outlets
output=streams_order threshold=25 order=strahler,shreve --o"
valgrind --tool=memcheck --trace-children=yes --track-origins=yes --log-
file=val.out.%p $CMD
}}}
This will create a number of val.out.* files which you could zip/tar.gz
and attach to this ticket. No segfault and nothing suspicious in the
valgrind output on my system.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3593#comment:5&gt;
GRASS GIS <https://grass.osgeo.org>

#3593: Error in v.stream.order add-on, segmentation fault
--------------------------+---------------------------------
  Reporter: jradinger | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Addons | Version: svn-releasebranch74
Resolution: | Keywords:
       CPU: Unspecified | Platform: Unspecified
--------------------------+---------------------------------

Comment (by jradinger):

I've not yet checked with `valgrind` or `pdb`, but I might have identified
the line that might cause the problem (L390,
https://trac.osgeo.org/grass/browser/grass-
addons/grass7/vector/v.stream.order/v.stream.order.py#L390): This is the
command `v.open(mode="r")`, where `v=VectorTopo('streams',
'test_stream_order')`.

I am not sure if this is a valid way to test but I ran the `v.open()`
command manually within python (within a GRASS session):
{{{
from grass.script import core as grass
from grass.pygrass.vector import VectorTopo
v = VectorTopo('streams','PERMANENT')
v.open(mode="r")
v.close()
}}}

This causes (not sure if this is a valid approach/related):
{{{
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File
"/Applications/GRASS-7.4.1.app/Contents/Resources/etc/python/grass/pygrass/vector/__init__.py",
line 638, in close
     super(VectorTopo, self).close(build=build)
   File
"/Applications/GRASS-7.4.1.app/Contents/Resources/etc/python/grass/pygrass/vector/abstract.py",
line 402, in close
     if self.is_open():
   File
"/Applications/GRASS-7.4.1.app/Contents/Resources/etc/python/grass/pygrass/vector/abstract.py",
line 283, in is_open
     return is_open(self.c_mapinfo)
   File
"/Applications/GRASS-7.4.1.app/Contents/Resources/etc/python/grass/pygrass/vector/abstract.py",
line 21, in is_open
     return (c_mapinfo.contents.open != 0 and
AttributeError: 'struct_Map_info' object has no attribute 'open'
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3593#comment:6&gt;
GRASS GIS <https://grass.osgeo.org>

#3593: Error in v.stream.order add-on, segmentation fault
--------------------------+---------------------------------
  Reporter: jradinger | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Addons | Version: svn-releasebranch74
Resolution: | Keywords:
       CPU: Unspecified | Platform: Unspecified
--------------------------+---------------------------------

Comment (by mmetz):

Replying to [comment:6 jradinger]:
> I've not yet checked with `valgrind` or `pdb`, but I might have
identified the line that might cause the problem (L390,
https://trac.osgeo.org/grass/browser/grass-
addons/grass7/vector/v.stream.order/v.stream.order.py#L390): This is the
command `v.open(mode="r")`, where `v=VectorTopo('streams',
'test_stream_order')`.
>
> I am not sure if this is a valid way to test but I ran the `v.open()`
command manually within python (within a GRASS session):
> {{{
> from grass.script import core as grass
> from grass.pygrass.vector import VectorTopo
> v = VectorTopo('streams','PERMANENT')
> v.open(mode="r")
> v.close()
> }}}
>
> This causes (not sure if this is a valid approach/related):
> {{{
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File
"/Applications/GRASS-7.4.1.app/Contents/Resources/etc/python/grass/pygrass/vector/__init__.py",
line 638, in close
> super(VectorTopo, self).close(build=build)
> File
"/Applications/GRASS-7.4.1.app/Contents/Resources/etc/python/grass/pygrass/vector/abstract.py",
line 402, in close
> if self.is_open():
> File
"/Applications/GRASS-7.4.1.app/Contents/Resources/etc/python/grass/pygrass/vector/abstract.py",
line 283, in is_open
> return is_open(self.c_mapinfo)
> File
"/Applications/GRASS-7.4.1.app/Contents/Resources/etc/python/grass/pygrass/vector/abstract.py",
line 21, in is_open
> return (c_mapinfo.contents.open != 0 and
> AttributeError: 'struct_Map_info' object has no attribute 'open'
> }}}

This does not make sense because the struct Map_info does have an
attribute open:

https://trac.osgeo.org/grass/browser/grass/trunk/include/vect/dig_structs.h#L1296

However, I found a problem in pygrass:

https://trac.osgeo.org/grass/browser/grass/trunk/lib/python/pygrass/vector/abstract.py#L77

self.c_mapinfo must be a struct, not a pointer to a struct. This can cause
a segfault because it seems that the memory for self.c_mapinfo has not
been allocated.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3593#comment:7&gt;
GRASS GIS <https://grass.osgeo.org>