[GRASS-dev] Testing needed for v.buffer2

Rosen_Matev · September 9, 2008, 10:45am

Hello all,

My mentor Wolf and I are experiencing problems with the new buffer
module. For some certain input it works well on my system but doesn't
finish at all on Wolf's. We need some help to figure out what's
happening. Here are the details:
1) checkout https://svn.osgeo.org/grass/grass-addons/vector/v.buffer2
and compile the module
2) set DEBUG=3
3) run "v.buffer2 input=busroute11 output=busroute11_buf distance=31",
where busroute11 is a map from nc_spm
(distances < 31.3 are problematic, 32 is fine on Wolf's system)
4) say whether the module finishes ok, and send your debug output in
case it didn't finish
(on my machine it takes ~15 seconds for the module to finish
outputting the debug info)
Thanks for your time

Rosen Matev

martinl · September 9, 2008, 12:19pm

Hi,

2008/9/9 Росен Матев <r.matev@gmail.com>:

My mentor Wolf and I are experiencing problems with the new buffer
module. For some certain input it works well on my system but doesn't
finish at all on Wolf's. We need some help to figure out what's

[...]

hm strange, I cannot compile v.buffer2

gcc -I/home/landa/src/grass6_devel/dist.i686-pc-linux-gnu/include
-Wall -Werror-implicit-function-declaration -fno-common -Wextra
-Wunused -I/home/landa/usr/include -DPACKAGE=\""grassmods"\"
-I/home/landa/src/grass6_devel/dist.i686-pc-linux-gnu/include -o
OBJ.i686-pc-linux-gnu/e_intersect.o -c e_intersect.c
e_intersect.c: In function 'segment_intersection_2d_tol':
e_intersect.c:352: error: implicit declaration of function 'fmax'
e_intersect.c:352: warning: incompatible implicit declaration of
built-in function 'fmax'
e_intersect.c:369: error: implicit declaration of function 'fmin'
e_intersect.c:369: warning: incompatible implicit declaration of
built-in function 'fmin'

fmax and fmin should be declared in math.h (which is included) ...

Martin
--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa *

Glynn_Clements1 · September 10, 2008, 12:23am

Martin Landa wrote:

> My mentor Wolf and I are experiencing problems with the new buffer
> module. For some certain input it works well on my system but doesn't
> finish at all on Wolf's. We need some help to figure out what's

[...]

hm strange, I cannot compile v.buffer2

gcc -I/home/landa/src/grass6_devel/dist.i686-pc-linux-gnu/include
-Wall -Werror-implicit-function-declaration -fno-common -Wextra
-Wunused -I/home/landa/usr/include -DPACKAGE=\""grassmods"\"
-I/home/landa/src/grass6_devel/dist.i686-pc-linux-gnu/include -o
OBJ.i686-pc-linux-gnu/e_intersect.o -c e_intersect.c
e_intersect.c: In function 'segment_intersection_2d_tol':
e_intersect.c:352: error: implicit declaration of function 'fmax'
e_intersect.c:352: warning: incompatible implicit declaration of
built-in function 'fmax'
e_intersect.c:369: error: implicit declaration of function 'fmin'
e_intersect.c:369: warning: incompatible implicit declaration of
built-in function 'fmin'

fmax and fmin should be declared in math.h (which is included) ...

fmax/fmin are C99. You may need to use e.g. -D_ISOC99_SOURCE in order
for them to be available.

This should be fixed (i.e. replace with a macro) before the code is
moved into the main tree. We shouldn't require C99 unnecessarily (and
this is unnecessary).

--
Glynn Clements <glynn@gclements.plus.com>

Rosen_Matev · September 10, 2008, 9:01am

I've removed the C99 calls.

On Wed, Sep 10, 2008 at 3:23 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Martin Landa wrote:

> My mentor Wolf and I are experiencing problems with the new buffer
> module. For some certain input it works well on my system but doesn't
> finish at all on Wolf's. We need some help to figure out what's

[...]

hm strange, I cannot compile v.buffer2

gcc -I/home/landa/src/grass6_devel/dist.i686-pc-linux-gnu/include
-Wall -Werror-implicit-function-declaration -fno-common -Wextra
-Wunused -I/home/landa/usr/include -DPACKAGE=\""grassmods"\"
-I/home/landa/src/grass6_devel/dist.i686-pc-linux-gnu/include -o
OBJ.i686-pc-linux-gnu/e_intersect.o -c e_intersect.c
e_intersect.c: In function 'segment_intersection_2d_tol':
e_intersect.c:352: error: implicit declaration of function 'fmax'
e_intersect.c:352: warning: incompatible implicit declaration of
built-in function 'fmax'
e_intersect.c:369: error: implicit declaration of function 'fmin'
e_intersect.c:369: warning: incompatible implicit declaration of
built-in function 'fmin'

fmax and fmin should be declared in math.h (which is included) ...

fmax/fmin are C99. You may need to use e.g. -D_ISOC99_SOURCE in order
for them to be available.

This should be fixed (i.e. replace with a macro) before the code is
moved into the main tree. We shouldn't require C99 unnecessarily (and
this is unnecessary).

--
Glynn Clements <glynn@gclements.plus.com>

Rosen_Matev · September 10, 2008, 5:54pm

Hi,

Thanks for pointing out all that. Strangely, I don't get those
warnings here, probably command line switches... I think they are
fixed now.
I don't seem to have problems with centroids. Can you show me a map,
on which that occurs?
And again, please help with the busroute11 test, cause it's behaving weirdly.

Rosen Matev

On Wed, Sep 10, 2008 at 5:58 PM, Hamish <hamish_b@yahoo.com> wrote:

Hi,

Glad to see things coming together. Some observations:

#include <math.h> needs to go back into e_intersect.c for fabs()

main.c still has a call to fmin()

functions like Vect_point_buffer2() should have prototypes declared in
a local_proto.h file. (even if it is just temporary until it moves into
the vector libraries)

besides that I get these warnings which need to be addressed.

dgraph.c: In function 'find_all_intersections':
dgraph.c:184: warning: unused variable 'z2_'
dgraph.c:184: warning: unused variable 'z1_'
dgraph.c:184: warning: unused variable 'y2_'
dgraph.c:184: warning: unused variable 'x2_'
dgraph.c:184: warning: unused variable 'y1_'
dgraph.c:184: warning: unused variable 'x1_'
dgraph.c:183: warning: unused variable 'res2'
dgraph.c: In function 'pg_create_struct':
dgraph.c:294: warning: unused variable 'i'

e_intersect.c: In function 'segment_intersection_2d':
e_intersect.c:565: warning: type defaults to 'int' in declaration of 'DLEVEL'
e_intersect.c:568: warning: unused variable 'min_by'
e_intersect.c:568: warning: unused variable 'max_by'
e_intersect.c:568: warning: unused variable 'min_bx'
e_intersect.c:568: warning: unused variable 'max_bx'
e_intersect.c:567: warning: unused variable 'min_ay'
e_intersect.c:567: warning: unused variable 'max_ay'
e_intersect.c:567: warning: unused variable 'min_ax'
e_intersect.c:567: warning: unused variable 'max_ax'

main.c: In function 'db_CatValArray_get_value_di':
main.c:48: warning: unused variable 'val_d'
main.c: In function 'main':
main.c:231: warning: implicit declaration of function 'fmin'
main.c:231: warning: incompatible implicit declaration of built-in function 'fmin'
main.c:339: warning: incompatible implicit declaration of built-in function 'fmin'
main.c:346: warning: implicit declaration of function 'Vect_point_buffer2'
main.c:353: warning: implicit declaration of function 'Vect_line_buffer2'
main.c:393: warning: incompatible implicit declaration of built-in function 'fmin'
main.c:399: warning: implicit declaration of function 'Vect_area_buffer2'
main.c:124: warning: unused variable 'size_val_int'
main.c:112: warning: unused variable 'debug'
main.c:111: warning: unused variable 'dtmp'

vlib_buffer.c: In function 'angular_tolerance':
vlib_buffer.c:130: warning: unused variable 'b'
vlib_buffer.c: In function 'parallel_line':
vlib_buffer.c:159: warning: unused variable 'r'
vlib_buffer.c:159: warning: unused variable 'sina'
vlib_buffer.c:159: warning: unused variable 'cosa'
vlib_buffer.c: In function 'convolution_line':
vlib_buffer.c:310: warning: unused variable 'r'
vlib_buffer.c:310: warning: unused variable 'sina'
vlib_buffer.c:310: warning: unused variable 'cosa'
vlib_buffer.c:459:15: warning: "/*" within comment
vlib_buffer.c: In function 'extract_contour':
vlib_buffer.c:451: warning: unused variable 'i'
vlib_buffer.c: In function 'extract_outer_contour':
vlib_buffer.c:581: warning: unused variable 'ta'
vlib_buffer.c: In function 'Vect_area_buffer2':
vlib_buffer.c:911: warning: unused variable 'max'
vlib_buffer.c:908: warning: unused variable 'winding'
vlib_buffer.c:908: warning: unused variable 'res'

the unused variables are just noise, but too much noise can hide a signal.

I ran the module on the Spearfish test dataset roads layer, all seemed to
work well. I noticed that the resulting areas were missing centroids and
the number of lines shrunk considerably with v.build.polylines.

regards,
Hamish

Hamish · September 10, 2008, 10:56pm

Росен Матев wrote:

Thanks for pointing out all that. Strangely, I don't get those
warnings here, probably command line switches...

this is what I use:

CFLAGS="-g -Wall -Werror-implicit-function-declaration" ./configure \
...

For debugging do not use -O for optimization and do not strip the
binaries with LDFLAGS="-s".

I think they are fixed now.

thanks.
local_proto.h is missing from SVN.

rather than #define'ing MAX,MIN 4 times it is probably better to move
that into local_proto.h or so and include that in each .c file.

I don't seem to have problems with centroids. Can you show me a map,
on which that occurs?

they are all ok, just missing cat numbers.

spearfish test dataset, roads vector map:

GRASS64> v.buffer2 in=roads out=roads_30mbuff_try2 type=line dist=30
The tolerance in map units = 0.3
Lines buffers...
100%
Building parts of topology...
Building topology for vector map <roads_30mbuff_try2>...
822 primitives registered
Topology was built
Number of nodes : 822
Number of primitives: 822
Number of points : 0
Number of lines : 0
Number of boundaries: 822
Number of centroids : 0
Number of areas : -
Number of isles : -
Snapping boundaries...
All vertices : 35618
Registered points : 34788
Nodes marked as anchor : 34500
Nodes marked to be snapped: 288
Snapped vertices : 293
New vertices : 26
Breaking boundaries...
Intersections: 29912
Removing duplicates...
Duplicates: 13
Attaching islands...
Building topology for vector map <roads_30mbuff_try2>...
Building areas: 100%
14958 areas built
7 isles built
Attaching islands: 100%
Topology was built
Number of nodes : 16735
Number of primitives: 40014
Number of points : 0
Number of lines : 0
Number of boundaries: 40014
Number of centroids : 0
Number of areas : 14958
Number of isles : 7
Number of areas without centroid : 14958
Attaching centroids...
Building topology for vector map <roads_30mbuff_try2>...
Attaching centroids: 100%
Topology was built
Number of nodes : 16739
Number of primitives: 40018
Number of points : 0
Number of lines : 0
Number of boundaries: 40014
Number of centroids : 4
Number of areas : 20472
Number of isles : 216
Number of areas without centroid : 211
Rebuilding topology...
Building topology for vector map <roads_30mbuff_try2>...
1783 primitives registered
Building areas: 100%
215 areas built
215 isles built
Attaching islands: 100%
Attaching centroids: 100%
Topology was built
Number of nodes : 1783
Number of primitives: 1783
Number of points : 0
Number of lines : 0
Number of boundaries: 1779
Number of centroids : 4
Number of areas : 215
Number of isles : 215
Number of areas without centroid : 211

# don't know if this really matters; combine line segments:

GRASS64> v.build.polylines in=roads_30mbuff_try2 out=roads_30mbufftry2b
1779 lines or boundaries found in vector map <roads_30mbuff_try2@user2>
215 polylines stored to vector map <roads_30mbufftryb2user2>
Building topology for vector map <roads_30mbufftry2b>...
219 primitives registered
Building areas: 100%
215 areas built
215 isles built
Attaching islands: 100%
Attaching centroids: 100%
Topology was built
Number of nodes : 219
Number of primitives: 219
Number of points : 0
Number of lines : 0
Number of boundaries: 215
Number of centroids : 4
Number of areas : 215
Number of isles : 215
Number of areas without centroid : 211

# the areas without centroids above are really holes/islands.

d.mon x0
d.vect roads_30mbuff_try2b type=area
d.vect roads_30mbuff_try2b type=centroid width=3 color=red

The 4 centroids that are there are fine, but missing cat numbers

# add them with:
v.category in=roads_30mbuff_try2b out=roads_30mbuff_try2bC type=centroid

Without "type=centroid" above it adds new centroids to all holes,
due to default includsion of type=area

Your code should give all centroids cat numbers (but not boundaries),
most processing modules (v.extract, ...) do so by looping through cats
and will not work without them. A centroid's cat number (as thus DB
attributes) is inherited by the surrounding area.

And again, please help with the busroute11 test, cause
it's behaving weirdly.

right, will do.

Hamish

Hamish · September 12, 2008, 2:29am

Росен Матев wrote:

My mentor Wolf and I are experiencing problems with the new buffer
module. For some certain input it works well on my system but doesn't
finish at all on Wolf's. We need some help to figure out what's
happening. Here are the details:

1) checkout https://svn.osgeo.org/grass/grass-addons/vector/v.buffer2
and compile the module

now compiles cleanly- cheers!

2) set DEBUG=3

ok,
g.gisenv set="DEBUG=3"

3) run "v.buffer2 input=busroute11 output=busroute11_buf distance=31",
where busroute11 is a map from nc_spm
(distances < 31.3 are problematic, 32 is fine on Wolf's system)

I get the same, 31.4 and up are fine and take <1 sec to complete.
I notice this is when the sliver along parallel paths of Western Blvd.
first opens up.

v.buffer2 input=busroute11 output=busroute11_buf31 \
distance=31 2> br11_buf31.txt

[wait wait wait]
^C

4) say whether the module finishes ok, and send your debug output in
case it didn't finish

Less than 31 never completes. Log attached.

(on my machine it takes ~15 seconds for the module to finish
outputting the debug info)

It will be much faster if you redirect it to a file versus displaying it
on the terminal. Some terms like gnome-terminal are very slow. xterm and
rxvt will be much faster but still not as fast as writing to a file.
v.buffer2 ... 2> logfile.txt

Use "tee" if you want to see it both in the terminal & saved to a file:
v.buffer2 ... 2>&1 | tee logfile.txt

I was curious to see where it was getting stuck, and so I threw it into
gdb (by way of kdbg):

kdbg `which v.buffer2`
Execution -> Arguments -> "input=busroute11 output=..."
Run
whiz bang whirl ... freeze
Execution -> Break
Execution -> Step Into
.... link_dispose() in lib/linkm/dispose.c
Execution -> Step out
.... destroy_links() in lib/vector/Vlib/poly.c

and if you step through from there it is stuck in the destroy_links()
loop.

if you go to View->Locals and expand the *tmp struct you will see as
you step along that *p and *tmp just bounce around in an endless circle
between two memory valid addresses, so it never reaches NULL.

The x value for those two are 638631.01569480076 and 637951.01569479937

Hamish

(attachments)

br11_buf31.txt.gz (13 KB)

Rosen_Matev · September 12, 2008, 12:36pm

Hi all,

First of all, thanks Hamish for the helpful responses.
I was looking into the lib/linkm sources and I'm confused. Either the
library is pretty messy or I'm overlooking something. This linked list
manager should be rewritten. Luckily it is used in quite a few places
in GRASS. Please take a look at list_new() [1] and link__get_next(),
link__set_next() [2]. And then see how the list is used in [3].

[1] http://download.osgeo.org/grass/grass6_progman/new_8c-source.html#l00012
[2] http://download.osgeo.org/grass/grass6_progman/linkm_2next_8c-source.html
[3] http://download.osgeo.org/grass/grass6_progman/Vlib_2poly_8c-source.html#l00180

Regards,
Rosen Matev

On Fri, Sep 12, 2008 at 5:29 AM, Hamish <hamish_b@yahoo.com> wrote:

Росен Матев wrote:

My mentor Wolf and I are experiencing problems with the new buffer
module. For some certain input it works well on my system but doesn't
finish at all on Wolf's. We need some help to figure out what's
happening. Here are the details:

1) checkout https://svn.osgeo.org/grass/grass-addons/vector/v.buffer2
and compile the module

now compiles cleanly- cheers!

2) set DEBUG=3

ok,
g.gisenv set="DEBUG=3"

3) run "v.buffer2 input=busroute11 output=busroute11_buf distance=31",
where busroute11 is a map from nc_spm
(distances < 31.3 are problematic, 32 is fine on Wolf's system)

I get the same, 31.4 and up are fine and take <1 sec to complete.
I notice this is when the sliver along parallel paths of Western Blvd.
first opens up.

v.buffer2 input=busroute11 output=busroute11_buf31 \
distance=31 2> br11_buf31.txt

[wait wait wait]
^C

4) say whether the module finishes ok, and send your debug output in
case it didn't finish

Less than 31 never completes. Log attached.

(on my machine it takes ~15 seconds for the module to finish
outputting the debug info)

It will be much faster if you redirect it to a file versus displaying it
on the terminal. Some terms like gnome-terminal are very slow. xterm and
rxvt will be much faster but still not as fast as writing to a file.
v.buffer2 ... 2> logfile.txt

Use "tee" if you want to see it both in the terminal & saved to a file:
v.buffer2 ... 2>&1 | tee logfile.txt

I was curious to see where it was getting stuck, and so I threw it into
gdb (by way of kdbg):

kdbg `which v.buffer2`
Execution -> Arguments -> "input=busroute11 output=..."
Run
whiz bang whirl ... freeze
Execution -> Break
Execution -> Step Into
.... link_dispose() in lib/linkm/dispose.c
Execution -> Step out
.... destroy_links() in lib/vector/Vlib/poly.c

and if you step through from there it is stuck in the destroy_links()
loop.

if you go to View->Locals and expand the *tmp struct you will see as
you step along that *p and *tmp just bounce around in an endless circle
between two memory valid addresses, so it never reaches NULL.

The x value for those two are 638631.01569480076 and 637951.01569479937

Hamish

Glynn_Clements1 · September 13, 2008, 12:27am

Росен Матев wrote:

First of all, thanks Hamish for the helpful responses.
I was looking into the lib/linkm sources and I'm confused. Either the
library is pretty messy or I'm overlooking something.

The former.

This linked list manager should be rewritten.

Agreed.

--
Glynn Clements <glynn@gclements.plus.com>