[GRASS-dev] v.what.rast: topology really necessary?

Blumentrath_Stefan1 · January 10, 2017, 10:09am

Hi again,

Is there any reason that v.what.rast requires topology level 2? “r.what” works without topology…

Cheers

Stefan

martinl · January 10, 2017, 11:40am

Hi Stefan,

2017-01-10 11:09 GMT+01:00 Blumentrath, Stefan <Stefan.Blumentrath@nina.no>:

Is there any reason that v.what.rast requires topology level 2? “r.what”
works without topology…

AFAIU, topology is not required. I change that in r70331 (trunk).
After some testing could be backported to relbr72. Ma

--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa

Blumentrath_Stefan1 · January 10, 2017, 12:18pm

Hi Martin,

Thanks for your swift relpy. Unfortunately, without topology, v.what.rast segfaults in r70332.

v.external --overwrite input=PG:dbname=gisdata layer=points output=points_pg73ot -bo
v.what.rast map=points_pg73ot raster=rand column=morphology
Reading features from vector map...
Segmentation fault (core dumped)

Ciao,
Stefan

-----Original Message-----
From: Martin Landa [mailto:landa.martin@gmail.com]
Sent: tirsdag 10. januar 2017 12.40
To: Blumentrath, Stefan <Stefan.Blumentrath@nina.no>
Cc: GRASS developers list (grass-dev@lists.osgeo.org) <grass-dev@lists.osgeo.org>
Subject: Re: [GRASS-dev] v.what.rast: topology really necessary?

Hi Stefan,

2017-01-10 11:09 GMT+01:00 Blumentrath, Stefan <Stefan.Blumentrath@nina.no>:

Is there any reason that v.what.rast requires topology level 2? “r.what”
works without topology…

AFAIU, topology is not required. I change that in r70331 (trunk).
After some testing could be backported to relbr72. Ma

--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa

martinl · January 10, 2017, 2:38pm

Hi,

2017-01-10 13:18 GMT+01:00 Blumentrath, Stefan <Stefan.Blumentrath@nina.no>:

Thanks for your swift relpy. Unfortunately, without topology, v.what.rast segfaults in r70332.

v.external --overwrite input=PG:dbname=gisdata layer=points output=points_pg73ot -bo
v.what.rast map=points_pg73ot raster=rand column=morphology
Reading features from vector map...
Segmentation fault (core dumped)

hm, I can confirm it. Unfortunately it's something hidden. It segfaults at

#4 0x00007ffff7ba5d5d in get_feature (Map=0x7fffffffbed0, fid=-1,
type=-1) at read_pg.c:614
#5 0x00007ffff7ba5829 in read_next_line_pg (Map=0x7fffffffbed0,
line_p=0x63b3e0, line_c=0x60ad70, ignore_constraints=0) at
read_pg.c:468

when I comment out this line, it segfauls on completely different place:

#7 0x00007ffff729e187 in G__calloc (file=0x7ffff72e21a0
"lib/gis/parser.c", line=658, m=1024, n=1) at alloc.c:81
#8 0x00007ffff72c243b in recreate_command (original_path=0) at parser.c:658
#9 0x00007ffff72c2b77 in G_recreate_command () at parser.c:802

It seems to be memory issue. Please report an issue in trac. Ma

--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa

Blumentrath_Stefan1 · January 10, 2017, 9:49pm

Hei again,

And sorry for the numerous emails...

After repeated testing, a colleague of mine pointed out that I know occupy almost all available PostGIS connections on our server.
It seems that connections hang from testing and that either v.external or v.what.rast does not close the PG connection properly after the module finishes... Or are there other possible explanations?

Where should I look to find the source of the issue?

Ciao,
Stefan

-----Original Message-----
From: Martin Landa [mailto:landa.martin@gmail.com]
Sent: tirsdag 10. januar 2017 15.39
To: Blumentrath, Stefan <Stefan.Blumentrath@nina.no>
Cc: GRASS developers list (grass-dev@lists.osgeo.org) <grass-dev@lists.osgeo.org>
Subject: Re: [GRASS-dev] v.what.rast: topology really necessary?

Hi,

2017-01-10 13:18 GMT+01:00 Blumentrath, Stefan <Stefan.Blumentrath@nina.no>:

Thanks for your swift relpy. Unfortunately, without topology, v.what.rast segfaults in r70332.

v.external --overwrite input=PG:dbname=gisdata layer=points
output=points_pg73ot -bo v.what.rast map=points_pg73ot raster=rand
column=morphology Reading features from vector map...
Segmentation fault (core dumped)

hm, I can confirm it. Unfortunately it's something hidden. It segfaults at

#4 0x00007ffff7ba5d5d in get_feature (Map=0x7fffffffbed0, fid=-1,
type=-1) at read_pg.c:614
#5 0x00007ffff7ba5829 in read_next_line_pg (Map=0x7fffffffbed0, line_p=0x63b3e0, line_c=0x60ad70, ignore_constraints=0) at
read_pg.c:468

when I comment out this line, it segfauls on completely different place:

#7 0x00007ffff729e187 in G__calloc (file=0x7ffff72e21a0 "lib/gis/parser.c", line=658, m=1024, n=1) at alloc.c:81
#8 0x00007ffff72c243b in recreate_command (original_path=0) at parser.c:658
#9 0x00007ffff72c2b77 in G_recreate_command () at parser.c:802

It seems to be memory issue. Please report an issue in trac. Ma

--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa

martinl · January 11, 2017, 10:26am

Hi Stefan,

2017-01-10 22:49 GMT+01:00 Blumentrath, Stefan <Stefan.Blumentrath@nina.no>:

After repeated testing, a colleague of mine pointed out that I know occupy almost all available PostGIS connections on our server.
It seems that connections hang from testing and that either v.external or v.what.rast does not close the PG connection properly after the module finishes... Or are there other possible explanations?

I would guess that the command which segfauls could be a problem. I
tested all the commands including v.what.rast which segfaults and
check number of active connection via pg_top. No connection hanged
even I launched the command with segfault.

Where should I look to find the source of the issue?

Install pgtop on the server, login as postgres user and run pg_top
command. It will show you number of active connection. Run GRASS
command and check number of connection.

Martin

--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa

Blumentrath_Stefan1 · January 11, 2017, 10:38am

Hi Martin,

Thanks for the hint. Will do so...

Cheers
Stefan

-----Original Message-----
From: Martin Landa [mailto:landa.martin@gmail.com]
Sent: onsdag 11. januar 2017 11.26
To: Blumentrath, Stefan <Stefan.Blumentrath@nina.no>
Cc: GRASS developers list (grass-dev@lists.osgeo.org) <grass-dev@lists.osgeo.org>
Subject: Re: [GRASS-dev] v.what.rast: topology really necessary?

Hi Stefan,

2017-01-10 22:49 GMT+01:00 Blumentrath, Stefan <Stefan.Blumentrath@nina.no>:

After repeated testing, a colleague of mine pointed out that I know occupy almost all available PostGIS connections on our server.
It seems that connections hang from testing and that either v.external or v.what.rast does not close the PG connection properly after the module finishes... Or are there other possible explanations?

I would guess that the command which segfauls could be a problem. I tested all the commands including v.what.rast which segfaults and check number of active connection via pg_top. No connection hanged even I launched the command with segfault.

Where should I look to find the source of the issue?

Install pgtop on the server, login as postgres user and run pg_top command. It will show you number of active connection. Run GRASS command and check number of connection.

Martin

--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa

martinl · January 14, 2017, 4:05pm

Hi Stefan,

2017-01-11 11:38 GMT+01:00 Blumentrath, Stefan <Stefan.Blumentrath@nina.no>:

could you try patch for v.what.rast [1]? Ma

[1] https://trac.osgeo.org/grass/ticket/3249#comment:13

--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa

Blumentrath_Stefan1 · January 14, 2017, 9:38pm

Hi Martin,

And thanks so much for your effort!

I just patched r70332 and tested in NC, where it works perfectly fine with:

v.random --overwrite --verbose output=random_topo npoints=5000
v.db.addtable map=random_topo columns="elev double precision"

v.out.ascii -c --overwrite --verbose input=random_topo type=point output=$HOME/test.asc columns=elev format=point
v.in.ascii -b --overwrite --verbose input=$HOME/test.asc output=random_notopo skip=1

time v.what.rast --verbose map=random_notopo raster=elev_ned_30m@PERMANENT column=elev
time v.what.rast --verbose map=random_topo raster=elev_ned_30m@PERMANENT column=elev

BTW. during testing I noticed that v.db.addtable also requires topology level 2 (therefore I exported to ASCII and imported without topology)...

Again, thanks so much! This can provide a quite efficient workflow for my colleagues with GPS-tracking data (points) in PostGIS...

Cheers
Stefan

-----Original Message-----
From: Martin Landa [mailto:landa.martin@gmail.com]
Sent: lørdag 14. januar 2017 17.06
To: Blumentrath, Stefan <Stefan.Blumentrath@nina.no>
Cc: GRASS developers list (grass-dev@lists.osgeo.org) <grass-dev@lists.osgeo.org>
Subject: Re: [GRASS-dev] v.what.rast: topology really necessary?

Hi Stefan,

2017-01-11 11:38 GMT+01:00 Blumentrath, Stefan <Stefan.Blumentrath@nina.no>:

could you try patch for v.what.rast [1]? Ma

[1] https://trac.osgeo.org/grass/ticket/3249#comment:13

--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa

Blumentrath_Stefan1 · January 14, 2017, 9:49pm

Forgot to ask: could this go into the release branch none the less (you recently changed milestone to 7.4)?

-----Original Message-----
From: Martin Landa [mailto:landa.martin@gmail.com]
Sent: lørdag 14. januar 2017 17.06
To: Blumentrath, Stefan <Stefan.Blumentrath@nina.no>
Cc: GRASS developers list (grass-dev@lists.osgeo.org) <grass-dev@lists.osgeo.org>
Subject: Re: [GRASS-dev] v.what.rast: topology really necessary?

Hi Stefan,

2017-01-11 11:38 GMT+01:00 Blumentrath, Stefan <Stefan.Blumentrath@nina.no>:

could you try patch for v.what.rast [1]? Ma

[1] https://trac.osgeo.org/grass/ticket/3249#comment:13

--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa

martinl · January 15, 2017, 9:47am

Hi,

2017-01-14 22:49 GMT+01:00 Blumentrath, Stefan <Stefan.Blumentrath@nina.no>:

Forgot to ask: could this go into the release branch none the less (you recently changed milestone to 7.4)?

I hope it could go into relbr72 (so 7.2.1). Ma

--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa

martinl · January 17, 2017, 12:55pm

Hi,

2017-01-15 10:47 GMT+01:00 Martin Landa <landa.martin@gmail.com>:

Forgot to ask: could this go into the release branch none the less (you recently changed milestone to 7.4)?

I hope it could go into relbr72 (so 7.2.1). Ma

committed in trunk, see https://trac.osgeo.org/grass/ticket/3249#comment:16

Ma

--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa

martinl · January 24, 2017, 3:40pm

Hi,

2017-01-15 10:47 GMT+01:00 Martin Landa <landa.martin@gmail.com>:

Forgot to ask: could this go into the release branch none the less (you recently changed milestone to 7.4)?

I hope it could go into relbr72 (so 7.2.1). Ma

FYI, backport included in relbr72 (7.2.1) [1]. Ma

[1] https://trac.osgeo.org/grass/ticket/3249#comment:18

--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa