[GRASS-user] Checking and fixing roads for network analysis

Mark_Wynter · January 15, 2015, 9:40pm

Hi Daniel
When you ran v.clean, did you v.clean tool=break,rmdupl ?

I suspect you still may encounter a problem with road T junctions if the vectors don't cross or intersect. Take this simple example of 2 roads - NS, and EW. Each line has only 2 nodes - at the line’s start and end points.
|—
The problem is when the horizontal line “undershoots” the vertical line. The break command will have no effect, and snapping won’t make any difference because there is no midpoint node to which the horizontal line can attach itself to.

I’d be interested to hear how others have dealt with what is essentially a “data preparation” issue in GRASS?

[The solution I ended up implementing used a PostGIS SQL custom function to “break” overshoots and undershoots, with rules for correct treatment of bridges and tunnels. The function is fast, and can be run in parallel when working with continental-sized road networks.
After PostGIS, I import the “broken-up” network into GRASS and use v.clean to snap the nodes, but NOT break the lines - as my PostGIS function has already dealt with this.
The GRASS procedure involves running the v.clean in 5 steps, to avoid inadvertently snapping a tunnel node to say a bridge node, just because they are spatially close.
1) v.clean tool=snap # no bridges, no tunnels
2) v.clean tool=snap # bridges only
3) v.clean tool=snap # tunnels only
4) v.patch (steps 1-3)
5) v.clean tool=snap # on the patched layer, with a much smaller tolerance to that used in steps 1-3 to again avoid say tunnel nodes inadvertently snapping to bridge nodes.
We’re then good to go with v.net analysis.]

This may sound like a complicated process, but when using imperfect data to model complex road networks in dense urban areas, its important to get the vector topology as close as possible to the real world. I’ve found this process generates a highly representative, routable vector topology from imperfect OSM data.

Your situation might be much simpler, which is not to say you won’t encounter anomalies like you’ve already identified.

Mark

----------------------------------------------------------------------

Message: 1
Date: Thu, 15 Jan 2015 15:22:27 -0200
From: Daniel Victoria <daniel.victoria@gmail.com>
To: grass <grass-user@lists.osgeo.org>
Subject: [GRASS-user] Checking and fixing roads for network analysis
Message-ID:
<CA+irsJjeKZQ9EqQ7igZcgL92k7uMfdKUJv6A-KVbtP7wn-XqYQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi List,

I'll need to perform some network analysis and right now I'm trying to see
if my roads map is all good. I know that there are some clearly isolated
areas on my roads network but I'd like to identify and fix the less obvious
ones.

Here is what I did:
1) Import road lines (v.in.ogr)
2) Clean the topology with snap tool, thresh=0.0001 (it's a latlong
location)
2) Create network (v.net using the nodes option --not sure if that was the
correct option)
3) Ran v.net.components to find the weakly connected components, not
setting any cost columns
4) Color roads lines according to the component number

By doing that I could spot the lines that were clearly disconnected
(net_components_unlink.png).
But I also found some lines that where disconnected even thou there are
nodes (red squares) linking them (apparently) -
net_components_pseudo_link.png

Should I increase my snapping threshold?
How can I see if my nodes are endnodes or located between two lines?

Thanks
Daniel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150115/0dd532a1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: net_components_unlink.png
Type: image/png
Size: 25250 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150115/0dd532a1/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: net_components_pseudo_link.png
Type: image/png
Size: 15279 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150115/0dd532a1/attachment-0001.png>

------------------------------

_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

End of grass-user Digest, Vol 105, Issue 25
*******************************************

Daniel_Victoria · January 16, 2015, 5:35pm

Hi Mark,

I tried using v.clean with break, rmdup and snap. However, when I try to run v.net.components I get an error saying that

v.net.components input=rede_brk_rmdup_snap@PERMANENT node_layer=1 output=component_brk_rmdup_snap method=weak
Building graph…
Registering arcs…
Flattening the graph…
Graph was built
ERROR: Cannot insert new record: insert into component_brk_rmdup_snap values (3689, 1)

I’m using Grass v7RC1 on Windows and v.net.components work if I use only the snap tool in my roads network

Cheers
Daniel

···

On Thu, Jan 15, 2015 at 7:40 PM, Mark Wynter <mark@dimensionaledge.com> wrote:

Hi Daniel
When you ran v.clean, did you v.clean tool=break,rmdupl ?

I suspect you still may encounter a problem with road T junctions if the vectors don’t cross or intersect. Take this simple example of 2 roads - NS, and EW. Each line has only 2 nodes - at the line’s start and end points.
|—
The problem is when the horizontal line “undershoots” the vertical line. The break command will have no effect, and snapping won’t make any difference because there is no midpoint node to which the horizontal line can attach itself to.

I’d be interested to hear how others have dealt with what is essentially a “data preparation” issue in GRASS?

[The solution I ended up implementing used a PostGIS SQL custom function to “break” overshoots and undershoots, with rules for correct treatment of bridges and tunnels. The function is fast, and can be run in parallel when working with continental-sized road networks.
After PostGIS, I import the “broken-up” network into GRASS and use v.clean to snap the nodes, but NOT break the lines - as my PostGIS function has already dealt with this.
The GRASS procedure involves running the v.clean in 5 steps, to avoid inadvertently snapping a tunnel node to say a bridge node, just because they are spatially close.

v.clean tool=snap # no bridges, no tunnels

v.clean tool=snap # bridges only

v.clean tool=snap # tunnels only

v.patch (steps 1-3)

v.clean tool=snap # on the patched layer, with a much smaller tolerance to that used in steps 1-3 to again avoid say tunnel nodes inadvertently snapping to bridge nodes.
We’re then good to go with v.net analysis.]

This may sound like a complicated process, but when using imperfect data to model complex road networks in dense urban areas, its important to get the vector topology as close as possible to the real world. I’ve found this process generates a highly representative, routable vector topology from imperfect OSM data.

Your situation might be much simpler, which is not to say you won’t encounter anomalies like you’ve already identified.

Mark

Message: 1
Date: Thu, 15 Jan 2015 15:22:27 -0200
From: Daniel Victoria <daniel.victoria@gmail.com>
To: grass <grass-user@lists.osgeo.org>
Subject: [GRASS-user] Checking and fixing roads for network analysis
Message-ID:
<CA+irsJjeKZQ9EqQ7igZcgL92k7uMfdKUJv6A-KVbtP7wn-XqYQ@mail.gmail.com>
Content-Type: text/plain; charset=“utf-8”

Hi List,

I’ll need to perform some network analysis and right now I’m trying to see
if my roads map is all good. I know that there are some clearly isolated
areas on my roads network but I’d like to identify and fix the less obvious
ones.

Here is what I did:

Import road lines (v.in.ogr)

Clean the topology with snap tool, thresh=0.0001 (it’s a latlong
location)

Create network (v.net using the nodes option --not sure if that was the
correct option)

Ran v.net.components to find the weakly connected components, not
setting any cost columns

Color roads lines according to the component number

By doing that I could spot the lines that were clearly disconnected
(net_components_unlink.png).
But I also found some lines that where disconnected even thou there are
nodes (red squares) linking them (apparently) -
net_components_pseudo_link.png

Should I increase my snapping threshold?
How can I see if my nodes are endnodes or located between two lines?

Thanks
Daniel
-------------- next part --------------
An HTML attachment was scrubbed…
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150115/0dd532a1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed…
Name: net_components_unlink.png
Type: image/png
Size: 25250 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150115/0dd532a1/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed…
Name: net_components_pseudo_link.png
Type: image/png
Size: 15279 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150115/0dd532a1/attachment-0001.png>

grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

End of grass-user Digest, Vol 105, Issue 25

grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Mark_Wynter · January 16, 2015, 9:03pm

Hi Daniel - not sure why you’re getting that error - I’ll leave that to someone more familiar with the backend workings to comment on.

In the interests of you moving forward, one approach might be that you simply manually add nodes in say QGIS before running v.snap.
By that, I mean:

1) In QGIS, open up the shapefile
2) Click the Edit layer icon
3) Find the line at that is closest to the point you wish to snap to another line end. Double click on the line at the point where you want to insert a new node - a new node should appear. Make sure the other line end is within your snap tolerance of your newly added node. If not, drag the line end so that it is.
4) Repeat for other “disconnected” parts of your road network
5) Save.
6) Import into GRASS and run v.clean.

You still need to break the lines that cross - which are not tunnels or bridges.

Even if v.clean tool=break is working correctly, in my experience you may still encounter unconnected T junctions where the lines undershoot, and there is no node on the adjacent line to which the line ending can snap to. The QGIS approach I’ve outlined will fix that - provided you know where the disjointed lines are.

If you’re working with a large scale road networks, its not practical to manually detect or fix such anomalies - which is why I developed my own PostGIS method I outlined yesterday.

HTH
Mark

On 17 Jan 2015, at 4:05 am, Daniel Victoria <daniel.victoria@gmail.com> wrote:

Hi Mark,

I tried using v.clean with break, rmdup and snap. However, when I try to run v.net.components I get an error saying that

v.net.components input=rede_brk_rmdup_snap@PERMANENT node_layer=1 output=component_brk_rmdup_snap method=weak
Building graph...
Registering arcs...
Flattening the graph...
Graph was built
ERROR: Cannot insert new record: insert into component_brk_rmdup_snap values (3689, 1)

I'm using Grass v7RC1 on Windows and v.net.components work if I use only the snap tool in my roads network

Cheers
Daniel

Daniel_Victoria · January 19, 2015, 3:48pm

Hi Mark and others,

I figured out where the error I’m getting from v.net.components is coming from. I just don’t know how to fix it.

The problem is that when I clean my roads network it’s breaking some lines and these are kept with the same categories. So when I run v.net.components it tries to calculate the connectivity of the components based on the cat value. Since there are some repeated values for broken roads, it can’t continue. So I believe I need to get new cat values for the network. Just don’t know ow to do it (been away from GRASS for a while)…

How can I get new cat values? And can I keep the old values in order to keep the DB link to the roads network?

Cheers
Daniel

···

On Fri, Jan 16, 2015 at 3:35 PM, Daniel Victoria <daniel.victoria@gmail.com> wrote:

Hi Mark,

I tried using v.clean with break, rmdup and snap. However, when I try to run v.net.components I get an error saying that

v.net.components input=rede_brk_rmdup_snap@PERMANENT node_layer=1 output=component_brk_rmdup_snap method=weak
Building graph…
Registering arcs…
Flattening the graph…
Graph was built
ERROR: Cannot insert new record: insert into component_brk_rmdup_snap values (3689, 1)

I’m using Grass v7RC1 on Windows and v.net.components work if I use only the snap tool in my roads network

Cheers
Daniel

On Thu, Jan 15, 2015 at 7:40 PM, Mark Wynter <mark@dimensionaledge.com> wrote:

Hi Daniel
When you ran v.clean, did you v.clean tool=break,rmdupl ?

I suspect you still may encounter a problem with road T junctions if the vectors don’t cross or intersect. Take this simple example of 2 roads - NS, and EW. Each line has only 2 nodes - at the line’s start and end points.
|—
The problem is when the horizontal line “undershoots” the vertical line. The break command will have no effect, and snapping won’t make any difference because there is no midpoint node to which the horizontal line can attach itself to.

I’d be interested to hear how others have dealt with what is essentially a “data preparation” issue in GRASS?

[The solution I ended up implementing used a PostGIS SQL custom function to “break” overshoots and undershoots, with rules for correct treatment of bridges and tunnels. The function is fast, and can be run in parallel when working with continental-sized road networks.
After PostGIS, I import the “broken-up” network into GRASS and use v.clean to snap the nodes, but NOT break the lines - as my PostGIS function has already dealt with this.
The GRASS procedure involves running the v.clean in 5 steps, to avoid inadvertently snapping a tunnel node to say a bridge node, just because they are spatially close.

v.clean tool=snap # no bridges, no tunnels

v.clean tool=snap # bridges only

v.clean tool=snap # tunnels only

v.patch (steps 1-3)

v.clean tool=snap # on the patched layer, with a much smaller tolerance to that used in steps 1-3 to again avoid say tunnel nodes inadvertently snapping to bridge nodes.
We’re then good to go with v.net analysis.]

This may sound like a complicated process, but when using imperfect data to model complex road networks in dense urban areas, its important to get the vector topology as close as possible to the real world. I’ve found this process generates a highly representative, routable vector topology from imperfect OSM data.

Your situation might be much simpler, which is not to say you won’t encounter anomalies like you’ve already identified.

Mark

Message: 1
Date: Thu, 15 Jan 2015 15:22:27 -0200
From: Daniel Victoria <daniel.victoria@gmail.com>
To: grass <grass-user@lists.osgeo.org>
Subject: [GRASS-user] Checking and fixing roads for network analysis
Message-ID:
<CA+irsJjeKZQ9EqQ7igZcgL92k7uMfdKUJv6A-KVbtP7wn-XqYQ@mail.gmail.com>
Content-Type: text/plain; charset=“utf-8”

Hi List,

I’ll need to perform some network analysis and right now I’m trying to see
if my roads map is all good. I know that there are some clearly isolated
areas on my roads network but I’d like to identify and fix the less obvious
ones.

Here is what I did:

Import road lines (v.in.ogr)

Clean the topology with snap tool, thresh=0.0001 (it’s a latlong
location)

Create network (v.net using the nodes option --not sure if that was the
correct option)

Ran v.net.components to find the weakly connected components, not
setting any cost columns

Color roads lines according to the component number

By doing that I could spot the lines that were clearly disconnected
(net_components_unlink.png).
But I also found some lines that where disconnected even thou there are
nodes (red squares) linking them (apparently) -
net_components_pseudo_link.png

Should I increase my snapping threshold?
How can I see if my nodes are endnodes or located between two lines?

Thanks
Daniel
-------------- next part --------------
An HTML attachment was scrubbed…
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150115/0dd532a1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed…
Name: net_components_unlink.png
Type: image/png
Size: 25250 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150115/0dd532a1/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed…
Name: net_components_pseudo_link.png
Type: image/png
Size: 15279 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150115/0dd532a1/attachment-0001.png>

grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

End of grass-user Digest, Vol 105, Issue 25

grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

mlennert · January 19, 2015, 4:11pm

On 19/01/15 16:48, Daniel Victoria wrote:

Hi Mark and others,

I figured out where the error I'm getting from v.net.components is
coming from. I just don't know how to fix it.
The problem is that when I clean my roads network it's breaking some
lines and these are kept with the same categories. So when I run
v.net.components it tries to calculate the connectivity of the
components based on the cat value. Since there are some repeated values
for broken roads, it can't continue. So I believe I need to get new cat
values for the network. Just don't know ow to do it (been away from
GRASS for a while)...

How can I get new cat values? And can I keep the old values in order to
keep the DB link to the roads network?

v.category option=add.

You can create new cat values on another layer and keep the old ones on layer 1.

Moritz

Daniel_Victoria · January 19, 2015, 4:33pm

Thanks Moritz,

I was just getting confused because of the layers. I’ve now added a new layer to my roads vector, added new cats and created the network. v.net.components worked nicely. I just need to study more how these layers work so I can have a layer with the new categories and not loose the link with the old cats, where the roads name and type are stored.

Cheers
Daniel

···

On Mon, Jan 19, 2015 at 2:11 PM, Moritz Lennert <mlennert@club.worldonline.be> wrote:

On 19/01/15 16:48, Daniel Victoria wrote:

Hi Mark and others,

I figured out where the error I’m getting from v.net.components is
coming from. I just don’t know how to fix it.
The problem is that when I clean my roads network it’s breaking some
lines and these are kept with the same categories. So when I run
v.net.components it tries to calculate the connectivity of the
components based on the cat value. Since there are some repeated values
for broken roads, it can’t continue. So I believe I need to get new cat
values for the network. Just don’t know ow to do it (been away from
GRASS for a while)…

How can I get new cat values? And can I keep the old values in order to
keep the DB link to the roads network?

v.category option=add.

You can create new cat values on another layer and keep the old ones on layer 1.

Moritz