Hello,
I am trying to run 'shortest' on a DB with the routing network of the whole planet. I have indices for the id, source, target and geom columns, and apply a window to limit the data I pass to 'shortest'. However, the postgres server crashes unless the source and target nodes are very close.
I suspect the problem may come from a lack of RAM since the server only has 1GB, but still have not looked deeper into it. Has anyone put a server running with the whole planet routing network and experienced the same problem? How much memory is needed?
Thanks in advance for any feedback,
Tao
On 2/25/2013 4:22 AM, Tao Romera Martinez wrote:
Hello,
I am trying to run 'shortest' on a DB with the routing network of the
whole planet. I have indices for the id, source, target and geom
columns, and apply a window to limit the data I pass to 'shortest'.
However, the postgres server crashes unless the source and target
nodes are very close.
I suspect the problem may come from a lack of RAM since the server
only has 1GB, but still have not looked deeper into it. Has anyone
put a server running with the whole planet routing network and
experienced the same problem? How much memory is needed?
Yes, this sounds like a reason conclusion. pgRouting has to build a graph in memory before it can solve it so loading very large numbers of vectors into a graph will use a lot of memory. We use the Boost library to build and solve the graph and it does not have any support for paging data into memory as it is needed. Actually none of the other algorithms have that support either.
You might try something like:
select edges from streets where near the startpoint
union
select edges from streets where near the endpoint
union
select edges from streets where highway only
The idea here is to select all edges near the start and end points, where "near" is a large enough radius to get you into the major roads part of the network, and to only select the major roads network for the rest of the graph. This would eliminate millions of neighborhood streets that you don't car about.
Turn the above into a query and pass that to you solution.
-Steve
Le 25/02/2013 14:41, Stephen Woodbridge a écrit :
On 2/25/2013 4:22 AM, Tao Romera Martinez wrote:
Hello,
I am trying to run 'shortest' on a DB with the routing network of the
whole planet. I have indices for the id, source, target and geom
columns, and apply a window to limit the data I pass to 'shortest'.
However, the postgres server crashes unless the source and target
nodes are very close.
I suspect the problem may come from a lack of RAM since the server
only has 1GB, but still have not looked deeper into it. Has anyone
put a server running with the whole planet routing network and
experienced the same problem? How much memory is needed?
Yes, this sounds like a reason conclusion. pgRouting has to build a graph in memory before it can solve it so loading very large numbers of vectors into a graph will use a lot of memory. We use the Boost library to build and solve the graph and it does not have any support for paging data into memory as it is needed. Actually none of the other algorithms have that support either.
You might try something like:
select edges from streets where near the startpoint
union
select edges from streets where near the endpoint
union
select edges from streets where highway only
The idea here is to select all edges near the start and end points, where "near" is a large enough radius to get you into the major roads part of the network, and to only select the major roads network for the rest of the graph. This would eliminate millions of neighborhood streets that you don't car about.
Turn the above into a query and pass that to you solution.
-Steve
_______________________________________________
Pgrouting-users mailing list
Pgrouting-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-users
I was about to give quite the same answer as Stephen.
Another thing you could do if the approach he proposed is not sufficient (or not adapted) is to use a part of the server's HDD for swapping.
Regards,
Sylvain.
Stephen, Moi,
Thank you very much for your answers.
The problem actually got solved after a 'vacuum full' of the DB. It
was not a problem of insufficient RAM.
Nevertheless, the solution you proposed is very interesting and very
smart, so I keep it in mind for future services and products (as long
as you travel by car and are willing to take the highways, which is
not always the case, as in our latest release sweetway.zilapp.com).
Thank you to both of you for your help,
Tao
2013/3/7 Moi Lemien <hsylvio@gmail.com>:
Le 25/02/2013 14:41, Stephen Woodbridge a écrit :
On 2/25/2013 4:22 AM, Tao Romera Martinez wrote:
Hello,
I am trying to run 'shortest' on a DB with the routing network of the
whole planet. I have indices for the id, source, target and geom
columns, and apply a window to limit the data I pass to 'shortest'.
However, the postgres server crashes unless the source and target
nodes are very close.
I suspect the problem may come from a lack of RAM since the server
only has 1GB, but still have not looked deeper into it. Has anyone
put a server running with the whole planet routing network and
experienced the same problem? How much memory is needed?
Yes, this sounds like a reason conclusion. pgRouting has to build a graph
in memory before it can solve it so loading very large numbers of vectors
into a graph will use a lot of memory. We use the Boost library to build and
solve the graph and it does not have any support for paging data into memory
as it is needed. Actually none of the other algorithms have that support
either.
You might try something like:
select edges from streets where near the startpoint
union
select edges from streets where near the endpoint
union
select edges from streets where highway only
The idea here is to select all edges near the start and end points, where
"near" is a large enough radius to get you into the major roads part of the
network, and to only select the major roads network for the rest of the
graph. This would eliminate millions of neighborhood streets that you don't
car about.
Turn the above into a query and pass that to you solution.
-Steve
_______________________________________________
Pgrouting-users mailing list
Pgrouting-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-users
I was about to give quite the same answer as Stephen.
Another thing you could do if the approach he proposed is not sufficient (or
not adapted) is to use a part of the server's HDD for swapping.
Regards,
Sylvain.
_______________________________________________
Pgrouting-users mailing list
Pgrouting-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-users
--
Tao Romera Martínez
---------------------------------------------------------
Tel: 080-6805-0945
Address: Koganei-shi, Tokyo, Japan
Look for me on Facebook and LinkedIn
---------------------------------------------------------