[pgrouting-users] crash of driving_distance with large network

pgRouting pgrouting-users
1

Hi all,

I try to launch the driving_distance function with a large network consisting of 100,000 edges. Every time I launch the query, the server closes the connection unexpectedly.

Any idea why? Is there a way to get a more precise error message?
I only get as message: "Server closed the connection unexpectedly. This probably means the server terminated abnormally before or while processing the request"

thanks

--
Ph D. Student Markus Innerebner

DIS Research Group - Faculty of Computer Science
Free University Bozen-Bolzano

Dominikanerplatz 3 - Room 211
I - 39100 Bozen
Phone: +39-0471-016143
Mobile: +39-333-9392929

gpg --fingerprint
-------------------------------------
pub 1024D/588F6308 2007-01-09
       Key fingerprint = 6948 947E CBD2 89FD E773 E863 914F EB1B 588F 6308
sub 2048g/BF4877D0 2007-01-09

2

I have seen that message and believe it is not related to pgRouting.

I cannot remember the exact situation, but I think vacuuming the database
was the solution.

I also may have mis-remembered this.

Worth

  _____

From: pgrouting-users-bounces@lists.osgeo.org
[mailto:pgrouting-users-bounces@lists.osgeo.org] On Behalf Of Markus
Innerebner
Sent: Monday, January 17, 2011 12:59 PM
To: pgRouting users mailing list
Subject: [pgrouting-users] crash of driving_distance with large network

Hi all,

I try to launch the driving_distance function with a large network
consisting of 100,000 edges. Every time I launch the query, the server
closes the connection unexpectedly.

Any idea why? Is there a way to get a more precise error message?
I only get as message: "Server closed the connection unexpectedly. This
probably means the server terminated abnormally before or while
processing the request"

thanks

--
Ph D. Student Markus Innerebner

DIS Research Group - Faculty of Computer Science
Free University Bozen-Bolzano

Dominikanerplatz 3 - Room 211
I - 39100 Bozen
Phone: +39-0471-016143
Mobile: +39-333-9392929

gpg --fingerprint
-------------------------------------
pub 1024D/588F6308 2007-01-09
       Key fingerprint = 6948 947E CBD2 89FD E773 E863 914F EB1B 588F 6308
sub 2048g/BF4877D0 2007-01-09
_______________________________________________
Pgrouting-users mailing list
Pgrouting-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-users

  _____

No virus found in this message.
Checked by AVG - www.avg.com
Version: 10.0.1191 / Virus Database: 1435/3386 - Release Date: 01/17/11

3

What Worth mentions can be a solution.
I also remember cases when a vacuum (and logout/login?) solved such a problem.
But it can be also caused by pgRouting query.

Did you take a look at the PostgreSQL log file?

Daniel

2011/1/18 Worth Lutz <wal3@mindspring.com>

I have seen that message and believe it is not related to pgRouting.

I cannot remember the exact situation, but I think vacuuming the database was the solution.

I also may have mis-remembered this.

Worth

From: pgrouting-users-bounces@lists.osgeo.org [mailto:pgrouting-users-bounces@lists.osgeo.org] On Behalf Of Markus Innerebner
Sent: Monday, January 17, 2011 12:59 PM
To: pgRouting users mailing list
Subject: [pgrouting-users] crash of driving_distance with large network

Hi all,

I try to launch the driving_distance function with a large network
consisting of 100,000 edges. Every time I launch the query, the server
closes the connection unexpectedly.

Any idea why? Is there a way to get a more precise error message?
I only get as message: “Server closed the connection unexpectedly. This
probably means the server terminated abnormally before or while
processing the request”

thanks

–
Ph D. Student Markus Innerebner

DIS Research Group - Faculty of Computer Science
Free University Bozen-Bolzano

Dominikanerplatz 3 - Room 211
I - 39100 Bozen
Phone: +39-0471-016143
Mobile: +39-333-9392929

gpg --fingerprint

pub 1024D/588F6308 2007-01-09
Key fingerprint = 6948 947E CBD2 89FD E773 E863 914F EB1B 588F 6308
sub 2048g/BF4877D0 2007-01-09

Pgrouting-users mailing list
Pgrouting-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-users

No virus found in this message.
Checked by AVG - www.avg.com
Version: 10.0.1191 / Virus Database: 1435/3386 - Release Date: 01/17/11

Pgrouting-users mailing list
Pgrouting-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-users

–
Georepublic UG & Georepublic Japan
eMail: daniel.kastl@georepublic.de
Web: http://georepublic.de

4

Yeah, if your query requires too much memory to assemble and solve the graph, this can cause that problem. You might be able to solve the problem by allocating more memory to postgresql.

-Steve W

On 1/17/2011 9:57 PM, Daniel Kastl wrote:

What Worth mentions can be a solution.
I also remember cases when a vacuum (and logout/login?) solved such a
problem.
But it can be also caused by pgRouting query.

Did you take a look at the PostgreSQL log file?

Daniel

2011/1/18 Worth Lutz <wal3@mindspring.com <mailto:wal3@mindspring.com>>

    I have seen that message and believe it is not related to pgRouting.

    I cannot remember the exact situation, but I think vacuuming the
    database was the solution.

    I also may have mis-remembered this.

    Worth

    *From: * pgrouting-users-bounces@lists.osgeo.org
    <mailto:pgrouting-users-bounces@lists.osgeo.org>
    [mailto:pgrouting-users-bounces@lists.osgeo.org
    <mailto:pgrouting-users-bounces@lists.osgeo.org>] *On Behalf Of
    *Markus Innerebner
    *Sent:* Monday, January 17, 2011 12:59 PM
    *To:* pgRouting users mailing list
    *Subject:* [pgrouting-users] crash of driving_distance with large
    network

    Hi all,

    I try to launch the driving_distance function with a large network
    consisting of 100,000 edges. Every time I launch the query, the server
    closes the connection unexpectedly.

    Any idea why? Is there a way to get a more precise error message?
    I only get as message: "Server closed the connection unexpectedly. This
    probably means the server terminated abnormally before or while
    processing the request"

    thanks

    --
    Ph D. Student Markus Innerebner

    DIS Research Group - Faculty of Computer Science
    Free University Bozen-Bolzano

    Dominikanerplatz 3 - Room 211
    I - 39100 Bozen
    Phone: +39-0471-016143
    Mobile: +39-333-9392929

    gpg --fingerprint
    -------------------------------------
    pub 1024D/588F6308 2007-01-09
            Key fingerprint = 6948 947E CBD2 89FD E773 E863 914F EB1B
    588F 6308
    sub 2048g/BF4877D0 2007-01-09

5

You can also increase the log level of postgres in postgresql.conf to get more messages

2011/1/18 Stephen Woodbridge <woodbri@swoodbridge.com>

Yeah, if your query requires too much memory to assemble and solve the graph, this can cause that problem. You might be able to solve the problem by allocating more memory to postgresql.

-Steve W

On 1/17/2011 9:57 PM, Daniel Kastl wrote:

What Worth mentions can be a solution.
I also remember cases when a vacuum (and logout/login?) solved such a
problem.
But it can be also caused by pgRouting query.

Did you take a look at the PostgreSQL log file?

Daniel

2011/1/18 Worth Lutz <wal3@mindspring.com mailto:[wal3@mindspring.com](mailto:wal3@mindspring.com)>

I have seen that message and believe it is not related to pgRouting.

I cannot remember the exact situation, but I think vacuuming the
database was the solution.

I also may have mis-remembered this.

Worth

*From: * pgrouting-users-bounces@lists.osgeo.org
mailto:[pgrouting-users-bounces@lists.osgeo.org](mailto:pgrouting-users-bounces@lists.osgeo.org)
[mailto:pgrouting-users-bounces@lists.osgeo.org
mailto:[pgrouting-users-bounces@lists.osgeo.org](mailto:pgrouting-users-bounces@lists.osgeo.org)] *On Behalf Of
*Markus Innerebner
Sent: Monday, January 17, 2011 12:59 PM
To: pgRouting users mailing list
Subject: [pgrouting-users] crash of driving_distance with large
network

Hi all,

I try to launch the driving_distance function with a large network
consisting of 100,000 edges. Every time I launch the query, the server
closes the connection unexpectedly.

Any idea why? Is there a way to get a more precise error message?
I only get as message: “Server closed the connection unexpectedly. This
probably means the server terminated abnormally before or while
processing the request”

thanks

–
Ph D. Student Markus Innerebner

DIS Research Group - Faculty of Computer Science
Free University Bozen-Bolzano

Dominikanerplatz 3 - Room 211
I - 39100 Bozen
Phone: +39-0471-016143
Mobile: +39-333-9392929

gpg --fingerprint

pub 1024D/588F6308 2007-01-09
Key fingerprint = 6948 947E CBD2 89FD E773 E863 914F EB1B
588F 6308
sub 2048g/BF4877D0 2007-01-09

Pgrouting-users mailing list
Pgrouting-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-users

–
Lorenzo Amato
lorenzo.amato@geosdi.org
lorenzotlc@gmail.com
lorenzo.amato@nsdi.it

Consiglio Nazionale delle Ricerche
Istituto di Metodologie per l’Analisi Ambientale - geoSDI

6

Worth Lutz wrote:

I have seen that message and believe it is not related to pgRouting.

I cannot remember the exact situation, but I think vacuuming the database
was the solution.

Vacuuming the database brings in my case no success.

thanks anyway

--
Ph D. Student Markus Innerebner

DIS Research Group - Faculty of Computer Science
Free University Bozen-Bolzano

Dominikanerplatz 3 - Room 211
I - 39100 Bozen
Phone: +39-0471-016143
Mobile: +39-333-9392929

gpg --fingerprint
-------------------------------------
pub 1024D/588F6308 2007-01-09
       Key fingerprint = 6948 947E CBD2 89FD E773 E863 914F EB1B 588F 6308
sub 2048g/BF4877D0 2007-01-09

7

Did you take a look at the PostgreSQL log file?

Hi Daniel,

I took a look into the log file after changing the log level to debug5.

Additionally I only get:
DEBUG: name: unamed,blockState: DEFAULT; state: INPROGR,xid/subid/cid: 0/1/0, nestlvl:1, children:

server closed connection ....

No I tried to restrict the number of tuples to be loaded with a bounding box query (having at most 400 edges). Also in this case the function crashes.

So it looks like, as something is wrong in my data. Is there a way to validate my network with a postgis function?

regards

--
Ph D. Student Markus Innerebner

DIS Research Group - Faculty of Computer Science
Free University Bozen-Bolzano

Dominikanerplatz 3 - Room 211
I - 39100 Bozen
Phone: +39-0471-016143
Mobile: +39-333-9392929

gpg --fingerprint
-------------------------------------
pub 1024D/588F6308 2007-01-09
       Key fingerprint = 6948 947E CBD2 89FD E773 E863 914F EB1B 588F 6308
sub 2048g/BF4877D0 2007-01-09

8

Hi all

Just for your interests:
I found out the reason of the crash. the id of the edges and vertexes exceed the range of int4. I used as data type integer.

regards

--
Ph D. Student Markus Innerebner

DIS Research Group - Faculty of Computer Science
Free University Bozen-Bolzano

Dominikanerplatz 3 - Room 211
I - 39100 Bozen
Phone: +39-0471-016143
Mobile: +39-333-9392929

gpg --fingerprint
-------------------------------------
pub 1024D/588F6308 2007-01-09
       Key fingerprint = 6948 947E CBD2 89FD E773 E863 914F EB1B 588F 6308
sub 2048g/BF4877D0 2007-01-09

9

On 1/19/2011 8:46 AM, Markus Innerebner wrote:

Hi all

Just for your interests:
I found out the reason of the crash. the id of the edges and vertexes
exceed the range of int4. I used as data type integer.

Thanks for posting this follow up.

So for performance, if I'm not mistaken, it is best to renumber all you id to start with 1 (one) and to be sequential, ie: avoid large gaps in the numbering. If you need to preserve the original ids then create new columns and to renumber into. Then use the renumbered columns when passing data to pgRouting. This obviously does not help with the int4 overflow issue if you have a huge number of ids.

-Steve W