[pgrouting-users] Nasty crash from pgRouting

pgRouting pgrouting-users
1

I’m currently getting a puzzling crash from pgRouting which reliably takes the entire Postgres server down with it!

I’m calling pgRouting using Psycopg from Python. My Python code has some fancy multi-processor code, so I’ve managed to isolate the problem using just the psql command line program.
The crash messages were not very informative, so I’ve also managed to track down all the Postgres logging switches and switched everything on - or all that I can find. I can forward the resulting log file if anyone is interested, but it is a bit big to post to a public listserver.

There’s a lot of what I assume is tracing through pgRouting, followed by:

------8<-----8<----
2011-01-30 20:05:29 CST CONTEXT: SQL statement “SELECT gid as id, source::integer, target::integer, length::double precision as cost FROM view_topo where setSRID(‘BOX3D(145.6460916 -8.1986466,150.5752148 -5.2633439)’::BOX3D, 4326) && the_geom”
PL/pgSQL function “dijkstra_sp_delta_directed” line 110 at FOR over EXECUTE statement
PL/pgSQL function “dijkstra_sp_delta” line 18 at FOR over EXECUTE statement
2011-01-30 20:05:29 CST LOCATION: elog_node_display, .\src\backend\nodes\print.c:85
2011-01-30 20:05:29 CST STATEMENT: select sum(ST_Length( geography( ST_Transform(the_geom, 4326))) ) AS LENGTH FROM dijkstra_sp_delta(‘view_topo’, 2429451, 2475147, 1.0);

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application’s support team for more information.
---------8<------8<-------

Then what looks to be recovery logging.

The SELECT statement that causes this is logged as:
2011-01-30 20:04:57 CST STATEMENT: select sum(ST_Length( geography( ST_Transform(the_geom, 4326))) ) AS LENGTH FROM dijkstra_sp_delta(‘view_topo’, 2429451, 2475147, 1.0);

Nodes 2429451 and 2475147 both exist in view_topo - I’ve checked.
The database is the OSM planet.osm file, imported with osm2po. The above two nodes are in Papua New Guinea. Initial (limited) tests with the UK are fine; Guam is completely fine; but New Zealand and PG cause this crash - haven’t tried anywhere else yet.

I’ve already tried the above query with non-routable routes in the UK (eg. from England to Belfast), and those have behaved as expected with zero rows returned. So this isn’t the problem.

My system details:

Windows 7 x32
Postgres 8.4
PostGIS 1.5
I’m having trouble determining my pgRouting version, but routing_core_wrappers.sql has a “last changes” date of 14/02/2008

Yes I’m using mainly older versions, but I am forced to do this due to availability of pre-built versions. Eg. PostGIS does not have a 64 bit version; and the general lack of later Windows builds of pgRouting.

Has anyone seen anything like this before?
Any thoughts as to what the problem might be?
It seems pretty bad if it can bring the entire server down with it…

Thanks in advance,

Richard Marsden

2

Hi Richard,

Unfortunately Win binaries are quite old because nobody is willing to
make new ones.
There can be few different reasons why it crushes (data problem for
example). Do you have any Linux box around to test your data with
newer version of pgRouting?

Anton.

3

Hi Richard,

You could try OSGeo Live DVD with already installed pgRouting in Virtualbox for example just to see if a newer version of would solve your issue: http://live.osgeo.org/
As Anton mentioned, there seem to be not many developers out there, who are willing (or have the knowledge) to contribute Windows binaries.
My feeling is that most pgRouting installations are running on Linux servers, while most users, who just want to try out pgRouting, prefer to do this on Windows.

Daniel

2011/1/31 Anton Patrushev <anton.patrushev@georepublic.de>

Hi Richard,

Unfortunately Win binaries are quite old because nobody is willing to
make new ones.
There can be few different reasons why it crushes (data problem for
example). Do you have any Linux box around to test your data with
newer version of pgRouting?

Anton.

Pgrouting-users mailing list
Pgrouting-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-users

–
Georepublic UG & Georepublic Japan
eMail: daniel.kastl@georepublic.de
Web: http://georepublic.de

4

Anton & Daniel,

Thanks.

I saw some references to problems with negative reverse_cost. I’ve checked the data and I don’t have any negative cost or reverse_cost values. Plenty of zeroes though.

Highest node value is 49,608,314 - so I don’t think it is the integer overflow problem I’m seeing discussed.

I also tried Iran (northern hemisphere, not too big, imperfect road data in osm) and it also crashes. The common thread seems to be that countries where I can’t match all (geonames-derived) cities with osm nodes, crash using valid nodes. This isn’t a simple “route cannot be found” as my UK Belfast example demonstrates (no routes possible from Belfast to the England - no crash occurs)

One problem is that all I’m getting is a big crash - nothing indicates what the problem actually is.

My laptop can boot into Ubuntu but the partition will be too small and the cpu too slow.

The machine I’m using is my main batch processing /testing machine. It currently doesn’t have space for a Unix/Linux partition, but has enough spare disk bays. I was already thinking of buying a large (1TB+) hard disk for this project. Although I’ve managed to squeak by with what I have, and PostGres’s requirements aren’t too big, manipulating planet.osm files does take some disk space!

Perhaps this is the way forward then: New disk and try building an Ubuntu partition with postgres/etc. Assuming I don’t have to rerun osm2po, it should take me a couple of days to rebuild everything.
Most of my ‘stack’ should run fine on Ubuntu - Postgres, PostGis, Python. Windows is being used partly for historical reasons (earlier versions of my Python scripts used Windows for a good reason which no longer applies), but also I’m using Excel for my output. Perhaps there’s a cross-platform Python library I can use. If not, I could create CSV files and “tie them together” with a Windows-based script (they’re simple data sheets but I am relying on some formatting to make the data easier to use).

re. building: I actually do most of my dev in Windows but using Visual Studio. I’ve seen the pgRouting Windows build instructions and they don’t look very Windows friendly! Believe it or not I would be happier doing such builds in a Unix/Linux type of environment… (I’ve built my own UMN Mapserver with FreeBSD in the past).

The project can’t go forward if these crashes continue. So a little bit of a gamble that a new disk (or two if I decide on RAID) and Ubuntu will fix it…

Richard

On Sun, Jan 30, 2011 at 9:51 PM, Daniel Kastl <daniel@georepublic.de> wrote:

Hi Richard,

You could try OSGeo Live DVD with already installed pgRouting in Virtualbox for example just to see if a newer version of would solve your issue: http://live.osgeo.org/
As Anton mentioned, there seem to be not many developers out there, who are willing (or have the knowledge) to contribute Windows binaries.
My feeling is that most pgRouting installations are running on Linux servers, while most users, who just want to try out pgRouting, prefer to do this on Windows.

Daniel

2011/1/31 Anton Patrushev <anton.patrushev@georepublic.de>

Hi Richard,

Unfortunately Win binaries are quite old because nobody is willing to
make new ones.
There can be few different reasons why it crushes (data problem for
example). Do you have any Linux box around to test your data with
newer version of pgRouting?

Anton.

5

On 1/31/2011 10:36 AM, Richard Marsden wrote:

Anton & Daniel,

Thanks.

I saw some references to problems with negative reverse_cost. I've
checked the data and I don't have any negative cost or reverse_cost
values. Plenty of zeroes though.

Ahhh, I'm guessing the zero cost is just as bad as negative costs, but I might be wrong on this. I would try setting all the zero costs to some really small positive cost and see if that helps.

Or if you really have zero length segments, then why not reduce the size of your graph by removing all of them.

You should also check the the source and target ids are not the same. as this is problematic also. If you have a cul d'sac there the start and end are the same but it does a loop, then you should split the loop into two segments.

-Steve

Highest node value is 49,608,314 - so I don't think it is the integer
overflow problem I'm seeing discussed.

I also tried Iran (northern hemisphere, not too big, imperfect road data
in osm) and it also crashes. The common thread seems to be that
countries where I can't match all (geonames-derived) cities with osm
nodes, crash using valid nodes. This isn't a simple "route cannot be
found" as my UK Belfast example demonstrates (no routes possible from
Belfast to the England - no crash occurs)

One problem is that all I'm getting is a big crash - nothing indicates
what the problem actually is.

My laptop can boot into Ubuntu but the partition will be too small and
the cpu too slow.

The machine I'm using is my main batch processing /testing machine. It
currently doesn't have space for a Unix/Linux partition, but has enough
spare disk bays. I was already thinking of buying a large (1TB+) hard
disk for this project. Although I've managed to squeak by with what I
have, and PostGres's requirements aren't too big, manipulating
planet.osm files does take some disk space!

Perhaps this is the way forward then: New disk and try building an
Ubuntu partition with postgres/etc. Assuming I don't have to rerun
osm2po, it should take me a couple of days to rebuild everything.
Most of my 'stack' should run fine on Ubuntu - Postgres, PostGis,
Python. Windows is being used partly for historical reasons (earlier
versions of my Python scripts used Windows for a good reason which no
longer applies), but also I'm using Excel for my output. Perhaps there's
a cross-platform Python library I can use. If not, I could create CSV
files and "tie them together" with a Windows-based script (they're
simple data sheets but I am relying on some formatting to make the data
easier to use).

re. building: I actually do most of my dev in Windows but using Visual
Studio. I've seen the pgRouting Windows build instructions and they
don't look very Windows friendly! Believe it or not I would be happier
doing such builds in a Unix/Linux type of environment... (I've built
my own UMN Mapserver with FreeBSD in the past).

The project can't go forward if these crashes continue. So a little bit
of a gamble that a new disk (or two if I decide on RAID) and Ubuntu will
fix it...

Richard

On Sun, Jan 30, 2011 at 9:51 PM, Daniel Kastl <daniel@georepublic.de
<mailto:daniel@georepublic.de>> wrote:

    Hi Richard,

    You could try OSGeo Live DVD with already installed pgRouting in
    Virtualbox for example just to see if a newer version of would solve
    your issue: http://live.osgeo.org/
    <http://live.osgeo.org/&gt;As Anton mentioned, there seem to be not
    many developers out there, who are willing (or have the knowledge)
    to contribute Windows binaries.
    My feeling is that most pgRouting installations are running on Linux
    servers, while most users, who just want to try out pgRouting,
    prefer to do this on Windows.

    Daniel

    2011/1/31 Anton Patrushev <anton.patrushev@georepublic.de
    <mailto:anton.patrushev@georepublic.de>>

        Hi Richard,

        Unfortunately Win binaries are quite old because nobody is
        willing to
        make new ones.
        There can be few different reasons why it crushes (data problem for
        example). Do you have any Linux box around to test your data with
        newer version of pgRouting?

        Anton.
        _______________________________________________

_______________________________________________
Pgrouting-users mailing list
Pgrouting-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-users

6

Steve,

I’m a little confused, well not confused, unclear perhaps. :slight_smile:
I’m using the dijkstra_sp_delta() function. This doesn’t explicitly say whether it uses directions and reverse costs. My results so far suggests it ignores directed graphs completely.

If it is using cost and reverse_cost, presumably I could just set these to the length values.
So I tried this, but I’m still getting the crash.
I do have some very small lengths - the smallest is being reported as 1.0E-7.
The data was imported by osm2po from planet.osm.
I’m assuming this (or these) tiny distances are due to what are essentially duplicated nodes which are sufficiently far apart that osm2po does not recognize them as duplicates.

I’ve just done a search for links with the same source and target node identifiers - no matches.
(for my application, I could actually delete the cul-de-sac loops that you describe)

The new hard disk is on order for an attempt to build everything in Ubuntu.

Richard

On Mon, Jan 31, 2011 at 1:11 PM, Stephen Woodbridge <woodbri@swoodbridge.com> wrote:

On 1/31/2011 10:36 AM, Richard Marsden wrote:

Anton & Daniel,

Thanks.

I saw some references to problems with negative reverse_cost. I’ve
checked the data and I don’t have any negative cost or reverse_cost
values. Plenty of zeroes though.

Ahhh, I’m guessing the zero cost is just as bad as negative costs, but I might be wrong on this. I would try setting all the zero costs to some really small positive cost and see if that helps.

Or if you really have zero length segments, then why not reduce the size of your graph by removing all of them.

You should also check the the source and target ids are not the same. as this is problematic also. If you have a cul d’sac there the start and end are the same but it does a loop, then you should split the loop into two segments.

-Steve

Highest node value is 49,608,314 - so I don’t think it is the integer
overflow problem I’m seeing discussed.

I also tried Iran (northern hemisphere, not too big, imperfect road data
in osm) and it also crashes. The common thread seems to be that
countries where I can’t match all (geonames-derived) cities with osm
nodes, crash using valid nodes. This isn’t a simple “route cannot be
found” as my UK Belfast example demonstrates (no routes possible from
Belfast to the England - no crash occurs)

One problem is that all I’m getting is a big crash - nothing indicates
what the problem actually is.

My laptop can boot into Ubuntu but the partition will be too small and
the cpu too slow.

The machine I’m using is my main batch processing /testing machine. It
currently doesn’t have space for a Unix/Linux partition, but has enough
spare disk bays. I was already thinking of buying a large (1TB+) hard
disk for this project. Although I’ve managed to squeak by with what I
have, and PostGres’s requirements aren’t too big, manipulating
planet.osm files does take some disk space!

Perhaps this is the way forward then: New disk and try building an
Ubuntu partition with postgres/etc. Assuming I don’t have to rerun
osm2po, it should take me a couple of days to rebuild everything.
Most of my ‘stack’ should run fine on Ubuntu - Postgres, PostGis,
Python. Windows is being used partly for historical reasons (earlier
versions of my Python scripts used Windows for a good reason which no
longer applies), but also I’m using Excel for my output. Perhaps there’s
a cross-platform Python library I can use. If not, I could create CSV
files and “tie them together” with a Windows-based script (they’re
simple data sheets but I am relying on some formatting to make the data
easier to use).

re. building: I actually do most of my dev in Windows but using Visual
Studio. I’ve seen the pgRouting Windows build instructions and they
don’t look very Windows friendly! Believe it or not I would be happier
doing such builds in a Unix/Linux type of environment… (I’ve built
my own UMN Mapserver with FreeBSD in the past).

The project can’t go forward if these crashes continue. So a little bit
of a gamble that a new disk (or two if I decide on RAID) and Ubuntu will
fix it…

Richard

On Sun, Jan 30, 2011 at 9:51 PM, Daniel Kastl <daniel@georepublic.de

mailto:[daniel@georepublic.de](mailto:daniel@georepublic.de)> wrote:

Hi Richard,

You could try OSGeo Live DVD with already installed pgRouting in
Virtualbox for example just to see if a newer version of would solve
your issue: http://live.osgeo.org/
<http://live.osgeo.org/>As Anton mentioned, there seem to be not
many developers out there, who are willing (or have the knowledge)
to contribute Windows binaries.
My feeling is that most pgRouting installations are running on Linux
servers, while most users, who just want to try out pgRouting,
prefer to do this on Windows.

Daniel

2011/1/31 Anton Patrushev <anton.patrushev@georepublic.de

mailto:[anton.patrushev@georepublic.de](mailto:anton.patrushev@georepublic.de)>

Hi Richard,

Unfortunately Win binaries are quite old because nobody is
willing to
make new ones.
There can be few different reasons why it crushes (data problem for
example). Do you have any Linux box around to test your data with
newer version of pgRouting?

Anton.

Pgrouting-users mailing list
Pgrouting-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-users

Pgrouting-users mailing list
Pgrouting-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-users

7

As a follow up…

I still don’t know the reason for the crash, but I’ve finally got a large enough ubuntu system installed and loaded up with PostGIS, pgrouting 1.5, etc. It appears to be working okay, although psycopg2 is returning “no result” in a different manner - that threw my Python code a little.

So whatever the bug was, it appears to have been fixed in pgRouting 1.5.

Whilst waiting for the disk to arrive, I did run a series of scripts to remove links less than a fixed distance (1m I think it was) and links with the same start and end node. This did not fix the problem.
The scripts also cleaned up the length & cost information, so I shall be running them again this evening. With the whole planet.osm dataset loaded, they took a while to run & re-index…

Richard (M)