Hi all,
I think I have just submitted changes to the bidirectional dijkstra and astar routines the resolve the server crashes we were seeing. This closes on major outstanding bug against 2.0.
Razequl,
I made two simple changes to your code BiDirDijkstra.cpp and the astar version:
1. in BiDirDijkstra::initall I add:
m_vecNodeVector.reserve(maxNode + 1);
2. in BiDirDijkstra::bidir_dijkstra I move the call to initall(maxNode); to before construct_graph(edges, edge_count, maxNode);
This does two major things:
1. std::vector doubles the size of the array every time it needs to increase its size and then needs to copy the old data to the new area.
2. reserve() pre allocates all the memory we need once, which avoid realloc memory fragmentation and avoids the copy each time it reallocs so this improves performance.
Anytime you know how much space you are going to need you should reserve it up front.
So things look good for now.
Thanks,
-Steve
Hello,
I've just pulled from the develop branch and built fresh and I'm getting a server crash from pgr_bdastar. If I run exactly the same thing using pgr_astar, it works. This is using postgres 9.2.4 and postgis 2.0.3 from Homebrew on Mac 10.8.3
I'm still working on a bulk routing function using bdastar, but just realized that the regular version was crashing without my modifications. I'm happy to try to track this down, but have little knowledge of how to debug within postgresql. Any pointers would be appreciated.
Best,
Alec
On Jun 1, 2013, at 2:13 PM, Stephen Woodbridge <woodbri@swoodbridge.com> wrote:
Hi all,
I think I have just submitted changes to the bidirectional dijkstra and astar routines the resolve the server crashes we were seeing. This closes on major outstanding bug against 2.0.
Razequl,
I made two simple changes to your code BiDirDijkstra.cpp and the astar version:
1. in BiDirDijkstra::initall I add:
m_vecNodeVector.reserve(maxNode + 1);
2. in BiDirDijkstra::bidir_dijkstra I move the call to initall(maxNode); to before construct_graph(edges, edge_count, maxNode);
This does two major things:
1. std::vector doubles the size of the array every time it needs to increase its size and then needs to copy the old data to the new area.
2. reserve() pre allocates all the memory we need once, which avoid realloc memory fragmentation and avoids the copy each time it reallocs so this improves performance.
Anytime you know how much space you are going to need you should reserve it up front.
So things look good for now.
Thanks,
-Steve
_______________________________________________
pgrouting-dev mailing list
pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
On 6/4/2013 3:53 PM, Alec Gosse wrote:
Hello,
I've just pulled from the develop branch and built fresh and I'm
getting a server crash from pgr_bdastar. If I run exactly the same
thing using pgr_astar, it works. This is using postgres 9.2.4 and
postgis 2.0.3 from Homebrew on Mac 10.8.3
I'm using pg9.2.4 and postgis 2.0.x or 2.1.0beta3dev and I fixed these issues I thought. Evidently not.
I'm still working on a bulk routing function using bdastar, but just
realized that the regular version was crashing without my
modifications. I'm happy to try to track this down, but have little
knowledge of how to debug within postgresql. Any pointers would be
appreciated.
OK, start by uncommenting #define DEBUG 1 in both the C and C++ code. Then add more DBG('I got here message\n"). In the C++ code these messages get written to a file '/tmp/sew-debug' and you can do a tail -f on that in another window to see your progress.
This is likely a issue in the C++ code but it may also be something systemic about the way we are integrating code. If you build your own postgresql you might want to rebuild it with --enable-cassert option to configure which will put the database backend into a more rigorous memory check mode.
That is a start, I would be appy to help more if I can.
Thanks,
-Steve
Best, Alec
On Jun 1, 2013, at 2:13 PM, Stephen Woodbridge
<woodbri@swoodbridge.com> wrote:
Hi all,
I think I have just submitted changes to the bidirectional dijkstra
and astar routines the resolve the server crashes we were seeing.
This closes on major outstanding bug against 2.0.
Razequl,
I made two simple changes to your code BiDirDijkstra.cpp and the
astar version:
1. in BiDirDijkstra::initall I add: m_vecNodeVector.reserve(maxNode
+ 1);
2. in BiDirDijkstra::bidir_dijkstra I move the call to
initall(maxNode); to before construct_graph(edges, edge_count,
maxNode);
This does two major things:
1. std::vector doubles the size of the array every time it needs to
increase its size and then needs to copy the old data to the new
area.
2. reserve() pre allocates all the memory we need once, which avoid
realloc memory fragmentation and avoids the copy each time it
reallocs so this improves performance.
Anytime you know how much space you are going to need you should
reserve it up front.
So things look good for now.
Thanks, -Steve _______________________________________________
pgrouting-dev mailing list pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
_______________________________________________ pgrouting-dev mailing
list pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
On Wed, Jun 5, 2013 at 5:34 AM, Stephen Woodbridge
<woodbri@swoodbridge.com>wrote:
On 6/4/2013 3:53 PM, Alec Gosse wrote:
Hello,
I've just pulled from the develop branch and built fresh and I'm
getting a server crash from pgr_bdastar. If I run exactly the same
thing using pgr_astar, it works. This is using postgres 9.2.4 and
postgis 2.0.3 from Homebrew on Mac 10.8.3
I'm using pg9.2.4 and postgis 2.0.x or 2.1.0beta3dev and I fixed these
issues I thought. Evidently not.
I'm still working on a bulk routing function using bdastar, but just
realized that the regular version was crashing without my
modifications. I'm happy to try to track this down, but have little
knowledge of how to debug within postgresql. Any pointers would be
appreciated.
OK, start by uncommenting #define DEBUG 1 in both the C and C++ code. Then
add more DBG('I got here message\n"). In the C++ code these messages get
written to a file '/tmp/sew-debug' and you can do a tail -f on that in
another window to see your progress.
This is likely a issue in the C++ code but it may also be something
systemic about the way we are integrating code. If you build your own
postgresql you might want to rebuild it with --enable-cassert option to
configure which will put the database backend into a more rigorous memory
check mode.
That is a start, I would be appy to help more if I can.
Hi Steve,
I have started a Wiki page to collect that kind of information, because
that's easier to find someday than a years old email thread:
https://github.com/pgRouting/pgrouting/wiki/Developer---How-to-debug
I have copied there a few collected notes.
Daniel
--
Georepublic UG & Georepublic Japan
eMail: daniel.kastl@georepublic.de
Web: http://georepublic.de
Daniel,
Thanks, I just added some more content to that.
-Steve
On 6/4/2013 9:48 PM, Daniel Kastl wrote:
On Wed, Jun 5, 2013 at 5:34 AM, Stephen Woodbridge
<woodbri@swoodbridge.com <mailto:woodbri@swoodbridge.com>> wrote:
On 6/4/2013 3:53 PM, Alec Gosse wrote:
Hello,
I've just pulled from the develop branch and built fresh and I'm
getting a server crash from pgr_bdastar. If I run exactly the same
thing using pgr_astar, it works. This is using postgres 9.2.4 and
postgis 2.0.3 from Homebrew on Mac 10.8.3
I'm using pg9.2.4 and postgis 2.0.x or 2.1.0beta3dev and I fixed
these issues I thought. Evidently not.
I'm still working on a bulk routing function using bdastar, but just
realized that the regular version was crashing without my
modifications. I'm happy to try to track this down, but have little
knowledge of how to debug within postgresql. Any pointers would be
appreciated.
OK, start by uncommenting #define DEBUG 1 in both the C and C++
code. Then add more DBG('I got here message\n"). In the C++ code
these messages get written to a file '/tmp/sew-debug' and you can do
a tail -f on that in another window to see your progress.
This is likely a issue in the C++ code but it may also be something
systemic about the way we are integrating code. If you build your
own postgresql you might want to rebuild it with --enable-cassert
option to configure which will put the database backend into a more
rigorous memory check mode.
That is a start, I would be appy to help more if I can.
Hi Steve,
I have started a Wiki page to collect that kind of information, because
that's easier to find someday than a years old email thread:
https://github.com/pgRouting/pgrouting/wiki/Developer---How-to-debug
I have copied there a few collected notes.
Daniel
--
Georepublic UG & Georepublic Japan
eMail: daniel.kastl@georepublic.de <mailto:daniel.kastl@georepublic.de>
Web: http://georepublic.de/>
_______________________________________________
pgrouting-dev mailing list
pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
It would seem that it was simply running out of memory due to giant default vector resize requests. Adding
m_vecEdgeVector.reserve(edge_count);
before the loop calling addEdge(..) in the construct_graph function seems to do the trick. Probably best to do m_vecNodeVector.reserve(maxNode) as well, just for good measure;
I recall seeing something similar to this discussed, but perhaps it was somewhere else in the code.
Alec
On Jun 4, 2013, at 4:34 PM, Stephen Woodbridge <woodbri@swoodbridge.com> wrote:
On 6/4/2013 3:53 PM, Alec Gosse wrote:
Hello,
I've just pulled from the develop branch and built fresh and I'm
getting a server crash from pgr_bdastar. If I run exactly the same
thing using pgr_astar, it works. This is using postgres 9.2.4 and
postgis 2.0.3 from Homebrew on Mac 10.8.3
I'm using pg9.2.4 and postgis 2.0.x or 2.1.0beta3dev and I fixed these issues I thought. Evidently not.
I'm still working on a bulk routing function using bdastar, but just
realized that the regular version was crashing without my
modifications. I'm happy to try to track this down, but have little
knowledge of how to debug within postgresql. Any pointers would be
appreciated.
OK, start by uncommenting #define DEBUG 1 in both the C and C++ code. Then add more DBG('I got here message\n"). In the C++ code these messages get written to a file '/tmp/sew-debug' and you can do a tail -f on that in another window to see your progress.
This is likely a issue in the C++ code but it may also be something systemic about the way we are integrating code. If you build your own postgresql you might want to rebuild it with --enable-cassert option to configure which will put the database backend into a more rigorous memory check mode.
That is a start, I would be appy to help more if I can.
Thanks,
-Steve
Best, Alec
On Jun 1, 2013, at 2:13 PM, Stephen Woodbridge
<woodbri@swoodbridge.com> wrote:
Hi all,
I think I have just submitted changes to the bidirectional dijkstra
and astar routines the resolve the server crashes we were seeing.
This closes on major outstanding bug against 2.0.
Razequl,
I made two simple changes to your code BiDirDijkstra.cpp and the
astar version:
1. in BiDirDijkstra::initall I add: m_vecNodeVector.reserve(maxNode
+ 1);
2. in BiDirDijkstra::bidir_dijkstra I move the call to
initall(maxNode); to before construct_graph(edges, edge_count,
maxNode);
This does two major things:
1. std::vector doubles the size of the array every time it needs to
increase its size and then needs to copy the old data to the new
area.
2. reserve() pre allocates all the memory we need once, which avoid
realloc memory fragmentation and avoids the copy each time it
reallocs so this improves performance.
Anytime you know how much space you are going to need you should
reserve it up front.
So things look good for now.
Thanks, -Steve _______________________________________________
pgrouting-dev mailing list pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
_______________________________________________ pgrouting-dev mailing
list pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
_______________________________________________
pgrouting-dev mailing list
pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
Alec,
Yes, that is the change I made also. I thought I merged that into the develop, but maybe a missed some of them.
I will look at this again and make sure that is added in branch "develop" which is the current active development branch.
Thank you,
-Steve
On 6/5/2013 10:13 AM, Alec Gosse wrote:
It would seem that it was simply running out of memory due to giant
default vector resize requests. Adding
m_vecEdgeVector.reserve(edge_count); before the loop calling
addEdge(..) in the construct_graph function seems to do the trick.
Probably best to do m_vecNodeVector.reserve(maxNode) as well, just
for good measure;
I recall seeing something similar to this discussed, but perhaps it
was somewhere else in the code.
Alec
On Jun 4, 2013, at 4:34 PM, Stephen Woodbridge
<woodbri@swoodbridge.com> wrote:
On 6/4/2013 3:53 PM, Alec Gosse wrote:
Hello,
I've just pulled from the develop branch and built fresh and I'm
getting a server crash from pgr_bdastar. If I run exactly the
same thing using pgr_astar, it works. This is using postgres
9.2.4 and postgis 2.0.3 from Homebrew on Mac 10.8.3
I'm using pg9.2.4 and postgis 2.0.x or 2.1.0beta3dev and I fixed
these issues I thought. Evidently not.
I'm still working on a bulk routing function using bdastar, but
just realized that the regular version was crashing without my
modifications. I'm happy to try to track this down, but have
little knowledge of how to debug within postgresql. Any pointers
would be appreciated.
OK, start by uncommenting #define DEBUG 1 in both the C and C++
code. Then add more DBG('I got here message\n"). In the C++ code
these messages get written to a file '/tmp/sew-debug' and you can
do a tail -f on that in another window to see your progress.
This is likely a issue in the C++ code but it may also be something
systemic about the way we are integrating code. If you build your
own postgresql you might want to rebuild it with --enable-cassert
option to configure which will put the database backend into a more
rigorous memory check mode.
That is a start, I would be appy to help more if I can.
Thanks, -Steve
Best, Alec
On Jun 1, 2013, at 2:13 PM, Stephen Woodbridge
<woodbri@swoodbridge.com> wrote:
Hi all,
I think I have just submitted changes to the bidirectional
dijkstra and astar routines the resolve the server crashes we
were seeing. This closes on major outstanding bug against 2.0.
Razequl,
I made two simple changes to your code BiDirDijkstra.cpp and
the astar version:
1. in BiDirDijkstra::initall I add:
m_vecNodeVector.reserve(maxNode + 1);
2. in BiDirDijkstra::bidir_dijkstra I move the call to
initall(maxNode); to before construct_graph(edges, edge_count,
maxNode);
This does two major things:
1. std::vector doubles the size of the array every time it
needs to increase its size and then needs to copy the old data
to the new area.
2. reserve() pre allocates all the memory we need once, which
avoid realloc memory fragmentation and avoids the copy each
time it reallocs so this improves performance.
Anytime you know how much space you are going to need you
should reserve it up front.
So things look good for now.
Thanks, -Steve _______________________________________________
pgrouting-dev mailing list pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
_______________________________________________ pgrouting-dev
mailing list pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
_______________________________________________ pgrouting-dev
mailing list pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
_______________________________________________ pgrouting-dev mailing
list pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
Hi Alec,
What branch are you working off of? sew-devel-2_0 or develop?
Because I checked in those fixes 4 days ago:
https://github.com/pgRouting/pgrouting/blob/develop/src/bd_astar/src/BiDirAStar.cpp
Look at line 72
Please make sure you are using branch "develop"
Thanks,
-Steve
On 6/5/2013 10:13 AM, Alec Gosse wrote:
It would seem that it was simply running out of memory due to giant default vector resize requests. Adding
m_vecEdgeVector.reserve(edge_count);
before the loop calling addEdge(..) in the construct_graph function seems to do the trick. Probably best to do m_vecNodeVector.reserve(maxNode) as well, just for good measure;
I recall seeing something similar to this discussed, but perhaps it was somewhere else in the code.
Alec
On Jun 4, 2013, at 4:34 PM, Stephen Woodbridge <woodbri@swoodbridge.com> wrote:
On 6/4/2013 3:53 PM, Alec Gosse wrote:
Hello,
I've just pulled from the develop branch and built fresh and I'm
getting a server crash from pgr_bdastar. If I run exactly the same
thing using pgr_astar, it works. This is using postgres 9.2.4 and
postgis 2.0.3 from Homebrew on Mac 10.8.3
I'm using pg9.2.4 and postgis 2.0.x or 2.1.0beta3dev and I fixed these issues I thought. Evidently not.
I'm still working on a bulk routing function using bdastar, but just
realized that the regular version was crashing without my
modifications. I'm happy to try to track this down, but have little
knowledge of how to debug within postgresql. Any pointers would be
appreciated.
OK, start by uncommenting #define DEBUG 1 in both the C and C++ code. Then add more DBG('I got here message\n"). In the C++ code these messages get written to a file '/tmp/sew-debug' and you can do a tail -f on that in another window to see your progress.
This is likely a issue in the C++ code but it may also be something systemic about the way we are integrating code. If you build your own postgresql you might want to rebuild it with --enable-cassert option to configure which will put the database backend into a more rigorous memory check mode.
That is a start, I would be appy to help more if I can.
Thanks,
-Steve
Best, Alec
On Jun 1, 2013, at 2:13 PM, Stephen Woodbridge
<woodbri@swoodbridge.com> wrote:
Hi all,
I think I have just submitted changes to the bidirectional dijkstra
and astar routines the resolve the server crashes we were seeing.
This closes on major outstanding bug against 2.0.
Razequl,
I made two simple changes to your code BiDirDijkstra.cpp and the
astar version:
1. in BiDirDijkstra::initall I add: m_vecNodeVector.reserve(maxNode
+ 1);
2. in BiDirDijkstra::bidir_dijkstra I move the call to
initall(maxNode); to before construct_graph(edges, edge_count,
maxNode);
This does two major things:
1. std::vector doubles the size of the array every time it needs to
increase its size and then needs to copy the old data to the new
area.
2. reserve() pre allocates all the memory we need once, which avoid
realloc memory fragmentation and avoids the copy each time it
reallocs so this improves performance.
Anytime you know how much space you are going to need you should
reserve it up front.
So things look good for now.
Thanks, -Steve _______________________________________________
pgrouting-dev mailing list pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
_______________________________________________ pgrouting-dev mailing
list pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
_______________________________________________
pgrouting-dev mailing list
pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev
_______________________________________________
pgrouting-dev mailing list
pgrouting-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/pgrouting-dev