[pgrouting-dev] pgRouting 2.0 Update

Stephen_Woodbridge1 · March 17, 2013, 5:02pm

Hi all,

The last couple of days has been spent on building a simple test infrastructure and a few trivial test cases. At the top level is a script test-runner.pl that at the moment does not take any arguments. It runs all the test configured tests and at the moment just dumps the results structure of the test. This can be prettied up later. It also assumes that you have installed the libraries as it tests using the installed postgresql. This probably needs to be made smarter so we can test out of the build tree. I'll need to think about that.

I already found a bug that crashes the server when building the tests. I think this might already be in the tracking system, but if you pass a negative cost to dijkstra or astar it will crash the server.

Basically each .../test/ directory should include one test.conf file that is a perl script fragment that defines what data files to load and what tests to run. I have built in some mechanisms to allow test and data to be pg version and postgis version specific, but I'm not using that yet.

core/trsp/test/test-any-00.data is a sql plain text dump that will load and needed data for a set of tests. You can specify multiple files to load, but as a group they need to have unique names.

core/trsp/test/test-any-00.test is a sql command to be run. It will get run as:
psql ... -A -t -q -f file.test dbname > tmpfile
diff file.rest tmpfile

Then if there is a difference then an test failure is reported.
At some point I would be happy if someone wants to replace this with some better infrastructure for testing, but I needed some to get things started.

I also created a top level doc directory with a README.rst Which is the start of a developers guide to the system. I will add to this as I make changes and think of things.

Testing and doc still need to be worked into the build system along with other stuff.

-Steve

strk · March 17, 2013, 6:56pm

On Sun, Mar 17, 2013 at 01:02:12PM -0400, Stephen Woodbridge wrote:

The last couple of days has been spent on building a simple test
infrastructure and a few trivial test cases.

Good work !

This probably needs to be
made smarter so we can test out of the build tree. I'll need to
think about that.

Note it'd currently be impossible to test with the "CREATE EXTENTION"
mechanism (no way to load an extension from outside the systemwide
directory).

--strk;

Stephen_Woodbridge1 · March 18, 2013, 12:22am

On 3/17/2013 2:56 PM, Sandro Santilli wrote:

On Sun, Mar 17, 2013 at 01:02:12PM -0400, Stephen Woodbridge wrote:

The last couple of days has been spent on building a simple test
infrastructure and a few trivial test cases.

Good work !

Thanks, I'm trying to evolve our tree into something that is more sustainable long term and better supports collaboration. We will see how well that works out.

I would really like to get around to hacking the code, but it seems that we need to get the infrastructure in place and it is an incredible amount of work.

This probably needs to be
made smarter so we can test out of the build tree. I'll need to
think about that.

One thing I found that might improve my perl test-runner is Test::Postgresql this doesn't solve the problem you were addressing below, but I have a thought on how to work around those issues.

Note it'd currently be impossible to test with the "CREATE EXTENTION"
mechanism (no way to load an extension from outside the systemwide
directory).

I just read the thread you started on pgsql-hackers about this issue last year. It seems that extension are version wide rather than database wide, because the current system assume everything is store in a versioned directory tree.

So if you build postgresql from src you can set the PREFIX to something like ~/pgN.N which would give us our private version install tree, as a regular user so it does not try to overwrite system files. I think you should then be able to install your staging code into this tree, start this database and test in this environment. This might require a little customizing of the things but it should allow you to have multiple versions each in their own tree that you can startup via a test script, run some tests, and shutdown. There probably needs to be some cleanup scripts to remove staged files so your environment doesn't get cruft buildup over time. This also might facilitate running postgresql in the gdb to catch server crashes and get a backtrace.

I have not tried this but it would be the direction I would pursue. I guess something similar could also be done with the system installation, but that assumes that it is not getting used for other stuff on a multi-use machine like most of my servers.

Any thoughts on this?

Are there other developers/users that would be interested in researching this and reporting back.

strk · March 18, 2013, 12:09pm

On Sun, Mar 17, 2013 at 08:22:04PM -0400, Stephen Woodbridge wrote:

On 3/17/2013 2:56 PM, Sandro Santilli wrote:

>Note it'd currently be impossible to test with the "CREATE EXTENTION"
>mechanism (no way to load an extension from outside the systemwide
>directory).

I just read the thread you started on pgsql-hackers about this issue
last year. It seems that extension are version wide rather than
database wide, because the current system assume everything is store
in a versioned directory tree.

So if you build postgresql from src you can set the PREFIX to
something like ~/pgN.N which would give us our private version
install tree, as a regular user so it does not try to overwrite
system files.

...

Any thoughts on this?

Making tests rely on a specific build of PostgreSQL doesn't seem
a good idea to me. I mean, it's the same as NOT providing pre-install
testing, and delegating to the user the installation in a staging
environment for the sole purpose of testing.

I'm not familiar with extensions, but as long as you still need
enabling and disabling scripts, can't you just always _also_ provide
those scripts and use them for setting up the testing database ?

--strk;

Stephen_Woodbridge1 · March 18, 2013, 12:37pm

On 3/18/2013 8:09 AM, Sandro Santilli wrote:

On Sun, Mar 17, 2013 at 08:22:04PM -0400, Stephen Woodbridge wrote:

On 3/17/2013 2:56 PM, Sandro Santilli wrote:

Note it'd currently be impossible to test with the "CREATE EXTENTION"
mechanism (no way to load an extension from outside the systemwide
directory).

I just read the thread you started on pgsql-hackers about this issue
last year. It seems that extension are version wide rather than
database wide, because the current system assume everything is store
in a versioned directory tree.

So if you build postgresql from src you can set the PREFIX to
something like ~/pgN.N which would give us our private version
install tree, as a regular user so it does not try to overwrite
system files.

...

Any thoughts on this?

Making tests rely on a specific build of PostgreSQL doesn't seem
a good idea to me. I mean, it's the same as NOT providing pre-install
testing, and delegating to the user the installation in a staging
environment for the sole purpose of testing.

I'm not familiar with extensions, but as long as you still need
enabling and disabling scripts, can't you just always _also_ provide
those scripts and use them for setting up the testing database ?

I'm thinking about how we can test multiple versions of postgresql and multiple versions of postgis.

So if it were possible to install multiple postgresql instances in separate trees, then you could do something like:

for each version tree
   start postgres for this tree
   cmake for this tree
   build and stage for this tree
   test on this tree
   stop postgres for this tree
endfor

May be we need to think about setting up travis-ci or jenkins testing frameworks. I might leave this to someone else to tackle if anyone is interested.

-Steve

strk · March 18, 2013, 12:45pm

On Mon, Mar 18, 2013 at 08:37:06AM -0400, Stephen Woodbridge wrote:

On 3/18/2013 8:09 AM, Sandro Santilli wrote:

>I'm not familiar with extensions, but as long as you still need
>enabling and disabling scripts, can't you just always _also_ provide
>those scripts and use them for setting up the testing database ?

I'm thinking about how we can test multiple versions of postgresql
and multiple versions of postgis.

So if it were possible to install multiple postgresql instances in
separate trees, then you could do something like:

for each version tree
  start postgres for this tree
  cmake for this tree
  build and stage for this tree
  test on this tree
  stop postgres for this tree
endfor

This would be already possible with the setup used by PostGIS,
in that "make check" relies on your environment to figure out
where to find PostgreSQL, so if you have multiple versions you
may write your script so to use each version in turn.

I'm actually a fan of distributed testing (ala buildbot) so that
each slave has a different postgresql/postgis version installed.

Overcomplex testing frameworks may discourage people from using them.

--strk;