[GeoNetwork-devel] Agenda for GeoNetwork hacking in Bolsena &release planning [SEC=UNCLASSIFIED]

Hi Jeroen,
I would like to see discussion on:

1) Unit tests

GeoNetwork currently has an old version of junit.jar (version 2?) in
web/geonetwork/WEB-INF/lib. It's not used. Could the group discuss the
proposal that a) unit testing using JUnit be adopted b) JUnit version 4.x
(latest) version be adopted.

GeoNetwork is based on Java 1.5. The good news is that this allows
annotations. Would anybody be terribly upset if a /test directory was added
to the project and that future work included creating applicable unit tests
using JUnit annotated test cases?

2) Adoption of a persistence framework

As of version 2.2.0 the GeoNetwork application cannot be deployed to a
cluster. Existing deployments probably haven't gotten to the size where
clustering is necessary, but if this were to happen, deployment to a cluster
will fail.

There are several reasons for this. Firstly, the application is storing
non-Serializable objects in the HttpSession. Not terribly difficult to fix
but is still a show stopper.

Secondly, and this is the real killer, the current mechanism of generating
unique primary keys in jeeves.util.SerialFactory will fail in a cluster due
to duplicate primary keys. The SerialFactory caches the max primary key
values for each table. In a cluster multiple SerialFactory instances will
exist and are oblivious of each other. The first node to insert a record will
succeed, other nodes will fail.

Geoscience Australia has deployed GeoNetwork using Oracle. The correct way to
deal with this in Oracle is to use a SEQUENCE. This requires generating
Oracle specific SQL, something the project has avoided doing.

In my humble opinion, if GeoNetwork is to achieve its full potential it needs
to be scalable. Issues like in memory key generation prevent this from
occurring. The bottom line is you need to need to be DB independent but
scalable. The project should seriously consider the adoption of a persistence
framework such as Hibernate.

That's my 20 cents worth,
Stephen Davies

-----Original Message-----
From: geonetwork-devel-bounces@lists.sourceforge.net
[mailto:geonetwork-devel-bounces@lists.sourceforge.net] On Behalf Of Jeroen
Ticheler
Sent: Saturday, 7 June 2008 12:53
To: geonetwork-devel@lists.sourceforge.net
Subject: [GeoNetwork-devel] Agenda for GeoNetwork hacking in Bolsena &release
planning

Hi all,
16-22 June we'll meet with quite some of the GeoNetwork developers in
Bolsena. I wanted to start making an agenda for that week and also see
what can be done ahead to make the meeting more efficient and give
others that are not in Bolsena a chance to provide their inputs early
on.

We have a range of projects lined up around the world that plan
developing functionality and we have to make sure these are aligned as
much as possible or otherwise have a clear strategy on what, where and
when functionality comes together.

At present I see two major releases we have to plan for, being version
2.3 and version 3.0. Version 2.3 could be released by 25 September
(before FOSS4G2008 starting 29 Sep in Cape Town), version 3.0 by March
2009. We may discuss a version 2.4 release if 3.0 is to risky for some
or there are functions that can't be finished before 2.3 and can't be
ported into 3.0 in time.

I would like v2.3 to be finished before FOSS4G 2008

Rough plans for the versions (with surely many things missing and I'm
also suggesting some directions, please add if I didn't add yours):

v2.3
- CSW 2.0.2 ISO
- W*S harvesting
- GeoServer integration with direct deployment of W*S services on
upload of shapefiles and geotiffs
- Improved metadata editor (possibly in two flavors)
- First cut GUI plugin support

v3.0
- ebRIM CSW ISO support
- Application Profile plugin support
- upgraded web map client (discussion is needed here, using Open
Layers in combination with EXTjs may be the logical way to go. Also
with MapFish and with GeoServer GUI choices in mind).
- Search interface (probably rewritten from scratch) to be more
modular, removing a.o. tables, switching JavaScript API to EXTjs and
GUI themes as module support.

For the agenda I will put a table on the developer WIKI that we can
fill in further.

Looking forward to responses & ideas!
Ciao,
Jeroen

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

Hi Stephen,
Thanks for your points! I'll put those on the agenda. JUnit is for sure a requirement that has to be covered in future development.

On hybernate as a persistence layer, it would be good to know what people have been doing so far? Should we add a hybernate datastore to GeoTools, or is it already there?

Did any of you already look into the capacities of MAJAS Hybernate? (http://www.cadrie.com/). From the website: "an extension to the Hibernate Object-Relational Mapping solution, enabling it to deal transparently with geographic data"

Cheers,
Jeroen

On Jun 11, 2008, at 7:03 AM, Stephen.Davies@anonymised.com wrote:

Hi Jeroen,
I would like to see discussion on:

1) Unit tests

GeoNetwork currently has an old version of junit.jar (version 2?) in
web/geonetwork/WEB-INF/lib. It's not used. Could the group discuss the
proposal that a) unit testing using JUnit be adopted b) JUnit version 4.x
(latest) version be adopted.

GeoNetwork is based on Java 1.5. The good news is that this allows
annotations. Would anybody be terribly upset if a /test directory was added
to the project and that future work included creating applicable unit tests
using JUnit annotated test cases?

2) Adoption of a persistence framework

As of version 2.2.0 the GeoNetwork application cannot be deployed to a
cluster. Existing deployments probably haven't gotten to the size where
clustering is necessary, but if this were to happen, deployment to a cluster
will fail.

There are several reasons for this. Firstly, the application is storing
non-Serializable objects in the HttpSession. Not terribly difficult to fix
but is still a show stopper.

Secondly, and this is the real killer, the current mechanism of generating
unique primary keys in jeeves.util.SerialFactory will fail in a cluster due
to duplicate primary keys. The SerialFactory caches the max primary key
values for each table. In a cluster multiple SerialFactory instances will
exist and are oblivious of each other. The first node to insert a record will
succeed, other nodes will fail.

Geoscience Australia has deployed GeoNetwork using Oracle. The correct way to
deal with this in Oracle is to use a SEQUENCE. This requires generating
Oracle specific SQL, something the project has avoided doing.

In my humble opinion, if GeoNetwork is to achieve its full potential it needs
to be scalable. Issues like in memory key generation prevent this from
occurring. The bottom line is you need to need to be DB independent but
scalable. The project should seriously consider the adoption of a persistence
framework such as Hibernate.

That's my 20 cents worth,
Stephen Davies

-----Original Message-----
From: geonetwork-devel-bounces@lists.sourceforge.net
[mailto:geonetwork-devel-bounces@lists.sourceforge.net] On Behalf Of Jeroen
Ticheler
Sent: Saturday, 7 June 2008 12:53
To: geonetwork-devel@lists.sourceforge.net
Subject: [GeoNetwork-devel] Agenda for GeoNetwork hacking in Bolsena &release
planning

Hi all,
16-22 June we'll meet with quite some of the GeoNetwork developers in
Bolsena. I wanted to start making an agenda for that week and also see
what can be done ahead to make the meeting more efficient and give
others that are not in Bolsena a chance to provide their inputs early
on.

We have a range of projects lined up around the world that plan
developing functionality and we have to make sure these are aligned as
much as possible or otherwise have a clear strategy on what, where and
when functionality comes together.

At present I see two major releases we have to plan for, being version
2.3 and version 3.0. Version 2.3 could be released by 25 September
(before FOSS4G2008 starting 29 Sep in Cape Town), version 3.0 by March
2009. We may discuss a version 2.4 release if 3.0 is to risky for some
or there are functions that can't be finished before 2.3 and can't be
ported into 3.0 in time.

I would like v2.3 to be finished before FOSS4G 2008

Rough plans for the versions (with surely many things missing and I'm
also suggesting some directions, please add if I didn't add yours):

v2.3
- CSW 2.0.2 ISO
- W*S harvesting
- GeoServer integration with direct deployment of W*S services on
upload of shapefiles and geotiffs
- Improved metadata editor (possibly in two flavors)
- First cut GUI plugin support

v3.0
- ebRIM CSW ISO support
- Application Profile plugin support
- upgraded web map client (discussion is needed here, using Open
Layers in combination with EXTjs may be the logical way to go. Also
with MapFish and with GeoServer GUI choices in mind).
- Search interface (probably rewritten from scratch) to be more
modular, removing a.o. tables, switching JavaScript API to EXTjs and
GUI themes as module support.

For the agenda I will put a table on the developer WIKI that we can
fill in further.

Looking forward to responses & ideas!
Ciao,
Jeroen

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Jeroen Ticheler wrote:

Hi Stephen,

Thanks for your points! I'll put those on the agenda. JUnit is for sure a requirement that has to be covered in future development.

On hybernate as a persistence layer, it would be good to know what people have been doing so far?

GeoServer just transitioned to a new config system; rather than depend directly on hibernate they a formal API. With an implementation for xstream and another making use of hibernate.

Should we add a hybernate datastore to GeoTools, or is it already there?
  

Andrea and myself (mostly Andrea) have implemented a hibernate datastore for a commercial client and were unable to "open source it". There is a large amount of overlap between using hibernate mappings (to capture relationships) and using a feature model to capture relationships. A hibernate datastore in geotools makes sense if your Pojos are Features....

My recommendation is to use Hibernate as is; and write an adapter to features for a first cut. You only need the adapter if you need GeoTools / GeoServer to render the content...

Did any of you already look into the capacities of MAJAS Hybernate? (http://www.cadrie.com/ ). From the website: "an extension to the Hibernate Object-Relational Mapping solution, enabling it to deal transparently with geographic data"
  

There are several projects along those lines; we had the JPOX project work closely with us for a while.
Jody

Hi Jeroen & Stephen
On mer, 2008-06-11 at 09:49 +0200, Jeroen Ticheler wrote:

Hi Stephen,
Thanks for your points! I'll put those on the agenda. JUnit is for
sure a requirement that has to be covered in future development.

JUnit is for sure an option for the java part, but then do you have any
recommendations to be able to unit test on the web interface also ?

I had a look to selenium some month ago:
http://selenium.openqa.org/ ... that could be an option.

other ideas ?

Ciao. Francois

Hi Stephen, one question on cluster deployment,

On mer, 2008-06-11 at 15:03 +1000, Stephen.Davies@anonymised.com wrote:

As of version 2.2.0 the GeoNetwork application cannot be deployed to a
cluster. Existing deployments probably haven't gotten to the size where
clustering is necessary, but if this were to happen, deployment to a cluster
will fail.
Secondly, and this is the real killer, the current mechanism of generating
unique primary keys in jeeves.util.SerialFactory will fail in a cluster due
to duplicate primary keys. The SerialFactory caches the max primary key
values for each table. In a cluster multiple SerialFactory instances will
exist and are oblivious of each other. The first node to insert a record will
succeed, other nodes will fail.

could we use safely the uuid generated by a UUID.randomUUID() as the
primary key for the metadata (we also use that uuid to have direct
access to metadata in the webinterface).
Jeroen added a unique constraint on that field some weeks ago:
http://www.nabble.com/SF.net-SVN%3A-geonetwork%
3A--1311--branches-2.2.x-gast-setup-sql-td17561379.html (this has not
been commmitted to trunk, only to branch 2.2.x, should we align trunk on
that Jeroen ?).

If that situation is safe in a cluster environment, it could be an
option to use the uuid as the internal identifier for GeoNetwork instead
of Jeeves's serialFactory ?

Geoscience Australia has deployed GeoNetwork using Oracle. The correct way to
deal with this in Oracle is to use a SEQUENCE. This requires generating
Oracle specific SQL, something the project has avoided doing.

Cheers. Francois