[Geoserver-devel] Proposed solution for SOS 2.0

[VT1]

Hi, I have drafted out a proposed solution based on JAX-B. Highsource have put in great effort in binding ogc schemas to jax-b and have added support for sos 2.0 and om 2.0. They are recommended by ogcnetwork.

Please do take a look and give your opinion.

We intend to create sos as a webservice on Geoserver, something similar to the recent work on CSW

The proposed solution in delivering the service are documented here

https://www.seegrid.csiro.au/wiki/SISS4BoM/SOSProposedModule

Proposed Solution for SOS 2.0 Development:## Introduction

This page is a description of a proposed solution to deliver SOS 2.0. A summary of the main functional criteria for this project are as follow:

• Easy to configure for most/basic usage
• Support multiple databases
• Modular enough to support evolutions of schemas
• For the current project we aim to support sos 2.0 with WaterML? but code should be extensible to support other result type such as SWE
• Deliver valid SOS 2.0 service.

Overview

Diagram: GetObservationResponse? example

Explaination

The solution to this implementation lies on the use of jax-b framework. There are currently an existing community that have taken the effort to create all the xjc binding for most of the ogc schemas.

http://www.ogcnetwork.net/jaxb4ogc

http://confluence.highsource.org/display/OGCS/Reference

and SOS 2.0 with OM 2.0 have recently been added to their development trunk.

Based on the diagram, that boils our work down to the following

• Create a user UI to map configuration to om:observation
• Create a user UI to map database field to WaterML? schema
• Identify the JAX-B entity classes to populate for marshalling the class out as response.
• Integrate to GeoServer

WorkFlow#### GeoServer? workflow

Workflow for GeoServer? is simple. Detect community module for SOS support. If found, load the sos config tab else resume normal GeoServer? Interface.

User workflow

Note: we are currently still designing how the UI will look like but the WaterML? model configuration UI will be similar to the simple feature UI except it will have a addtional drop down box alongside to map the database field to the right schema element.

We are restricting the users to certain models that we support for om:result. eg. WaterML?. If the user decide to use predefined model there are certain advantage and disadvantage

Advantage:

• faster performance on filtering since filtering will be performed on the database prior to parsing into a Document.
• Easier to use and manage since configuration are done via UI
• Other model for the om:result can be support by adding the right UI for them (might be possible to auto generate via reflection or custom annotation)

Disadvantage:

• WaterML are confined to a a single table in the database. Multi tables per model is not support (use app-schema if necessary but should be avoided)
• WaterML mapping are fixed therefore user lose the flexibility.

Alternatively if the configuration is something even more complex, user can opt for app-schema but bare the cost of performance and configuration learning curve.

Summary

Currently we used GetObservationResponse? as an example but the same architecture will apply to other domain such as getCapabilities etc.

Outstanding issues- How will this service be intergrated into GeoServer? as a web service. (Currently investigating)

• I am unsure how SOS 3.0 , om 3.0 will affect this architecture but it really depend on how big the changes are. Do we abstract out the sos 2.0 entites class and try to make it work with 3.0 or do we redo what we have done as a seperate module reusing common code as libraries.

Victor Tey

Software Engineer
ASRDC
CSIRO Earth Science and Resource Engineering

Phone: +61 8 6436 8944
Victor.Tey@anonymised.com | www.csiro.au |
Address: Australian Resources Research Centre, 26 Dick Perry Avenue, Kensington WA 6151

PLEASE NOTE
The information contained in this email may be confidential or privileged. Any unauthorised use or disclosure is prohibited. If you have received this email in error, please delete it immediately and notify the sender by return email. Thank you. To the extent permitted by law, CSIRO does not represent, warrant and/or guarantee that the integrity of this communication has been maintained or that the communication is free of errors, virus, interception or interference.

Please consider the environment before printing this email.

[jive]

We have previously had trouble integrating JAX-B XML stack, in particular with GML content which is often more flexible then the JAX-B code generation cycle.
I generally like the approach when compiling a one off client to work with a specific server.

In your case you may indeed be successful since you are targeting WaterML specifically. I am weary of the thought of two copies of the core OGC schemas lurking in our code base.

Can you review how the recent work on CSW makes use of XML binding (rather than JAX-B) and let us know what you think.

Jody Garnett

On Tuesday, 25 September 2012 at 7:14 PM, Victor.Tey@anonymised.com wrote:

Hi, I have drafted out a proposed solution based on JAX-B. Highsource have put in great effort in binding ogc schemas to jax-b and have added support for sos 2.0 and om 2.0. They are recommended by ogcnetwork.

Please do take a look and give your opinion.

We intend to create sos as a webservice on Geoserver, something similar to the recent work on CSW

The proposed solution in delivering the service are documented here

https://www.seegrid.csiro.au/wiki/SISS4BoM/SOSProposedModule

Proposed Solution for SOS 2.0 Development:## Introduction

This page is a description of a proposed solution to deliver SOS 2.0. A summary of the main functional criteria for this project are as follow:

• Easy to configure for most/basic usage
• Support multiple databases
• Modular enough to support evolutions of schemas
• For the current project we aim to support sos 2.0 with WaterML? but code should be extensible to support other result type such as SWE
• Deliver valid SOS 2.0 service.

Overview

Diagram: GetObservationResponse? example

Explaination

The solution to this implementation lies on the use of jax-b framework. There are currently an existing community that have taken the effort to create all the xjc binding for most of the ogc schemas.

http://www.ogcnetwork.net/jaxb4ogc

http://confluence.highsource.org/display/OGCS/Reference

and SOS 2.0 with OM 2.0 have recently been added to their development trunk.

Based on the diagram, that boils our work down to the following

• Create a user UI to map configuration to om:observation
• Create a user UI to map database field to WaterML? schema
• Identify the JAX-B entity classes to populate for marshalling the class out as response.
• Integrate to GeoServer

WorkFlow#### GeoServer? workflow

Workflow for GeoServer? is simple. Detect community module for SOS support. If found, load the sos config tab else resume normal GeoServer? Interface.

User workflow

Note: we are currently still designing how the UI will look like but the WaterML? model configuration UI will be similar to the simple feature UI except it will have a addtional drop down box alongside to map the database field to the right schema element.

We are restricting the users to certain models that we support for om:result. eg. WaterML?. If the user decide to use predefined model there are certain advantage and disadvantage

Advantage:

• faster performance on filtering since filtering will be performed on the database prior to parsing into a Document.
• Easier to use and manage since configuration are done via UI
• Other model for the om:result can be support by adding the right UI for them (might be possible to auto generate via reflection or custom annotation)

Disadvantage:

• WaterML are confined to a a single table in the database. Multi tables per model is not support (use app-schema if necessary but should be avoided)
• WaterML mapping are fixed therefore user lose the flexibility.

Alternatively if the configuration is something even more complex, user can opt for app-schema but bare the cost of performance and configuration learning curve.

Summary

Currently we used GetObservationResponse? as an example but the same architecture will apply to other domain such as getCapabilities etc.

Outstanding issues- How will this service be intergrated into GeoServer? as a web service. (Currently investigating)

• I am unsure how SOS 3.0 , om 3.0 will affect this architecture but it really depend on how big the changes are. Do we abstract out the sos 2.0 entites class and try to make it work with 3.0 or do we redo what we have done as a seperate module reusing common code as libraries.

Victor Tey

Software Engineer
ASRDC
CSIRO Earth Science and Resource Engineering

Phone: +61 8 6436 8944
Victor.Tey@anonymised.com | www.csiro.au |
Address: Australian Resources Research Centre, 26 Dick Perry Avenue, Kensington WA 6151

PLEASE NOTE
The information contained in this email may be confidential or privileged. Any unauthorised use or disclosure is prohibited. If you have received this email in error, please delete it immediately and notify the sender by return email. Thank you. To the extent permitted by law, CSIRO does not represent, warrant and/or guarantee that the integrity of this communication has been maintained or that the communication is free of errors, virus, interception or interference.

Please consider the environment before printing this email.

---

Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

---

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

[Andrea_Aime4]

On Wed, Sep 26, 2012 at 8:33 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

We have previously had trouble integrating JAX-B XML stack, in particular with GML content which is often more flexible then the JAX-B code generation cycle.
I generally like the approach when compiling a one off client to work with a specific server.

In your case you may indeed be successful since you are targeting WaterML specifically. I am weary of the thought of two copies of the core OGC schemas lurking in our code base.

We are also starting to have serious problems permgen wise because over time people carelessly added
more and more dependencies, without a properly setup JVM GeoServer today won’t start up or if it does
it won’t last more than 15 minutes before getting into a permgen OOM (see also my mails about the RC3
release and windows server users complaning they had to restart GS 2.2.x multiple times a day before
we also fixed the service installers to give the JVM more permgen).

Adding more dependencies should be done quite carefully.

Cheers
Andrea

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

---

[VT1]

I will try and break down points of concerns and address them.

1. We have previously had trouble integrating JAX-B XML stack, in particular with GML content which is often more flexible then the JAX-B code generation cycle.

VT: High source have done this for us. Furthermore we are only using sos 2.0 implementation

2. I am weary of the thought of two copies of the core OGC schemas lurking in our code base.

VT: No xsds will be added. HighSource have provide the class files for the schemas

3. Adding more dependencies should be done quite carefully

This will be a community module and I don’t expect the size of it to be big. There also won’t be a dependency from core to this module and dependency of this module should be relatively small since we are only using the SOS 2.0 schema binding.

Below are some examples from http://confluence.highsource.org/display/OGCS/Reference

To use the wms binding,

org.jvnet.ogc

wms-v_1_3_0-schema

1.0.0

Unmarshalling

// Create JAXB context for WMS 1.3.0

JAXBContext context = JAXBContext

                 .newInstance("net.opengis.wms.v_1_3_0");

// Use the created JAXB context to construct an unmarshaller

Unmarshaller unmarshaller = context.createUnmarshaller();

// GetCapabilities URL of the Demis WorldMap WMS Server

String url = "http://www2.demis.nl/wms/wms.asp?REQUEST=GetCapabilities&VERSION=1.3.0&wms=WorldMap";

// Unmarshal the given URL, retrieve WMSCapabilities element

JAXBElement<WMSCapabilities> wmsCapabilitiesElement = unmarshaller

                 .unmarshal(new StreamSource(url), WMSCapabilities.class);

// Retrieve WMSCapabilities instance

WMSCapabilities wmsCapabilities = wmsCapabilitiesElement.getValue();

// Iterate over layers, print out layer names

for (Layer layer : wmsCapabilities.getCapability().getLayer()

                 .getLayer()) {

         System.out.println(layer.getName());

**Creating Objects:**

private net.opengis.wms.v_1_3_0.ObjectFactory objectFactory =

         new net.opengis.wms.v_1_3_0.ObjectFactory();

private WMSCapabilities createWMSCapabilities() {

         final WMSCapabilities wmsCapabilities = objectFactory

                          .createWMSCapabilities();

         wmsCapabilities.setService(createService());

         wmsCapabilities.setCapability(createCapability());

         return wmsCapabilities;

}

private Service createService() {

         final Service service = objectFactory.createService();

         service.setName("WMS");

         service.setTitle("Dummy WMS service");

         service.setOnlineResource(createOnlineResource("http://localhost"));

         return service;

}

private Capability createCapability() {

         final Capability capability = objectFactory.createCapability();

         capability.setRequest(createRequest());

         capability.setException(createException());

         capability.setLayer(createLayer());

         return capability;

}

Marshalling:

// Construct the WMS capabilities object

WMSCapabilities wmsCapabilities = ...;

// Create a JAXB context

JAXBContext context = JAXBContext

                 .newInstance("net.opengis.wms.v_1_3_0");

// Create a marshaller

Marshaller marshaller = context.createMarshaller();

// Set a property for the formatted (indented) output

marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);

// Marshal capabilities to the system out

marshaller.marshal(wmsCapabilities, System.out);

From: Jody Garnett [mailto:jody.garnett@…403…]
Sent: Wednesday, 26 September 2012 2:34 PM
To: Tey, Victor (CESRE, Kensington)
Cc: geoserver-devel@lists.sourceforge.net
Subject: Re: [Geoserver-devel] Proposed solution for SOS 2.0

We have previously had trouble integrating JAX-B XML stack, in particular with GML content which is often more flexible then the JAX-B code generation cycle.

I generally like the approach when compiling a one off client to work with a specific server.

In your case you may indeed be successful since you are targeting WaterML specifically. I am weary of the thought of two copies of the core OGC schemas lurking in our code base.

Can you review how the recent work on CSW makes use of XML binding (rather than JAX-B) and let us know what you think.

–

Jody Garnett

On Tuesday, 25 September 2012 at 7:14 PM, Victor.Tey@…254… wrote:

Hi, I have drafted out a proposed solution based on JAX-B. Highsource have put in great effort in binding ogc schemas to jax-b and have added support for sos 2.0 and om 2.0. They are recommended by ogcnetwork.

Please do take a look and give your opinion.

We intend to create sos as a webservice on Geoserver, something similar to the recent work on CSW

The proposed solution in delivering the service are documented here

https://www.seegrid.csiro.au/wiki/SISS4BoM/SOSProposedModule

Proposed Solution for SOS 2.0 Development:## Introduction

This page is a description of a proposed solution to deliver SOS 2.0. A summary of the main functional criteria for this project are as follow:

• Easy to configure for most/basic usage
• Support multiple databases
• Modular enough to support evolutions of schemas
• For the current project we aim to support sos 2.0 with WaterML? but code should be extensible to support other result type such as SWE
• Deliver valid SOS 2.0 service.

Overview

Diagram: GetObservationResponse? example

Explaination

The solution to this implementation lies on the use of jax-b framework. There are currently an existing community that have taken the effort to create all the xjc binding for most of the ogc schemas.

http://www.ogcnetwork.net/jaxb4ogc

http://confluence.highsource.org/display/OGCS/Reference

and SOS 2.0 with OM 2.0 have recently been added to their development trunk.

Based on the diagram, that boils our work down to the following

• Create a user UI to map configuration to om:observation
• Create a user UI to map database field to WaterML? schema
• Identify the JAX-B entity classes to populate for marshalling the class out as response.
• Integrate to GeoServer

WorkFlow#### GeoServer? workflow

Workflow for GeoServer? is simple. Detect community module for SOS support. If found, load the sos config tab else resume normal GeoServer? Interface.

User workflow

Note: we are currently still designing how the UI will look like but the WaterML? model configuration UI will be similar to the simple feature UI except it will have a addtional drop down box alongside to map the database field to the right schema element.

We are restricting the users to certain models that we support for om:result. eg. WaterML?. If the user decide to use predefined model there are certain advantage and disadvantage

Advantage:

• faster performance on filtering since filtering will be performed on the database prior to parsing into a Document.
• Easier to use and manage since configuration are done via UI
• Other model for the om:result can be support by adding the right UI for them (might be possible to auto generate via reflection or custom annotation)

Disadvantage:

• WaterML are confined to a a single table in the database. Multi tables per model is not support (use app-schema if necessary but should be avoided)
• WaterML mapping are fixed therefore user lose the flexibility.

Alternatively if the configuration is something even more complex, user can opt for app-schema but bare the cost of performance and configuration learning curve.

Summary

Currently we used GetObservationResponse? as an example but the same architecture will apply to other domain such as getCapabilities etc.

Outstanding issues- How will this service be intergrated into GeoServer? as a web service. (Currently investigating)

• I am unsure how SOS 3.0 , om 3.0 will affect this architecture but it really depend on how big the changes are. Do we abstract out the sos 2.0 entites class and try to make it work with 3.0 or do we redo what we have done as a seperate module reusing common code as libraries.

Victor Tey

Software Engineer
ASRDC
CSIRO Earth Science and Resource Engineering

Phone: +61 8 6436 8944
Victor.Tey@…254… | www.csiro.au |
Address: Australian Resources Research Centre, 26 Dick Perry Avenue, Kensington WA 6151

PLEASE NOTE
The information contained in this email may be confidential or privileged. Any unauthorised use or disclosure is prohibited. If you have received this email in error, please delete it immediately and notify the sender by return email. Thank you. To the extent permitted by law, CSIRO does not represent, warrant and/or guarantee that the integrity of this communication has been maintained or that the communication is free of errors, virus, interception or interference.

Please consider the environment before printing this email.

---

Live Security Virtual Conference

Exclusive live event will cover all the ways today’s security and

threat landscape has changed and how IT managers can respond. Discussions

will include endpoint security, mobile security and the latest in malware

threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

---

Geoserver-devel mailing list

Geoserver-devel@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/geoserver-devel

(attachments)

[Andrea_Aime4]

On Wed, Sep 26, 2012 at 9:47 AM, <Victor.Tey@anonymised.com> wrote:

I will try and break down points of concerns and address them.

1. We have previously had trouble integrating JAX-B XML stack, in particular with GML content which is often more flexible then the JAX-B code generation cycle.

VT: High source have done this for us. Furthermore we are only using sos 2.0 implementation

Unless I misunderstand, this is not an answer.
In the past adding the JAXB libraries in the classpath made WFS stop working properly.
Now JAXB is in the JDK so the solution might have been solved, but I’d invite you to look into
the GeoTools ml archives and see what exactly was not working and check if the
issue still applies.

2. I am weary of the thought of two copies of the core OGC schemas lurking in our code base.

VT: No xsds will be added. HighSource have provide the class files for the schemas

3. Adding more dependencies should be done quite carefully

This will be a community module and I don’t expect the size of it to be big. There also won’t be a dependency from core to this module and dependency of this module should be relatively small since we are only using the SOS 2.0 schema binding.

So you mean the SOS is not meant to ever be integrated into core?
If you are generating complex features and handling the SOS protocol and the like I guess you’ll need classes
representing OWS, GML, Filter, SWE and SOS, that sounds like several megabytes worth of jars.

Are you going to use them only to parse the requests or also to encode the outputs?
Is JAXB able to stream out, or do you have to first load all the beans in a in memory list and then write them out?
In other words, how do you handle a large output without going OOM?

Cheers
Andrea

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

---