[Geoserver-devel] Broken GetCapabilities and Translation Errors

Ariel_Nunez · September 8, 2011, 9:32pm

Hello people,

I am experiencing a problem with my GeoServer the GetCapabilities of the WMS is broken [1]. My questions are:

#1 How can I diagnose which layer is causing this? I end up with a NullPointerException [2].

#2 Why does GeoServer return an invalid XML (and XML file inside another XML file)? I remember some settings about optimizing for speed vs optimizing for ‘correctness’ in earlier versions of GeoServer but I don’t see that now [3].

#3 What changes would be needed in GeoServer to better diagnose (or if possible self correct) this problem. This is an application where GeoServer is used as a backend storage and I would like to avoid a failed layer upload via the REST API from totally breaking the server

Thanks in advance for any help you can provide, I addressed this to the dev list because I intend to follow up with a patch to come with a more helpful behavior to the GeoServer Admin / User.

Ariel

[1] Trimmed version of the output:

------------ start of the document ------------

<?xml version="1.0" encoding="UTF-8"?>

<WMT_MS_Capabilities version=“1.1.1” updateSequence=“1704”>

OGC:WMS

My GeoServer WMS

[snip … ]

EPSG:62786405
EPSG:62796405
EPSG:62806405

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
javax.xml.transform.TransformerException: Translator error
Translator error
null

-----------------End of the document -------------

[2] http://dpaste.de/1ICYK/

[3] Information about GeoServer

2.1-SNAPSHOT
16252
17-Aug-2011 00:28
2.7-SNAPSHOT (rev 37850)

Ariel_Nunez · September 8, 2011, 9:49pm

Update:

I found the layer and the UI does not let me go to the ‘Publishing’ tab:

Field ‘latLonBoundingBox’ is required.
Field ‘Declared SRS’ is required.

It was one of the layers that was uploaded programatically via the REST API, my guess is that it should be disabled by default instead of enabled and breaking the GetCaps.

Is this as a bug in the behavior of the REST API (not verifying if the layers that are configured that way can be served)?

Ariel.

On Thu, Sep 8, 2011 at 5:32 PM, Ariel Nunez <ingenieroariel@anonymised.com> wrote:

Hello people,

I am experiencing a problem with my GeoServer the GetCapabilities of the WMS is broken [1]. My questions are:

#1 How can I diagnose which layer is causing this? I end up with a NullPointerException [2].

#2 Why does GeoServer return an invalid XML (and XML file inside another XML file)? I remember some settings about optimizing for speed vs optimizing for ‘correctness’ in earlier versions of GeoServer but I don’t see that now [3].

#3 What changes would be needed in GeoServer to better diagnose (or if possible self correct) this problem. This is an application where GeoServer is used as a backend storage and I would like to avoid a failed layer upload via the REST API from totally breaking the server

Thanks in advance for any help you can provide, I addressed this to the dev list because I intend to follow up with a patch to come with a more helpful behavior to the GeoServer Admin / User.

Ariel

[1] Trimmed version of the output:

------------ start of the document ------------
<?xml version="1.0" encoding="UTF-8"?>
<WMT_MS_Capabilities version=“1.1.1” updateSequence=“1704”>

OGC:WMS
My GeoServer WMS
[snip … ]

EPSG:62786405
EPSG:62796405
EPSG:62806405

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
javax.xml.transform.TransformerException: Translator error
Translator error
null

-----------------End of the document -------------

[2] http://dpaste.de/1ICYK/

[3] Information about GeoServer

2.1-SNAPSHOT

16252

17-Aug-2011 00:28

2.7-SNAPSHOT (rev 37850)

Andrea_Aime4 · September 9, 2011, 6:37am

On Thu, Sep 8, 2011 at 11:32 PM, Ariel Nunez <ingenieroariel@anonymised.com> wrote:

Hello people,
I am experiencing a problem with my GeoServer the GetCapabilities of the WMS
is broken [1]. My questions are:
#1 How can I diagnose which layer is causing this? I end up with a
NullPointerException [2].

More recent versions of Geoserver (2.1.x series) try harder to report
the layer name
in case of failure.
If it's not there and the GS is recent please open a bug report with the full
stack trace on jira.codehaus.org

#2 Why does GeoServer return an invalid XML (and XML file inside another XML
file)? I remember some settings about optimizing for speed vs optimizing for
'correctness' in earlier versions of GeoServer but I don't see that now [3].

The setting is still there where it has always been, in the web.xml file,
with a comment explaining the possible values.
Would be nice to have it in the GUI in fact, patches welcomed

#3 What changes would be needed in GeoServer to better diagnose (or if
possible self correct) this problem. This is an application where GeoServer
is used as a backend storage and I would like to avoid a failed layer upload
via the REST API from totally breaking the server

One way would be to harden the REST api against broken input.
David also added a validator, WMSValidator, that checked for some layer
validity criteria, but it does not check for bounding boxes.

Anyways, the idea of disabling the layers might be in the opposite direction
as what GeoSolutions intends to propose once we manage to secure funding
to improve the existing fault tolerance situation.
We routinely get complaints by enterprise customers that a store or a layer
"disappeared" after a restart, normally because the data source was not
available.
What a long running, unattended server needs is to have the caps be able
to filter out of the capabilities misbehaving layers/stores
without disabling them, so that they come back online as soon as the problem
is solved without any manual intervention from the administrator.
It might be for example that we run the catalog validators against the layer
before starting to encode it in the capabilities, for example.
Another thing that we might want to do is to annotate certain fields in the
catalog info interfaces as "mandatory", and have a validator complain
if one of the fields is missing (thinking about the bbox here).

At the moment we're looking for a project to sponsor this kind of development.

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

Andrea_Aime4 · September 9, 2011, 6:38am

On Thu, Sep 8, 2011 at 11:49 PM, Ariel Nunez <ingenieroariel@anonymised.com> wrote:

Update:
I found the layer and the UI does not let me go to the 'Publishing' tab:

Field 'latLonBoundingBox' is required.
Field 'Declared SRS' is required.

It was one of the layers that was uploaded programatically via the REST API,
my guess is that it should be disabled by default instead of enabled and
breaking the GetCaps.
Is this as a bug in the behavior of the REST API (not verifying if the
layers that are configured that way can be served)?

This is intended behavior, you cannot go in the publishing tab if the
data tab is not properly filled. Please fill those two fields before moving
to the publishing tab

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

David_Winslow · September 9, 2011, 1:06pm

Some users I’ve spoken to have asked why we do the connection test at startup - they suggested that instead we skip the connections during the catalog loading and have a tool in the GUI for doing the same thing on demand, but with handy links to the broken datastores, etc. Having the capabilities document hide layers instead of generating invalid XML would be a nice step in the same direction IMHO.

Just throwing out an idea, hope it’s not too much of a tangent.

–
David Winslow
OpenGeo - http://opengeo.org/

On Fri, Sep 9, 2011 at 2:38 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Thu, Sep 8, 2011 at 11:49 PM, Ariel Nunez <ingenieroariel@anonymised.com> wrote:

Update:
I found the layer and the UI does not let me go to the ‘Publishing’ tab:

Field ‘latLonBoundingBox’ is required.
Field ‘Declared SRS’ is required.

It was one of the layers that was uploaded programatically via the REST API,
my guess is that it should be disabled by default instead of enabled and
breaking the GetCaps.
Is this as a bug in the behavior of the REST API (not verifying if the
layers that are configured that way can be served)?

This is intended behavior, you cannot go in the publishing tab if the
data tab is not properly filled. Please fill those two fields before moving
to the publishing tab

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

Why Cloud-Based Security and Archiving Make Sense
Osterman Research conducted this study that outlines how and why cloud
computing security and archiving is rapidly being adopted across the IT
space for its ease of implementation, lower cost, and increased
reliability. Learn more. http://www.accelacomm.com/jaw/sfnl/114/51425301/

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Justin_Deoliveira · September 9, 2011, 2:03pm

On Fri, Sep 9, 2011 at 7:06 AM, David Winslow <dwinslow@anonymised.com> wrote:

Some users I’ve spoken to have asked why we do the connection test at startup - they suggested that instead we skip the connections during the catalog loading and have a tool in the GUI for doing the same thing on demand, but with handy links to the broken datastores, etc. Having the capabilities document hide layers instead of generating invalid XML would be a nice step in the same direction IMHO.

http://jira.codehaus.org/browse/GEOS-3712

Just throwing out an idea, hope it’s not too much of a tangent.

I think the ideas proposed by Andrea make sense… in the short term i think an acceptable solution could be to add a flag that controls whether the capabilities doc should fail fast or just hide the misconfigured layer, replacing it later with the more adaptive approach described by Andrea.

–
David Winslow
OpenGeo - http://opengeo.org/

On Fri, Sep 9, 2011 at 2:38 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Thu, Sep 8, 2011 at 11:49 PM, Ariel Nunez <ingenieroariel@anonymised.com> wrote:

Update:
I found the layer and the UI does not let me go to the ‘Publishing’ tab:

Field ‘latLonBoundingBox’ is required.
Field ‘Declared SRS’ is required.

It was one of the layers that was uploaded programatically via the REST API,
my guess is that it should be disabled by default instead of enabled and
breaking the GetCaps.
Is this as a bug in the behavior of the REST API (not verifying if the
layers that are configured that way can be served)?

This is intended behavior, you cannot go in the publishing tab if the
data tab is not properly filled. Please fill those two fields before moving
to the publishing tab

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

Why Cloud-Based Security and Archiving Make Sense
Osterman Research conducted this study that outlines how and why cloud
computing security and archiving is rapidly being adopted across the IT
space for its ease of implementation, lower cost, and increased
reliability. Learn more. http://www.accelacomm.com/jaw/sfnl/114/51425301/

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Why Cloud-Based Security and Archiving Make Sense
Osterman Research conducted this study that outlines how and why cloud
computing security and archiving is rapidly being adopted across the IT
space for its ease of implementation, lower cost, and increased
reliability. Learn more. http://www.accelacomm.com/jaw/sfnl/114/51425301/

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Andrea_Aime4 · September 9, 2011, 2:13pm

On Fri, Sep 9, 2011 at 4:03 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

On Fri, Sep 9, 2011 at 7:06 AM, David Winslow <dwinslow@anonymised.com> wrote:

Some users I've spoken to have asked why we do the connection test at
startup - they suggested that instead we skip the connections during the
catalog loading and have a tool in the GUI for doing the same thing on
demand, but with handy links to the broken datastores, etc. Having the
capabilities document hide layers instead of generating invalid XML would be
a nice step in the same direction IMHO.

http://jira.codehaus.org/browse/GEOS-3712

Just throwing out an idea, hope it's not too much of a tangent.

I think the ideas proposed by Andrea make sense... in the short term i think
an acceptable solution could be to add a flag that controls whether the
capabilities doc should fail fast or just hide the misconfigured layer,
replacing it later with the more adaptive approach described by Andrea.

I believe that running Cagalog.validate(...) on each layer should not be
too slow... hopefully one does not have too many of those stores that
take long to be created, which are the databases ones normally.
And on trunk we have the resource pool using the soft caches, so unless
the server is really under stress many stores should be cached already

The thing is, we have to determine that something is misconfigured or
unavailable _before_ starting to write the xml for it, otherwise there
is just no salvation, you end up with a broken xml and/or a service
exception.

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------