[Geoserver-devel] Proposing to set proper HTTP codes when returning service exceptions

GeoServer GeoServer Developer
1

Hi,
lately I've been going around talking to various people setting up
GeoServer in production,
in enviroments where HTTP request monitoring with some sort of Apache
plugin is standard practice,
and in all these cases I hear complaints about the fact GeoServer, and
OWS services in general,
return HTTP 200 even in case of exception.

The reason for the complaint is obvious, it does not allow proper
monitoring of exceptions
(I know about the monitoring module, but these people want GS to play
nicely in their existing,
enterprise wide monitoring enviroment, having an extra monitoring tool
to deal with is not something they
look for in the short time, though I'm sure they'll eventually see
the benefit of having something
that actually understands and classifies the various OGC requests).

So far I thought that the OGC specification simply never said anything
about the status codes,
and looking into OWS 1.0 I actually did not find anything at all (it
does not even say to return 200
actually).
However in OWS 1.1, page 110 of the printout, it says:

--------------------------------------------------------------------------------------------------------------------------------------------
A.4.1.5 HTTP response status code
a) Test purpose: Verify that a service request which generates an
exception produces a response that contains
1) a service exception report, and 2) a status code indicating an error.
--------------------------------------------------------------------------------------------------------------------------------------------

Ah ha!

So at the very least the WCS 1.1 and WPS 1.0 implementations, both
based on OWS 1.1,
are not setting the status codes as expected from the spec (yes, the
CITE tests are not complaining...)

Generally speaking I would really like to have all service exceptions
report back proper status codes
based on the on the service exception "exceptionCode":
- HTTP 400 for: OperationNotSupported, MissingParameterValue,
InvalidParameterValue,
  VersionNegotiationFailed, InvalidUpdateSequence
- HTTP 500 for NoApplicableCode or missing code
(status code reference here:
http://en.wikipedia.org/wiki/List_of_HTTP_status_codes)

The only concern I have at the moment is that some client might break
as a result of this
change in behavior. I mean, generally speaking OGC clients should not
be concerned at all
with the status codes, but one cannot be completely sure.

Should we make this behavior configurable? What do you think?

Cheers
Andrea

-----------------------------------------------------
Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-----------------------------------------------------

2

In OWS 2.0 (section 8.6) they actually made this more explicit, giving the
status code to be used:

http://portal.opengeospatial.org/files/?artifact_id=38867

Since it does not break CITE tests, I think it would be good to make this
"new" behaviour the default.

Best regards,
Bart

Hi,
lately I've been going around talking to various people setting up
GeoServer in production,
in enviroments where HTTP request monitoring with some sort of Apache
plugin is standard practice,
and in all these cases I hear complaints about the fact GeoServer, and
OWS services in general,
return HTTP 200 even in case of exception.

The reason for the complaint is obvious, it does not allow proper
monitoring of exceptions
(I know about the monitoring module, but these people want GS to play
nicely in their existing,
enterprise wide monitoring enviroment, having an extra monitoring tool
to deal with is not something they
look for in the short time, though I'm sure they'll eventually see
the benefit of having something
that actually understands and classifies the various OGC requests).

So far I thought that the OGC specification simply never said anything
about the status codes,
and looking into OWS 1.0 I actually did not find anything at all (it
does not even say to return 200
actually).
However in OWS 1.1, page 110 of the printout, it says:

--------------------------------------------------------------------------------------------------------------------------------------------
A.4.1.5 HTTP response status code
a) Test purpose: Verify that a service request which generates an
exception produces a response that contains
1) a service exception report, and 2) a status code indicating an error.
--------------------------------------------------------------------------------------------------------------------------------------------

Ah ha!

So at the very least the WCS 1.1 and WPS 1.0 implementations, both
based on OWS 1.1,
are not setting the status codes as expected from the spec (yes, the
CITE tests are not complaining...)

Generally speaking I would really like to have all service exceptions
report back proper status codes
based on the on the service exception "exceptionCode":
- HTTP 400 for: OperationNotSupported, MissingParameterValue,
InvalidParameterValue,
  VersionNegotiationFailed, InvalidUpdateSequence
- HTTP 500 for NoApplicableCode or missing code
(status code reference here:
http://en.wikipedia.org/wiki/List_of_HTTP_status_codes)

The only concern I have at the moment is that some client might break
as a result of this
change in behavior. I mean, generally speaking OGC clients should not
be concerned at all
with the status codes, but one cannot be completely sure.

Should we make this behavior configurable? What do you think?

Cheers
Andrea

-----------------------------------------------------
Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-----------------------------------------------------

------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

3

On Tue, Nov 30, 2010 at 10:26 AM, Bart van den Eijnden (OSGIS)
<bartvde@anonymised.com> wrote:

In OWS 2.0 (section 8.6) they actually made this more explicit, giving the
status code to be used:

http://portal.opengeospatial.org/files/?artifact_id=38867

Ah ha, interesting. Attaching a screenshot of the table for quick reference.
Not very far from what I proposed, but also specifies a message.

Cheers
Andrea

-----------------------------------------------------
Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-----------------------------------------------------

(attachments)

status_codes.png

4

+1. This would avoid having to use regex checks in nagios.

On 30/11/10 17:15, Andrea Aime wrote:

Hi,
lately I've been going around talking to various people setting up
GeoServer in production,
in enviroments where HTTP request monitoring with some sort of Apache
plugin is standard practice,
and in all these cases I hear complaints about the fact GeoServer, and
OWS services in general,
return HTTP 200 even in case of exception.

The reason for the complaint is obvious, it does not allow proper
monitoring of exceptions
(I know about the monitoring module, but these people want GS to play
nicely in their existing,
enterprise wide monitoring enviroment, having an extra monitoring tool
to deal with is not something they
  look for in the short time, though I'm sure they'll eventually see
the benefit of having something
  that actually understands and classifies the various OGC requests).

So far I thought that the OGC specification simply never said anything
about the status codes,
and looking into OWS 1.0 I actually did not find anything at all (it
does not even say to return 200
actually).
However in OWS 1.1, page 110 of the printout, it says:

--------------------------------------------------------------------------------------------------------------------------------------------
A.4.1.5 HTTP response status code
a) Test purpose: Verify that a service request which generates an
exception produces a response that contains
1) a service exception report, and 2) a status code indicating an error.
--------------------------------------------------------------------------------------------------------------------------------------------

Ah ha!

So at the very least the WCS 1.1 and WPS 1.0 implementations, both
based on OWS 1.1,
are not setting the status codes as expected from the spec (yes, the
CITE tests are not complaining...)

Generally speaking I would really like to have all service exceptions
report back proper status codes
based on the on the service exception "exceptionCode":
- HTTP 400 for: OperationNotSupported, MissingParameterValue,
InvalidParameterValue,
   VersionNegotiationFailed, InvalidUpdateSequence
- HTTP 500 for NoApplicableCode or missing code
(status code reference here:
http://en.wikipedia.org/wiki/List_of_HTTP_status_codes)

The only concern I have at the moment is that some client might break
as a result of this
change in behavior. I mean, generally speaking OGC clients should not
be concerned at all
with the status codes, but one cannot be completely sure.

Should we make this behavior configurable? What do you think?

Cheers
Andrea

-----------------------------------------------------
Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-----------------------------------------------------

------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App& Earn a Chance To Win $500!
Tap into the largest installed PC base& get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineering Team Leader
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre

5

I like the idea, but i agree I don’t think it is something we can turn on by default for reasons you stated. I think the code mappings make sense as well.

2c.

-Justin

On Tue, Nov 30, 2010 at 2:15 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
lately I’ve been going around talking to various people setting up
GeoServer in production,
in enviroments where HTTP request monitoring with some sort of Apache
plugin is standard practice,
and in all these cases I hear complaints about the fact GeoServer, and
OWS services in general,
return HTTP 200 even in case of exception.

The reason for the complaint is obvious, it does not allow proper
monitoring of exceptions
(I know about the monitoring module, but these people want GS to play
nicely in their existing,
enterprise wide monitoring enviroment, having an extra monitoring tool
to deal with is not something they
look for in the short time, though I’m sure they’ll eventually see
the benefit of having something
that actually understands and classifies the various OGC requests).

So far I thought that the OGC specification simply never said anything
about the status codes,
and looking into OWS 1.0 I actually did not find anything at all (it
does not even say to return 200
actually).
However in OWS 1.1, page 110 of the printout, it says:

A.4.1.5 HTTP response status code
a) Test purpose: Verify that a service request which generates an
exception produces a response that contains

  1. a service exception report, and 2) a status code indicating an error.

Ah ha!

So at the very least the WCS 1.1 and WPS 1.0 implementations, both
based on OWS 1.1,
are not setting the status codes as expected from the spec (yes, the
CITE tests are not complaining…)

Generally speaking I would really like to have all service exceptions
report back proper status codes
based on the on the service exception “exceptionCode”:

The only concern I have at the moment is that some client might break
as a result of this
change in behavior. I mean, generally speaking OGC clients should not
be concerned at all
with the status codes, but one cannot be completely sure.

Should we make this behavior configurable? What do you think?

Cheers
Andrea

Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

6

On Tue, Nov 30, 2010 at 3:47 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

I like the idea, but i agree I don't think it is something we can turn on by
default for reasons you stated. I think the code mappings make sense as
well.

Roger for the OWS 1.0 based protocols.
Wondering, what shall we do for WCS 1.1 and WPS 1.0 though?
For those the OWS 1.1 spec says to return status proper status codes.
I guess the DefaultServiceExceptionHandler can use some flag to decide
whether to use the http error codes or not, whilst the
OWS11ServiceExceptionHandler
should just set the status codes?
Opinions?

Cheers
Andrea

-----------------------------------------------------
Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-----------------------------------------------------

7

On Tue, Nov 30, 2010 at 8:06 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Tue, Nov 30, 2010 at 3:47 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

I like the idea, but i agree I don’t think it is something we can turn on by
default for reasons you stated. I think the code mappings make sense as
well.

Roger for the OWS 1.0 based protocols.
Wondering, what shall we do for WCS 1.1 and WPS 1.0 though?
For those the OWS 1.1 spec says to return status proper status codes.
I guess the DefaultServiceExceptionHandler can use some flag to decide
whether to use the http error codes or not, whilst the
OWS11ServiceExceptionHandler
should just set the status codes?
Opinions?

Yeah I think that approach makes sense since the spec supports it and I don’t know of many clients built from those specific services so any change will probably have minimal impact.

A couple of questions:

  1. How are we going to go about registing the code to http code mappings? Will they be fixed? or will a particular service be able to add its own mappings?

  2. Will we ever want to return specific codes other than 400 and 500. For example consider the case of a client requesting a layer that does not exist. A 404 would also make sense in this case.

Cheers
Andrea

Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

8

On Tue, Nov 30, 2010 at 4:19 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Yeah I think that approach makes sense since the spec supports it and I
don't know of many clients built from those specific services so any change
will probably have minimal impact.
A couple of questions:
1. How are we going to go about registing the code to http code mappings?
Will they be fixed? or will a particular service be able to add its own
mappings?

Yeah, good question. The mappings for the codes defined in OWS 2.0
are there and I think we'll use them.
OWS 2.0 also states services defining new codes will have to state which
http response code to use.
In general I'd like to see an enumeration like the one I did for WcsException,
it's easy to add two fields to specify the error code and the error message.
However there are two issues:
- the current code uses a string for the exceptionCode
- enumerations cannot be extended at runtime and cannot be subclassed either

Hmmm... what about defining an ExceptionCode class instead that has the
OGC code, the http code and the http error message, and then use the
plain old set of static final constants in ServiceException for the common
codes?

I guess we should also deprecate the constructors taking a string as
an argument... ugh, fixing that would result in a massive undertaking,
exceptions are everywhere (though exceptions with a code are not
exactly the most common thing around in the code).

Maybe for the time being we could have ServiceException provide a
method to register a new exception code and then have a lookup
for the proper ExceptionCode given the exception code as a string?
And then fix the code as we go over time...

Not sure, this one is sure nasty, ideas to do this without spening
a ton of time would be appreciated.

2. Will we ever want to return specific codes other than 400 and 500. For
example consider the case of a client requesting a layer that does not
exist. A 404 would also make sense in this case.

The OWS 2.0 spec appears to be pretty specific on the status codes, so
I did not consider chosing our own codes, but if we go for the above
ExceptionCode class a service could go and build its own object with
a specific http code and error message.

Cheers
Andrea

-----------------------------------------------------
Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-----------------------------------------------------