[Geoserver-devel] ScaledGridCoverage2D and CroppedCoverage2D

Simone3 · December 6, 2006, 12:37am

Ciao guys,
please, read below...

On 12/5/06, Martin Desruisseaux <martin.desruisseaux@anonymised.com> wrote:

Hello

In our attempt to uses NetCDF data with Geoserver, we get the attached stack
trace. The NetCDF image is an 16-bits indexed image (IndexColorModel with
DataBuffer.TYPE_USHORT). My feeling is that, at some place in the rendering
chain, the indexed image is transformed into a RGBA image. But because the last
lines in ScaledGridCoverage2D construct a new GridCoverage2D using the same
sample dimensions than the original one, we get an exception because we are
tring to apply a sample dimensions for 1-band images to a 4-bands image. I would
like to know from Simone: is it a raisonable hypothesis?

You are absolutely right. I started last week to work on improving
support for rasters which are natively indexed which means the
underlying pixel values are not actually intensity but an index in a
look-up table, a palette (a lot words to impress people and give the
impression I know what I am talking about ).

There many operations that under different conditions fails to address
requests on paletted images. One of this is scale and it is usually
the incriminated one because it is the first one that gets executed in
the pipeline of the grdicoveragerenderer. But, depending on the
conditions resample could also give wrong results as well as
filteredsubsample. It is not a bug however (at least IMHO), it is
something I knew we where short of and I had no time to fix properly
(if you look in the mailing list you'll find a couple of previous
messagees where I talk about this).

If this is a reasonable hypothesis, the next problem is to spot where in the
code the indexed image is transformed into a 4-bands image. I can't debug the
code because I do not yet run Geoserver on my machine, so the following is based
only from code inspection:

Do not worry man, I will fix this :-), I already know where to put my
hands I was only waiting for time to allocate on this and good news is
I have time between now and the 15th jan, so since the thing is
bothering you and your group time to put something in place is come.

I noticed that ScaledGridCoverage2D do not set the
JAI.KEY_REPLACE_INDEX_COLOR_MODEL hint. In the particular case of JAI "Scale"
operation, the default value is Boolean.TRUE.

Correct.

I would like to set the default in
Geotools to Boolean.FALSE, as I did for all other operations. The rational is
that, from my point of view, GIS field is different from photographic field in
that data are sometime more important than the looking (depending application).
A JAI.KEY_REPLACE_INDEX_COLOR_MODEL hint set to Boolean.TRUE "destroy" the data
and may also be responsible for the exception we are facing right now. It is
important to give the user the opportunity to set the hint explicitly to TRUE if
we wants, but my opinion is that in GIS field the default should be data
integrity. What do you think?

You are right and wrong at the same time. You are right because the
default behaviour should be not expanding the image, but this would
not fix the problem. With this solution you would avoid exceptions
(this is why your approach is correct, but what about scaling with
bilinear interpolation? Yoou would get an mage which does not even
resemble the original one (remember that pixels are indexes not real
data hence if do bilinear or higher interpolation, you make a mess
).

As of data integrity, going from a palette to rgb is not really
jeopardizing integrity since data are untouched, pixel values do not
change. It is more a matter of convenience. If you want to scale
bilinear or subsample average or something like it on a paletted image
the only way to do is by color expansion. Moreover here you are using
the WMS not even the WCS. WCS is for accessing raw unrendered data but
WMS is really meant for doing anything but accessing raw data (and
preserving them ).

Do you allows me to refactor "Scale" as a subclass of "OperationJAI" instead of
"Operation2D"? It would allow us to get right of "ScaledGridCoverage2D"
completly, since most of the work performed by ScaledGridCoverage2D duplicate
OperationJAI (the later try to be generic, which I admit is a cause of
complexity). The JAI.KEY_REPLACE_INDEX_COLOR_MODEL hint would become
Boolean.FALSE in the process.

I am not against refactoring scale as you suggest, but before dong
that I would like to apply the fixes I have scheduled for making
Indexed image work. Since you are reviewing this is a good occasion
for me to get 4 eyes on the fixes :-). I will send you an email
tomorrow about what should I do. I have a fixed deadline for doing
this which is 15th Jan.

Same would need to be done for CroppedCoverage2D,
because it use the "Translate" operation.

It uses the translate operation? I don't think so, it should use Crop
and Crop internally does nothing more than a simple check on the
boundary of the cropped region. It should not even make use of the
hint for index color model. I am not sure Crop should give any
problems in this case.

Note for Cédric: please try to add the following line in ScaledCoverage2D and
CroppedCoverage2D (package org.geotools.coverage.processing.operation) just
before a call to processor.createNS("Scale", ...) or
processor.createNS("Translate", ...) method (or JAI.create(...)):

hints.put(JAI.KEY_REPLACE_INDEX_COLOR_MODEL, Boolean.FALSE);

and tell us if it is of any help.

For the WMS the best place to add the hint is insde the
GridCoverageRenderer constructor like this

    // ///////////////////////////////////////////////////////////////////
    //
    // HINTS
    //
    // ///////////////////////////////////////////////////////////////////
    if (java2dHints != null)
      this.hints.add(java2dHints);
    // this prevents users from overriding leninet hint
    this.hints.add(LENIENT_HINT);
    this.hints.add(JAI.KEY_REPLACE_INDEX_COLOR_MODEL, Boolean.FALSE

However this would work ONLY if you also ask the WMS to render using
NearestNeighbor interpolation whis is NOT the default one. To change
that go to WMS/Rendering and set the interpolation to Neares Neighbor.
Default is bilinear for better quality.

Anyway Cedric, some guidance to come up with a test case would be
really great in this case. How do you suggest to proceed?

Martin, since we are here, tomorrow I will send you an email on my
thoughts about how to fix paletted images for good, bu there is one
thing that would be really needed, much more than moving and renaming
scale and the other operations, which is improving Raster
Symbolization. The actual code from me and Alessio really sucks, it
was a quick hack and we never really had time to improve it. What do
you think about taking a look at that while I improve the management
of the paletted images. Then we can switch, you can review and
refactor a bit and i can review the raster symbolizer work. Is that a
good idea?

Ciao,
Simone.

       Martin.

5 déc. 2006 14:32:43 Registry registerGeotoolsServices
CONFIG: Chargement des extensions de Geotools aux opérations de JAI.
5 déc. 2006 14:32:43 org.geotools.factory.FactoryRegistry scanForPlugins
CONFIG: Implémentations des fabriques de catégorie Operation:
org.geotools.coverage.processing.operation.Absolute
org.geotools.coverage.processing.operation.AddConst
org.geotools.coverage.processing.operation.Convolve
org.geotools.coverage.processing.operation.DivideByConst
org.geotools.coverage.processing.operation.Exp
org.geotools.coverage.processing.operation.GradientMagnitude
org.geotools.coverage.processing.operation.Interpolate
org.geotools.coverage.processing.operation.Invert
org.geotools.coverage.processing.operation.Log
org.geotools.coverage.processing.operation.MaxFilter
org.geotools.coverage.processing.operation.MedianFilter
org.geotools.coverage.processing.operation.MinFilter
org.geotools.coverage.processing.operation.MultiplyConst
org.geotools.coverage.processing.operation.NodataFilter
org.geotools.coverage.processing.operation.Resample
org.geotools.coverage.processing.operation.Rescale
org.geotools.coverage.processing.operation.SelectSampleDimension
org.geotools.coverage.processing.operation.SubtractConst
org.geotools.coverage.processing.operation.SubtractFromConst
org.geotools.coverage.processing.operation.Recolor
org.geotools.coverage.processing.operation.Crop
org.geotools.coverage.processing.operation.Scale
org.geotools.coverage.processing.operation.FilteredSubsample
org.geotools.coverage.processing.operation.SubsampleAverage
5 déc. 2006 14:32:45 org.vfny.geoserver.servlets.AbstractService doService
INFO: Service handled
5 déc. 2006 14:32:45 org.vfny.geoserver.servlets.AbstractService doService
INFO: handling request: org.vfny.geoserver.wms.requests.GetMapRequest@anonymised.com351...
5 déc. 2006 14:32:46 org.vfny.geoserver.servlets.AbstractService doService
INFO: Service handled
5 déc. 2006 14:35:57 org.vfny.geoserver.servlets.AbstractService doService
INFO: handling request: org.vfny.geoserver.wms.requests.GetMapRequest@anonymised.com351...
5 déc. 2006 14:35:59 org.geotools.renderer.lite.StreamingRenderer renderRaster
ATTENTION: Le nombre de bandes de l'image (4) ne correspond pas au nombre d'objets 'SampleDimension' spécifiés (1).
java.lang.IllegalArgumentException: Le nombre de bandes de l'image (4) ne correspond pas au nombre d'objets 'SampleDimension' spécifiés (1).
  at org.geotools.coverage.grid.Grid2DSampleDimension.create(Grid2DSampleDimension.java:118)
  at org.geotools.coverage.grid.GridCoverage2D.<init>(GridCoverage2D.java:271)
  at org.geotools.coverage.processing.operation.ScaledGridCoverage2D.<init>(ScaledGridCoverage2D.java:180)
  at org.geotools.coverage.processing.operation.ScaledGridCoverage2D.create(ScaledGridCoverage2D.java:172)
  at org.geotools.coverage.processing.operation.Scale.doOperation(Scale.java:158)
  at org.geotools.renderer.lite.gridcoverage2d.GridCoverageRenderer.scale(GridCoverageRenderer.java:736)
  at org.geotools.renderer.lite.gridcoverage2d.GridCoverageRenderer.paint(GridCoverageRenderer.java:564)
  at org.geotools.renderer.lite.StreamingRenderer.renderRaster(StreamingRenderer.java:1795)
  at org.geotools.renderer.lite.StreamingRenderer.processSymbolizers(StreamingRenderer.java:1586)
  at org.geotools.renderer.lite.StreamingRenderer.process(StreamingRenderer.java:1530)
  at org.geotools.renderer.lite.StreamingRenderer.processStylers(StreamingRenderer.java:1472)
  at org.geotools.renderer.lite.StreamingRenderer.paint(StreamingRenderer.java:687)
  at org.geotools.renderer.lite.StreamingRenderer.paint(StreamingRenderer.java:430)
  at org.vfny.geoserver.wms.responses.DefaultRasterMapProducer.produceMap(DefaultRasterMapProducer.java:269)
  at org.vfny.geoserver.wms.responses.GetMapResponse.execute(GetMapResponse.java:308)
  at org.vfny.geoserver.servlets.AbstractService.doService(AbstractService.java:535)
  at org.vfny.geoserver.servlets.AbstractService.doGet(AbstractService.java:340)
  at org.geoserver.request.Dispatcher.dispatch(Dispatcher.java:195)
  at org.geoserver.request.Dispatcher.handleRequestInternal(Dispatcher.java:58)
  at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:139)
  at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:44)
  at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:684)
  at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:625)
  at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:392)
  at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:347)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
  at org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:122)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
  at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
  at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
  at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
  at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
  at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
  at java.lang.Thread.run(Unknown Source)

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions

http://www.geo-solutions.it

-------------------------------------------------------

Martin_Desruisseaux · December 6, 2006, 5:07am

Simone Giannecchini a écrit :

Do not worry man, I will fix this :-), I already know where to put my
hands I was only waiting for time to allocate on this and good news is
I have time between now and the 15th jan, so since the thing is
bothering you and your group time to put something in place is come.

Thanks :). I will wait for your work then.

You are right and wrong at the same time. You are right because the
default behaviour should be not expanding the image, but this would
not fix the problem. With this solution you would avoid exceptions
(this is why your approach is correct, but what about scaling with
bilinear interpolation? Yoou would get an mage which does not even
resemble the original one (remember that pixels are indexes not real
data hence if do bilinear or higher interpolation, you make a mess

Interpolations are performed on the "geophysics" view, which is the "correct" place where to perform interpolations on oceanographic or meteorological remote sensing data (interpolation on RGB images produce nice-looking images, but wrong data). All coverage operations that I coded ("Resample", "GradiantMagnitude", etc.) automatically perform their work on the geophysics values (when available) and convert back (if needed) to the indexed image after the operation. I admit that this is not appropriate for indexed images where index are not related to geophysics values, like photographic images - we will need to find some way to handle that.

The whole coverage module was initially designed with the following principle in mind: a 2D grid coverage is a matrix of floating point values, not an image. The matrix may be tiled and packed into integers values, and sometime appears to be displayable (we do our best for that), but all computations are performed on the geophysics values, never on the integers (except in some cases where conversion to geophysics view are not necessary, like "Resample" with "nearest neighboard" interpolation).

As of data integrity, going from a palette to rgb is not really
jeopardizing integrity since data are untouched, pixel values do not
change. It is more a matter of convenience. If you want to scale
bilinear or subsample average or something like it on a paletted image
the only way to do is by color expansion. Moreover here you are using
the WMS not even the WCS. WCS is for accessing raw unrendered data but
WMS is really meant for doing anything but accessing raw data (and
preserving them ).

But Operations.scale(...) and its friends are general-purpose methods, they are not designed specifically for rendering or WMS. Operations.gradientMagnitude for example is used in real oceanographic models, with real economic activity behind them. Operations.resample(...) is also used for in statistical models combining different images from different projections. We are working on real data, not on visual colors. It is crucial in my opinion that *all* grid coverage operation work on real data as much as possible, including Operations.scale(...) and Operation.crop(...). I know that Operations.scale do not touch the source image, but my concern is about Operations.scale output, not the sources. I really want data integrity, because the scaled image may be the input for other operations in a chain. Example: I want to compute the change in temperature (mounthly average) between October and November. For some reason those images may have different sizes. First I may scale them to a common size using Operations.scale(...), then I compute the difference between the two images using Operations.substract(...). If Operations.scale(...) corrupts the data under the hood, I will get a totally wrong answer. In the context of economic or politic decisions based on GIS data, the consequence of converting data to RGB in a library without explicit user request can be very domageable, much more domageable than an image not looking nice in my opinion.

I am not against refactoring scale as you suggest, but before dong
that I would like to apply the fixes I have scheduled for making
Indexed image work. Since you are reviewing this is a good occasion
for me to get 4 eyes on the fixes :-). I will send you an email
tomorrow about what should I do. I have a fixed deadline for doing
this which is 15th Jan.

Thanks.

Same would need to be done for CroppedCoverage2D,
because it use the "Translate" operation.

It uses the translate operation? I don't think so, it should use Crop
and Crop internally does nothing more than a simple check on the
boundary of the cropped region. It should not even make use of the
hint for index color model. I am not sure Crop should give any
problems in this case.

Current implementation of CroppedCoverage2D uses both "Crop" and "Translate" JAI operations. I have not investigate what "Translate" is used for however...

Martin, since we are here, tomorrow I will send you an email on my
thoughts about how to fix paletted images for good, bu there is one
thing that would be really needed, much more than moving and renaming
scale and the other operations, which is improving Raster
Symbolization. The actual code from me and Alessio really sucks, it
was a quick hack and we never really had time to improve it. What do
you think about taking a look at that while I improve the management
of the paletted images. Then we can switch, you can review and
refactor a bit and i can review the raster symbolizer work. Is that a
good idea?

We can try, but I'm not sure about what we means by raster symbolizer (I never looked actually). I stand more on the numerical size of grid coverage...

Thanks again,

Martin