[Geoserver-devel] DynamicColorMap

GeoServer GeoServer Developer
1

Hi all,

Wondering if anyone has any hints as to what might be causing this issue:

https://osgeo-org.atlassian.net/browse/GEOS-8070

Even if it’s just where in the code I might look for the issue. I looked at the actual ColorMap generation, and that seems fairly straight forward so I’m not sure if the problem is there. Otherwise I’m not so sure. My first thought is that something’s not being buffered correctly or something along those lines, but I wouldn’t know where to start.

2

Hi Devon,
going by memory the dynamic color map module depends on coverage statistics metadata
derived in GDAL called “PAM_DATASET”. Did you check the requirements and followed the
necessary steps?

http://docs.geoserver.org/stable/en/user/community/colormap/index.html#dynamiccolormap-requirements

Cheers
Andrea

···

On Mon, Apr 3, 2017 at 11:26 PM, Devon Tucker <dtucker@anonymised.com> wrote:

Hi all,

Wondering if anyone has any hints as to what might be causing this issue:

https://osgeo-org.atlassian.net/browse/GEOS-8070

Even if it’s just where in the code I might look for the issue. I looked at the actual ColorMap generation, and that seems fairly straight forward so I’m not sure if the problem is there. Otherwise I’m not so sure. My first thought is that something’s not being buffered correctly or something along those lines, but I wouldn’t know where to start.

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.com.366…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

3

The statistics all seem to be there and fine. As far as I can tell, the ColorMap itself seems to generate fine. The example, at least when viewed at full extents looks fine and exactly like it does in the test case data. The problem only occurs at certain scales and extents.

Right now I’m looking at the various bits of the rendering pipeline (RenderedImageMapOutputFormat.directRasterRender) that get invoked when you have a a RenderingTransformation vs. without one. There are definitely some differences there, in terms of the code paths taken. The one thing I do notice is that the the GridGeometries of the coverage after the rendering transformation is applied vs. the grid geometry when the coverage is read without a rendering transform are slightly different, which I think is maybe causing the issue?

For example after the rendering transformation the coverage grid geometry looks like this:

GridGeometry2D[GridEnvelope2D[1…7, 2…12], PARAM_MT[“Affine”,
PARAMETER[“num_row”, 3],
PARAMETER[“num_col”, 3],
PARAMETER[“elt_0_0”, 0.5742214584350587],
PARAMETER[“elt_0_2”, 6.840767460515945],
PARAMETER[“elt_1_1”, -0.159840087890625],
PARAMETER[“elt_1_2”, 43.519122374398364]]]

And without doing the rendering transformation, the coverage looks like this:

GridGeometry2D[GridEnvelope2D[0…7, 0…11], PARAM_MT[“Affine”,
PARAMETER[“num_row”, 3],
PARAMETER[“num_col”, 3],
PARAMETER[“elt_0_0”, 0.5742214584350587],
PARAMETER[“elt_0_2”, 7.414988918951003],
PARAMETER[“elt_1_1”, -0.159840087890625],
PARAMETER[“elt_1_2”, 43.19944219861711]]]

I think this would explain the pixels cut off on the right and bottom? But I’m not sure. The big difference I’ve noticed in the two code paths is that when doing the coverage read while performing the rendering transformation the read is done against the coverage reader directly, but otherwise a GridCoverageReaderHelper is used. I still need to look into what the differences might be there, but there’s a lot of moving parts. I may just try to plug the GridCoverageReaderHelper into the former code path and see what pops out, just as an experiment.

Cheers

···

On Tue, Apr 4, 2017 at 2:33 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi Devon,
going by memory the dynamic color map module depends on coverage statistics metadata
derived in GDAL called “PAM_DATASET”. Did you check the requirements and followed the
necessary steps?

http://docs.geoserver.org/stable/en/user/community/colormap/index.html#dynamiccolormap-requirements

Cheers
Andrea

On Mon, Apr 3, 2017 at 11:26 PM, Devon Tucker <dtucker@anonymised.com> wrote:

Hi all,

Wondering if anyone has any hints as to what might be causing this issue:

https://osgeo-org.atlassian.net/browse/GEOS-8070

Even if it’s just where in the code I might look for the issue. I looked at the actual ColorMap generation, and that seems fairly straight forward so I’m not sure if the problem is there. Otherwise I’m not so sure. My first thought is that something’s not being buffered correctly or something along those lines, but I wouldn’t know where to start.

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.comrge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

4

Hi Devon,
short answer is… I have no clue. Yes, indeed the path between RT and non RT cannot be the same, one works off a reader,
the other off a coverage and the RT can influence what is being read (area and resolution).

I would not blame the geometry per se though, some time ago Niels was reporting an “off by one” error somewhere in
the coverage reading chain… but I don’t remember if the thread was here or elsewhere, or if a ticket was opened.
It could be the cause of the issue though.

Niels, do you remember the thread in question?

Cheers
Andrea

···

On Wed, Apr 5, 2017 at 2:15 AM, Devon Tucker <dtucker@anonymised.com> wrote:

The statistics all seem to be there and fine. As far as I can tell, the ColorMap itself seems to generate fine. The example, at least when viewed at full extents looks fine and exactly like it does in the test case data. The problem only occurs at certain scales and extents.

Right now I’m looking at the various bits of the rendering pipeline (RenderedImageMapOutputFormat.directRasterRender) that get invoked when you have a a RenderingTransformation vs. without one. There are definitely some differences there, in terms of the code paths taken. The one thing I do notice is that the the GridGeometries of the coverage after the rendering transformation is applied vs. the grid geometry when the coverage is read without a rendering transform are slightly different, which I think is maybe causing the issue?

For example after the rendering transformation the coverage grid geometry looks like this:

GridGeometry2D[GridEnvelope2D[1…7, 2…12], PARAM_MT[“Affine”,
PARAMETER[“num_row”, 3],
PARAMETER[“num_col”, 3],
PARAMETER[“elt_0_0”, 0.5742214584350587],
PARAMETER[“elt_0_2”, 6.840767460515945],
PARAMETER[“elt_1_1”, -0.159840087890625],
PARAMETER[“elt_1_2”, 43.519122374398364]]]

And without doing the rendering transformation, the coverage looks like this:

GridGeometry2D[GridEnvelope2D[0…7, 0…11], PARAM_MT[“Affine”,
PARAMETER[“num_row”, 3],
PARAMETER[“num_col”, 3],
PARAMETER[“elt_0_0”, 0.5742214584350587],
PARAMETER[“elt_0_2”, 7.414988918951003],
PARAMETER[“elt_1_1”, -0.159840087890625],
PARAMETER[“elt_1_2”, 43.19944219861711]]]

I think this would explain the pixels cut off on the right and bottom? But I’m not sure. The big difference I’ve noticed in the two code paths is that when doing the coverage read while performing the rendering transformation the read is done against the coverage reader directly, but otherwise a GridCoverageReaderHelper is used. I still need to look into what the differences might be there, but there’s a lot of moving parts. I may just try to plug the GridCoverageReaderHelper into the former code path and see what pops out, just as an experiment.

Cheers

–

On Tue, Apr 4, 2017 at 2:33 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi Devon,
going by memory the dynamic color map module depends on coverage statistics metadata
derived in GDAL called “PAM_DATASET”. Did you check the requirements and followed the
necessary steps?

http://docs.geoserver.org/stable/en/user/community/colormap/index.html#dynamiccolormap-requirements

Cheers
Andrea

On Mon, Apr 3, 2017 at 11:26 PM, Devon Tucker <dtucker@anonymised.com> wrote:

Hi all,

Wondering if anyone has any hints as to what might be causing this issue:

https://osgeo-org.atlassian.net/browse/GEOS-8070

Even if it’s just where in the code I might look for the issue. I looked at the actual ColorMap generation, and that seems fairly straight forward so I’m not sure if the problem is there. Otherwise I’m not so sure. My first thought is that something’s not being buffered correctly or something along those lines, but I wouldn’t know where to start.

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.comrge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

5

You mean this

https://osgeo-org.atlassian.net/browse/GEOS-7793

?

On 05-04-17 12:57, Andrea Aime wrote:

Hi Devon,
short answer is... I have no clue. Yes, indeed the path between RT and non RT cannot be the same, one works off a reader,
the other off a coverage and the RT can influence what is being read (area and resolution).

I would not blame the geometry per se though, some time ago Niels was reporting an "off by one" error somewhere in
the coverage reading chain... but I don't remember if the thread was here or elsewhere, or if a ticket was opened.
It could be the cause of the issue though.

Niels, do you remember the thread in question?

Cheers
Andrea

On Wed, Apr 5, 2017 at 2:15 AM, Devon Tucker <dtucker@anonymised.com <mailto:dtucker@anonymised.com>> wrote:

    The statistics all seem to be there and fine. As far as I can
    tell, the ColorMap itself seems to generate fine. The example, at
    least when viewed at full extents looks fine and exactly like it
    does in the test case data. The problem only occurs at certain
    scales and extents.

    Right now I'm looking at the various bits of the rendering
    pipeline (RenderedImageMapOutputFormat.directRasterRender) that
    get invoked when you have a a RenderingTransformation vs. without
    one. There are definitely some differences there, in terms of the
    code paths taken. The one thing I do notice is that the the
    GridGeometries of the coverage after the rendering transformation
    is applied vs. the grid geometry when the coverage is read without
    a rendering transform are slightly different, which I think is
    maybe causing the issue?

    For example after the rendering transformation the coverage grid
    geometry looks like this:

    GridGeometry2D[GridEnvelope2D[1..7, 2..12], PARAM_MT["Affine",
      PARAMETER["num_row", 3],
      PARAMETER["num_col", 3],
      PARAMETER["elt_0_0", 0.5742214584350587],
      PARAMETER["elt_0_2", 6.840767460515945],
      PARAMETER["elt_1_1", -0.159840087890625],
      PARAMETER["elt_1_2", 43.519122374398364]]]

    And without doing the rendering transformation, the coverage looks
    like this:

      GridGeometry2D[GridEnvelope2D[0..7, 0..11], PARAM_MT["Affine",
      PARAMETER["num_row", 3],
      PARAMETER["num_col", 3],
      PARAMETER["elt_0_0", 0.5742214584350587],
      PARAMETER["elt_0_2", 7.414988918951003],
      PARAMETER["elt_1_1", -0.159840087890625],
      PARAMETER["elt_1_2", 43.19944219861711]]]

    I think this would explain the pixels cut off on the right and
    bottom? But I'm not sure. The big difference I've noticed in the
    two code paths is that when doing the coverage read while
    performing the rendering transformation the read is done against
    the coverage reader directly, but otherwise a
    GridCoverageReaderHelper is used. I still need to look into what
    the differences might be there, but there's a lot of moving parts.
    I may just try to plug the GridCoverageReaderHelper into the
    former code path and see what pops out, just as an experiment.

    Cheers

    On Tue, Apr 4, 2017 at 2:33 AM, Andrea Aime
    <andrea.aime@anonymised.com
    <mailto:andrea.aime@anonymised.com>> wrote:

        Hi Devon,
        going by memory the dynamic color map module depends on
        coverage statistics metadata
        derived in GDAL called "PAM_DATASET". Did you check the
        requirements and followed the
        necessary steps?

        http://docs.geoserver.org/stable/en/user/community/colormap/index.html#dynamiccolormap-requirements
        <http://docs.geoserver.org/stable/en/user/community/colormap/index.html#dynamiccolormap-requirements&gt;

        Cheers
        Andrea

        On Mon, Apr 3, 2017 at 11:26 PM, Devon Tucker
        <dtucker@anonymised.com <mailto:dtucker@anonymised.com>>
        wrote:

            Hi all,

            Wondering if anyone has any hints as to what might be
            causing this issue:

            https://osgeo-org.atlassian.net/browse/GEOS-8070
            <https://osgeo-org.atlassian.net/browse/GEOS-8070&gt;

            Even if it's just where in the code I might look for the
            issue. I looked at the actual ColorMap generation, and
            that seems fairly straight forward so I'm not sure if the
            problem is there. Otherwise I'm not so sure. My first
            thought is that something's not being buffered correctly
            or something along those lines, but I wouldn't know where
            to start.

            ------------------------------------------------------------------------------
            Check out the vibrant tech community on one of the world's
            most
            engaging tech sites, Slashdot.org! http://sdm.link/slashdot
            _______________________________________________
            Geoserver-devel mailing list
            Geoserver-devel@lists.sourceforge.net
            <mailto:Geoserver-devel@lists.sourceforge.net>
            https://lists.sourceforge.net/lists/listinfo/geoserver-devel
            <https://lists.sourceforge.net/lists/listinfo/geoserver-devel&gt;

        -- ==
        GeoServer Professional Services from the experts! Visit
        http://goo.gl/it488V for more information.
        ==

        Ing. Andrea Aime
        @geowolf
        Technical Lead

        GeoSolutions S.A.S.
        Via di Montramito 3/A
        55054 Massarosa (LU)
        phone: +39 0584 962313 <tel:+39%200584%20962313>
        fax: +39 0584 1660272 <tel:+39%200584%20166%200272>
        mob: +39 339 8844549 <tel:+39%20339%20884%204549>

        http://www.geo-solutions.it
        http://twitter.com/geosolutions_it
        <http://twitter.com/geosolutions_it&gt;

        *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

        Le informazioni contenute in questo messaggio di posta
        elettronica e/o nel/i file/s allegato/i sono da considerarsi
        strettamente riservate. Il loro utilizzo è consentito
        esclusivamente al destinatario del messaggio, per le finalità
        indicate nel messaggio stesso. Qualora riceviate questo
        messaggio senza esserne il destinatario, Vi preghiamo
        cortesemente di darcene notizia via e-mail e di procedere alla
        distruzione del messaggio stesso, cancellandolo dal Vostro
        sistema. Conservare il messaggio stesso, divulgarlo anche in
        parte, distribuirlo ad altri soggetti, copiarlo, od
        utilizzarlo per finalità diverse, costituisce comportamento
        contrario ai principi dettati dal D.Lgs. 196/2003.

        The information in this message and/or attachments, is
        intended solely for the attention and use of the named
        addressee(s) and may be confidential or proprietary in nature
        or covered by the provisions of privacy act (Legislative
        Decree June, 30 2003, no.196 - Italy's New Data Protection
        Code).Any use not in accord with its purpose, any disclosure,
        reproduction, copying, distribution, or either dissemination,
        either whole or partial, is strictly forbidden except previous
        formal approval of the named addressee(s). If you are not the
        intended recipient, please contact immediately the sender by
        telephone, fax or e-mail and delete the information in this
        message that has been received in error. The sender does not
        give any warranty or accept liability as the content, accuracy
        or completeness of sent messages and accepts no
        responsibility for changes made after they were sent or for
        other risks which arise as a result of e-mail transmission,
        viruses, etc.

        -------------------------------------------------------

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

-------------------------------------------------------

6

Nope, not that one. You did find some issues with code removing 1 from the pixel grid extents, if memory
serves me right.

Btw that issue you are pointing to might have been solved recently by avoiding the large raster space coordinates
in image mosaic: https://osgeo-org.atlassian.net/browse/GEOT-5615

Cheers
Andrea

···

On Wed, Apr 5, 2017 at 1:59 PM, Niels Charlier <niels@anonymised.com> wrote:

You mean this

https://osgeo-org.atlassian.net/browse/GEOS-7793

?

On 05-04-17 12:57, Andrea Aime wrote:

Hi Devon,
short answer is… I have no clue. Yes, indeed the path between RT and non RT cannot be the same, one works off a reader,
the other off a coverage and the RT can influence what is being read (area and resolution).

I would not blame the geometry per se though, some time ago Niels was reporting an “off by one” error somewhere in
the coverage reading chain… but I don’t remember if the thread was here or elsewhere, or if a ticket was opened.
It could be the cause of the issue though.

Niels, do you remember the thread in question?

Cheers
Andrea

–

On Wed, Apr 5, 2017 at 2:15 AM, Devon Tucker <dtucker@anonymised.com> wrote:

The statistics all seem to be there and fine. As far as I can tell, the ColorMap itself seems to generate fine. The example, at least when viewed at full extents looks fine and exactly like it does in the test case data. The problem only occurs at certain scales and extents.

Right now I’m looking at the various bits of the rendering pipeline (RenderedImageMapOutputFormat.directRasterRender) that get invoked when you have a a RenderingTransformation vs. without one. There are definitely some differences there, in terms of the code paths taken. The one thing I do notice is that the the GridGeometries of the coverage after the rendering transformation is applied vs. the grid geometry when the coverage is read without a rendering transform are slightly different, which I think is maybe causing the issue?

For example after the rendering transformation the coverage grid geometry looks like this:

GridGeometry2D[GridEnvelope2D[1…7, 2…12], PARAM_MT[“Affine”,
PARAMETER[“num_row”, 3],
PARAMETER[“num_col”, 3],
PARAMETER[“elt_0_0”, 0.5742214584350587],
PARAMETER[“elt_0_2”, 6.840767460515945],
PARAMETER[“elt_1_1”, -0.159840087890625],
PARAMETER[“elt_1_2”, 43.519122374398364]]]

And without doing the rendering transformation, the coverage looks like this:

GridGeometry2D[GridEnvelope2D[0…7, 0…11], PARAM_MT[“Affine”,
PARAMETER[“num_row”, 3],
PARAMETER[“num_col”, 3],
PARAMETER[“elt_0_0”, 0.5742214584350587],
PARAMETER[“elt_0_2”, 7.414988918951003],
PARAMETER[“elt_1_1”, -0.159840087890625],
PARAMETER[“elt_1_2”, 43.19944219861711]]]

I think this would explain the pixels cut off on the right and bottom? But I’m not sure. The big difference I’ve noticed in the two code paths is that when doing the coverage read while performing the rendering transformation the read is done against the coverage reader directly, but otherwise a GridCoverageReaderHelper is used. I still need to look into what the differences might be there, but there’s a lot of moving parts. I may just try to plug the GridCoverageReaderHelper into the former code path and see what pops out, just as an experiment.

Cheers

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

On Tue, Apr 4, 2017 at 2:33 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi Devon,
going by memory the dynamic color map module depends on coverage statistics metadata
derived in GDAL called “PAM_DATASET”. Did you check the requirements and followed the
necessary steps?

http://docs.geoserver.org/stable/en/user/community/colormap/index.html#dynamiccolormap-requirements

Cheers
Andrea

On Mon, Apr 3, 2017 at 11:26 PM, Devon Tucker <dtucker@anonymised.com> wrote:

Hi all,

Wondering if anyone has any hints as to what might be causing this issue:

https://osgeo-org.atlassian.net/browse/GEOS-8070

Even if it’s just where in the code I might look for the issue. I looked at the actual ColorMap generation, and that seems fairly straight forward so I’m not sure if the problem is there. Otherwise I’m not so sure. My first thought is that something’s not being buffered correctly or something along those lines, but I wouldn’t know where to start.

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.comrge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

7

Either I have forgotten or it was somebody else... can't find it back.

Interesting about that other issue btw, that should resolve it indeed

On 05-04-17 14:18, Andrea Aime wrote:

Nope, not that one. You did find some issues with code removing 1 from the pixel grid extents, if memory
serves me right.

Btw that issue you are pointing to might have been solved recently by avoiding the large raster space coordinates
in image mosaic: https://osgeo-org.atlassian.net/browse/GEOT-5615

Cheers
Andrea

On Wed, Apr 5, 2017 at 1:59 PM, Niels Charlier <niels@anonymised.com <mailto:niels@anonymised.com>> wrote:

    You mean this

    https://osgeo-org.atlassian.net/browse/GEOS-7793
    <https://osgeo-org.atlassian.net/browse/GEOS-7793&gt;

    ?

    On 05-04-17 12:57, Andrea Aime wrote:

    Hi Devon,
    short answer is... I have no clue. Yes, indeed the path between
    RT and non RT cannot be the same, one works off a reader,
    the other off a coverage and the RT can influence what is being
    read (area and resolution).

    I would not blame the geometry per se though, some time ago Niels
    was reporting an "off by one" error somewhere in
    the coverage reading chain... but I don't remember if the thread
    was here or elsewhere, or if a ticket was opened.
    It could be the cause of the issue though.

    Niels, do you remember the thread in question?

    Cheers
    Andrea

    On Wed, Apr 5, 2017 at 2:15 AM, Devon Tucker
    <dtucker@anonymised.com <mailto:dtucker@anonymised.com>> wrote:

        The statistics all seem to be there and fine. As far as I can
        tell, the ColorMap itself seems to generate fine. The
        example, at least when viewed at full extents looks fine and
        exactly like it does in the test case data. The problem only
        occurs at certain scales and extents.

        Right now I'm looking at the various bits of the rendering
        pipeline (RenderedImageMapOutputFormat.directRasterRender)
        that get invoked when you have a a RenderingTransformation
        vs. without one. There are definitely some differences there,
        in terms of the code paths taken. The one thing I do notice
        is that the the GridGeometries of the coverage after the
        rendering transformation is applied vs. the grid geometry
        when the coverage is read without a rendering transform are
        slightly different, which I think is maybe causing the issue?

        For example after the rendering transformation the coverage
        grid geometry looks like this:

        GridGeometry2D[GridEnvelope2D[1..7, 2..12], PARAM_MT["Affine",
          PARAMETER["num_row", 3],
          PARAMETER["num_col", 3],
          PARAMETER["elt_0_0", 0.5742214584350587],
          PARAMETER["elt_0_2", 6.840767460515945],
          PARAMETER["elt_1_1", -0.159840087890625],
          PARAMETER["elt_1_2", 43.519122374398364]]]

        And without doing the rendering transformation, the coverage
        looks like this:

          GridGeometry2D[GridEnvelope2D[0..7, 0..11], PARAM_MT["Affine",
          PARAMETER["num_row", 3],
          PARAMETER["num_col", 3],
          PARAMETER["elt_0_0", 0.5742214584350587],
          PARAMETER["elt_0_2", 7.414988918951003],
          PARAMETER["elt_1_1", -0.159840087890625],
          PARAMETER["elt_1_2", 43.19944219861711]]]

        I think this would explain the pixels cut off on the right
        and bottom? But I'm not sure. The big difference I've noticed
        in the two code paths is that when doing the coverage read
        while performing the rendering transformation the read is
        done against the coverage reader directly, but otherwise a
        GridCoverageReaderHelper is used. I still need to look into
        what the differences might be there, but there's a lot of
        moving parts. I may just try to plug the
        GridCoverageReaderHelper into the former code path and see
        what pops out, just as an experiment.

        Cheers

        On Tue, Apr 4, 2017 at 2:33 AM, Andrea Aime
        <andrea.aime@anonymised.com
        <mailto:andrea.aime@anonymised.com>> wrote:

            Hi Devon,
            going by memory the dynamic color map module depends on
            coverage statistics metadata
            derived in GDAL called "PAM_DATASET". Did you check the
            requirements and followed the
            necessary steps?

            http://docs.geoserver.org/stable/en/user/community/colormap/index.html#dynamiccolormap-requirements
            <http://docs.geoserver.org/stable/en/user/community/colormap/index.html#dynamiccolormap-requirements&gt;

            Cheers
            Andrea

            On Mon, Apr 3, 2017 at 11:26 PM, Devon Tucker
            <dtucker@anonymised.com
            <mailto:dtucker@anonymised.com>> wrote:

                Hi all,

                Wondering if anyone has any hints as to what might be
                causing this issue:

                https://osgeo-org.atlassian.net/browse/GEOS-8070
                <https://osgeo-org.atlassian.net/browse/GEOS-8070&gt;

                Even if it's just where in the code I might look for
                the issue. I looked at the actual ColorMap
                generation, and that seems fairly straight forward so
                I'm not sure if the problem is there. Otherwise I'm
                not so sure. My first thought is that something's not
                being buffered correctly or something along those
                lines, but I wouldn't know where to start.

                ------------------------------------------------------------------------------
                Check out the vibrant tech community on one of the
                world's most
                engaging tech sites, Slashdot.org!
                http://sdm.link/slashdot
                _______________________________________________
                Geoserver-devel mailing list
                Geoserver-devel@lists.sourceforge.net
                <mailto:Geoserver-devel@lists.sourceforge.net>
                https://lists.sourceforge.net/lists/listinfo/geoserver-devel
                <https://lists.sourceforge.net/lists/listinfo/geoserver-devel&gt;

            -- ==
            GeoServer Professional Services from the experts! Visit
            http://goo.gl/it488V for more information.
            ==

            Ing. Andrea Aime
            @geowolf
            Technical Lead

            GeoSolutions S.A.S.
            Via di Montramito 3/A
            55054 Massarosa (LU)
            phone: +39 0584 962313 <tel:+39%200584%20962313>
            fax: +39 0584 1660272 <tel:+39%200584%20166%200272>
            mob: +39 339 8844549 <tel:+39%20339%20884%204549>

            http://www.geo-solutions.it
            http://twitter.com/geosolutions_it
            <http://twitter.com/geosolutions_it&gt;

            *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

            Le informazioni contenute in questo messaggio di posta
            elettronica e/o nel/i file/s allegato/i sono da
            considerarsi strettamente riservate. Il loro utilizzo è
            consentito esclusivamente al destinatario del messaggio,
            per le finalità indicate nel messaggio stesso. Qualora
            riceviate questo messaggio senza esserne il destinatario,
            Vi preghiamo cortesemente di darcene notizia via e-mail e
            di procedere alla distruzione del messaggio stesso,
            cancellandolo dal Vostro sistema. Conservare il messaggio
            stesso, divulgarlo anche in parte, distribuirlo ad altri
            soggetti, copiarlo, od utilizzarlo per finalità diverse,
            costituisce comportamento contrario ai principi dettati
            dal D.Lgs. 196/2003.

            The information in this message and/or attachments, is
            intended solely for the attention and use of the named
            addressee(s) and may be confidential or proprietary in
            nature or covered by the provisions of privacy act
            (Legislative Decree June, 30 2003, no.196 - Italy's New
            Data Protection Code).Any use not in accord with its
            purpose, any disclosure, reproduction, copying,
            distribution, or either dissemination, either whole or
            partial, is strictly forbidden except previous formal
            approval of the named addressee(s). If you are not the
            intended recipient, please contact immediately the sender
            by telephone, fax or e-mail and delete the information in
            this message that has been received in error. The sender
            does not give any warranty or accept liability as the
            content, accuracy or completeness of sent messages and
            accepts no responsibility for changes made after they
            were sent or for other risks which arise as a result of
            e-mail transmission, viruses, etc.

            -------------------------------------------------------

    -- ==
    GeoServer Professional Services from the experts! Visit
    http://goo.gl/it488V for more information.
    ==

    Ing. Andrea Aime
    @geowolf
    Technical Lead

    GeoSolutions S.A.S.
    Via di Montramito 3/A
    55054 Massarosa (LU)
    phone: +39 0584 962313 <tel:+39%200584%20962313>
    fax: +39 0584 1660272 <tel:+39%200584%20166%200272>
    mob: +39 339 8844549 <tel:+39%20339%20884%204549>

    http://www.geo-solutions.it
    http://twitter.com/geosolutions_it
    <http://twitter.com/geosolutions_it&gt;

    *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

    Le informazioni contenute in questo messaggio di posta
    elettronica e/o nel/i file/s allegato/i sono da considerarsi
    strettamente riservate. Il loro utilizzo è consentito
    esclusivamente al destinatario del messaggio, per le finalità
    indicate nel messaggio stesso. Qualora riceviate questo messaggio
    senza esserne il destinatario, Vi preghiamo cortesemente di
    darcene notizia via e-mail e di procedere alla distruzione del
    messaggio stesso, cancellandolo dal Vostro sistema. Conservare il
    messaggio stesso, divulgarlo anche in parte, distribuirlo ad
    altri soggetti, copiarlo, od utilizzarlo per finalità diverse,
    costituisce comportamento contrario ai principi dettati dal
    D.Lgs. 196/2003.

    The information in this message and/or attachments, is intended
    solely for the attention and use of the named addressee(s) and
    may be confidential or proprietary in nature or covered by the
    provisions of privacy act (Legislative Decree June, 30 2003,
    no.196 - Italy's New Data Protection Code).Any use not in accord
    with its purpose, any disclosure, reproduction, copying,
    distribution, or either dissemination, either whole or partial,
    is strictly forbidden except previous formal approval of the
    named addressee(s). If you are not the intended recipient, please
    contact immediately the sender by telephone, fax or e-mail and
    delete the information in this message that has been received in
    error. The sender does not give any warranty or accept liability
    as the content, accuracy or completeness of sent messages and
    accepts no responsibility for changes made after they were sent
    or for other risks which arise as a result of e-mail
    transmission, viruses, etc.

    -------------------------------------------------------

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

-------------------------------------------------------

8

I've also seen issues with GeoServer potentially losing the right-most and
bottom-most pixels when using rendering transformations to style raster
data. I've reproduced this in 2.9.x and 2.10.x with GeoTIFF coverages and
with raster-to-raster and raster-to-vector transformations. This can create
noticeable artifacts when using tiling.

The data appears to be getting lost from a Crop operation and I think that
it is when CoverageCrop is called in
org.geotools.renderer.lite.RenderingTransformationHelper but I haven't had a
chance to do more testing yet.

Steve Ikeoka

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/DynamicColorMap-tp5315555p5315917.html
Sent from the GeoServer - Dev mailing list archive at Nabble.com.

9

You could start by providing a reproducable test case in Jira :slight_smile:

Cheers
Andrea

···

On Wed, Apr 5, 2017 at 4:40 PM, sikeoka <steve.ikeoka@anonymised.com> wrote:

I’ve also seen issues with GeoServer potentially losing the right-most and
bottom-most pixels when using rendering transformations to style raster
data. I’ve reproduced this in 2.9.x and 2.10.x with GeoTIFF coverages and
with raster-to-raster and raster-to-vector transformations. This can create
noticeable artifacts when using tiling.

The data appears to be getting lost from a Crop operation and I think that
it is when CoverageCrop is called in
org.geotools.renderer.lite.RenderingTransformationHelper but I haven’t had a
chance to do more testing yet.

Steve Ikeoka

–
View this message in context: http://osgeo-org.1560.x6.nabble.com/DynamicColorMap-tp5315555p5315917.html
Sent from the GeoServer - Dev mailing list archive at Nabble.com.

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.com.366…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

10

I don't use the dynamic colormap module but the JIRA ticket in the first post
has a test case using it:
https://osgeo-org.atlassian.net/browse/GEOS-8070

Adding 1 to the envelope width and height here seems to fix the problem:

Old: Rectangle rect = new Rectangle(minx, miny, (maxx - minx), (maxy -
miny));
New: Rectangle rect = new Rectangle(minx, miny, (maxx - minx + 1), (maxy -
miny + 1));

A quick search of the GeoServer code found another case where this is also
done:

Steve Ikeoka

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/DynamicColorMap-tp5315555p5315956.html
Sent from the GeoServer - Dev mailing list archive at Nabble.com.

11

Hmm, I think the issue is in that area though:

I changed this:

MathTransform g2w = reader.getOriginalGridToWorld(PixelInCell.CELL_CENTER);

to this:

MathTransform g2w = reader.getOriginalGridToWorld(PixelInCell.CELL_CORNER);

(full diff here: https://github.com/dvntucker/geotools/pull/6/files)

And it seems to fix the issue, which sort of make sense intuitively, given that it seemed that a fraction of the pixels on the right and bottom were cut off. It also matches what the GridCoverageRenderer currently does:

https://github.com/geotools/geotools/blob/master/modules/library/render/src/main/java/org/geotools/renderer/lite/gridcoverage2d/GridCoverageRenderer.java#L304

Does anyone see any potential issues with this change? All of the tests in GeoServer and GeoTools pass for me with this change. If no one sees anything obvious then I’ll try to write a unit test for the issue and do a proper PR.

···

On Wed, Apr 5, 2017 at 11:13 AM, sikeoka <steve.ikeoka@anonymised.com> wrote:

I don’t use the dynamic colormap module but the JIRA ticket in the first post
has a test case using it:
https://osgeo-org.atlassian.net/browse/GEOS-8070

Adding 1 to the envelope width and height here seems to fix the problem:
https://github.com/geotools/geotools/blob/master/modules/library/render/src/main/java/org/geotools/renderer/lite/RenderingTransformationHelper.java#L127
Old: Rectangle rect = new Rectangle(minx, miny, (maxx - minx), (maxy -
miny));
New: Rectangle rect = new Rectangle(minx, miny, (maxx - minx + 1), (maxy -
miny + 1));

A quick search of the GeoServer code found another case where this is also
done:
https://github.com/geoserver/geoserver/blob/master/src/wcs2_0/src/main/java/org/geoserver/wcs2_0/ScalingPolicy.java#L288

Steve Ikeoka

–
View this message in context: http://osgeo-org.1560.x6.nabble.com/DynamicColorMap-tp5315555p5315956.html

Sent from the GeoServer - Dev mailing list archive at Nabble.com.

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.com.366…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

12

On Thu, Apr 6, 2017 at 1:37 AM, Devon Tucker <dtucker@anonymised.com>
wrote:

Hmm, I think the issue is in that area though:

I changed this:

MathTransform g2w = reader.getOriginalGridToWorld(
PixelInCell.CELL_CENTER);

to this:

MathTransform g2w = reader.getOriginalGridToWorld(
PixelInCell.CELL_CORNER);

(full diff here: https://github.com/dvntucker/geotools/pull/6/files)

And it seems to fix the issue, which sort of make sense intuitively, given
that it seemed that a fraction of the pixels on the right and bottom were
cut off. It also matches what the GridCoverageRenderer currently does:

https://github.com/geotools/geotools/blob/master/modules/lib
rary/render/src/main/java/org/geotools/renderer/lite/gridcov
erage2d/GridCoverageRenderer.java#L304

Does anyone see any potential issues with this change? All of the tests in
GeoServer and GeoTools pass for me with this change. If no one sees
anything obvious then I'll try to write a unit test for the issue and do a
proper PR.

I'd go ahead with a PR

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

13

I tested changing CELL_CENTER to CELL_CORNER and it only seemed to fix the
extra cropping about half of the time for me. The gap can be up to one
pixel width with the current code on master and changing CELL_CENTER to
CELL_CORNER reduces the gap to up to a half pixel width but does not
eliminate it completely.

Steve Ikeoka

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/DynamicColorMap-tp5315555p5316152.html
Sent from the GeoServer - Dev mailing list archive at Nabble.com.

14

Thanks Steve, I’ll play around with it a bit more. I didn’t have time to fully explore the results yesterday.

···

On Thu, Apr 6, 2017 at 7:42 AM, sikeoka <steve.ikeoka@anonymised.com> wrote:

I tested changing CELL_CENTER to CELL_CORNER and it only seemed to fix the
extra cropping about half of the time for me. The gap can be up to one
pixel width with the current code on master and changing CELL_CENTER to
CELL_CORNER reduces the gap to up to a half pixel width but does not
eliminate it completely.

Steve Ikeoka

–
View this message in context: http://osgeo-org.1560.x6.nabble.com/DynamicColorMap-tp5315555p5316152.html

Sent from the GeoServer - Dev mailing list archive at Nabble.com.

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.com.366…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

15

Steve do you have an example layer and request for which the CELL_CORNER fix doesn’t work? Playing around with the water temperature layer in my ticket it I don’t see the issue anywhere.

···

On Thu, Apr 6, 2017 at 9:46 AM, Devon Tucker <dtucker@anonymised.com> wrote:

Thanks Steve, I’ll play around with it a bit more. I didn’t have time to fully explore the results yesterday.

On Thu, Apr 6, 2017 at 7:42 AM, sikeoka <steve.ikeoka@anonymised.com> wrote:

I tested changing CELL_CENTER to CELL_CORNER and it only seemed to fix the
extra cropping about half of the time for me. The gap can be up to one
pixel width with the current code on master and changing CELL_CENTER to
CELL_CORNER reduces the gap to up to a half pixel width but does not
eliminate it completely.

Steve Ikeoka

–
View this message in context: http://osgeo-org.1560.x6.nabble.com/DynamicColorMap-tp5315555p5316152.html

Sent from the GeoServer - Dev mailing list archive at Nabble.com.

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.comrge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

16

I downloaded the 10.47 MB GeoTIFF from here and loaded it into GeoServer:
http://www.naturalearthdata.com/downloads/50m-raster-data/50m-gray-earth/

I created a new style called contour_test (full SLD at the end). I then
used the following URLs against a GeoServer that is patched with the
CELL_CORNER fix. All three URLs work fine when adding 1 to the width and
height and all three have significant gaps without either fix. I tested
against GeoServer 2.10.1 using native JAI.

works fine:
http://localhost:8080/geoserver/wms?service=WMS&version=1.3.0&request=GetMap&format=image/png&crs=CRS:84&layers=gis:GRAY_50M_SR_W&styles=contour_test&bgcolor=0xFF0000&width=800&height=800&bbox=71.0,35.0,71.08,35.08

has missing contours on the right side of the image:
http://localhost:8080/geoserver/wms?service=WMS&version=1.3.0&request=GetMap&format=image/png&crs=CRS:84&layers=gis:GRAY_50M_SR_W&styles=contour_test&bgcolor=0xFF0000&width=800&height=800&bbox=71.0,35.0,71.09,35.09

works fine:
http://localhost:8080/geoserver/wms?service=WMS&version=1.3.0&request=GetMap&format=image/png&crs=CRS:84&layers=gis:GRAY_50M_SR_W&styles=contour_test&bgcolor=0xFF0000&width=800&height=800&bbox=71.0,35.0,71.1,35.1

<?xml version="1.0" encoding="ISO-8859-1"?>
<StyledLayerDescriptor version="1.0.0" xmlns="http://www.opengis.net/sld&quot;
xmlns:ogc="http://www.opengis.net/ogc&quot;
  xmlns:xlink="http://www.w3.org/1999/xlink&quot;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot;
  xsi:schemaLocation="http://www.opengis.net/sld
http://schemas.opengis.net/sld/1.0.0/StyledLayerDescriptor.xsd&quot;&gt;
    <NamedLayer>
        <Name>contour_test</Name>
        <LayerFeatureConstraints>
            <FeatureTypeConstraint/>
        </LayerFeatureConstraints>
        <UserStyle>
            <FeatureTypeStyle>
                <Transformation>
                    <ogc:Function name="ras:Contour">
                        <ogc:Function name="parameter">
                            <ogc:Literal>data</ogc:Literal>
                        </ogc:Function>
                        <ogc:Function name="parameter">
                            <ogc:Literal>interval</ogc:Literal>
                            <ogc:Literal>5</ogc:Literal>
                        </ogc:Function>
                    </ogc:Function>
                </Transformation>
                <Rule>
                    <LineSymbolizer>
                        <Stroke>
                            <CssParameter
name="stroke">#000000</CssParameter>
                        </Stroke>
                    </LineSymbolizer>
                </Rule>
            </FeatureTypeStyle>
        </UserStyle>
    </NamedLayer>
</StyledLayerDescriptor>

Steve Ikeoka

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/DynamicColorMap-tp5315555p5316209.html
Sent from the GeoServer - Dev mailing list archive at Nabble.com.

17

Thanks Steve, you’re right, just changing the cell anchor does not fix it.

I’m actually a decent bit confused by this bug at this point. I think it may be related to cropping the coverage after the read? The rendering transform pipeline increases the size of the bounding box to ensure that at least one pixel is returned, then in the very next step the coverage is cropped to the exact same envelope in case the coverage store returns more than the requested bounds. This is where things go wrong, because the bbox of the coverage that returns ends up being smaller than the one requested for the crop. I put the coverage + the bboxes into QGIS to see what’s going on:

http://imgur.com/a/xxR81

Apologies for the awful image, I’ll try to explain it:

So, I think something is going wrong in that crop operating (https://github.com/geotools/geotools/blob/master/modules/library/render/src/main/java/org/geotools/renderer/lite/RenderingTransformationHelper.java#L152). The BBOX handed in to the crop is the biggest expanded one, but for whatever reason the resulting coverage has the smaller bbox. Not quite sure what’s going on there. I’m continuing to look at it today.

Cheers

···

On Thu, Apr 6, 2017 at 2:47 PM, sikeoka <steve.ikeoka@anonymised.com> wrote:

I downloaded the 10.47 MB GeoTIFF from here and loaded it into GeoServer:
http://www.naturalearthdata.com/downloads/50m-raster-data/50m-gray-earth/

I created a new style called contour_test (full SLD at the end). I then
used the following URLs against a GeoServer that is patched with the
CELL_CORNER fix. All three URLs work fine when adding 1 to the width and
height and all three have significant gaps without either fix. I tested
against GeoServer 2.10.1 using native JAI.

works fine:
http://localhost:8080/geoserver/wms?service=WMS&version=1.3.0&request=GetMap&format=image/png&crs=CRS:84&layers=gis:GRAY_50M_SR_W&styles=contour_test&bgcolor=0xFF0000&width=800&height=800&bbox=71.0,35.0,71.08,35.08

has missing contours on the right side of the image:
http://localhost:8080/geoserver/wms?service=WMS&version=1.3.0&request=GetMap&format=image/png&crs=CRS:84&layers=gis:GRAY_50M_SR_W&styles=contour_test&bgcolor=0xFF0000&width=800&height=800&bbox=71.0,35.0,71.09,35.09

works fine:
http://localhost:8080/geoserver/wms?service=WMS&version=1.3.0&request=GetMap&format=image/png&crs=CRS:84&layers=gis:GRAY_50M_SR_W&styles=contour_test&bgcolor=0xFF0000&width=800&height=800&bbox=71.0,35.0,71.1,35.1

<?xml version="1.0" encoding="ISO-8859-1"?>



contour_test






<ogc:Function name=“ras:Contour”>
<ogc:Function name=“parameter”>
ogc:Literaldata</ogc:Literal>
</ogc:Function>
<ogc:Function name=“parameter”>
ogc:Literalinterval</ogc:Literal>
ogc:Literal5</ogc:Literal>
</ogc:Function>
</ogc:Function>




#000000






Steve Ikeoka

–
View this message in context: http://osgeo-org.1560.x6.nabble.com/DynamicColorMap-tp5315555p5316209.html

Sent from the GeoServer - Dev mailing list archive at Nabble.com.

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.com.366…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel