[Geoserver-devel] ResourceInfo as process input

Hi folks,

I have a scenario where I want to write a GeoServer WPS process that takes in raster input but then passes the raster off to another service to do the actual processing. The other service takes the path to the raster as it’s input. So my problem is that it seems the established way to write a raster process is to take a GridCoverage object as input, and from what I can tell there is no easy way to get the original source of the coverage (which makes sense). So right now to pull this off the grid coverage is written out to a temporary geotiff file which is then passed off. I would like to avoid this copy step and work on the original file directly if possible.

So I guess my first question is if folks have done anything like this and if so how they approached it. If not I was thinking of the following solution:

Rather than declare my input of type GridCoverage, declare it of type CoverageInfo. From there I can get to the store and then to the file backing it. I am also aware that a grid coverage could potentially come from a non-file source. For these cases I still plan to keep the “copy to geotiff” in place as a fallback.

Now I could just have my process take a string as input, interpret that as a layer name, and then look it up in the catalog. But I was thinking adding a bit of infrastructure for it (like what’s there for accepting GridCoverage and FeatureCollection) could be of general use. Also with that approach I lose some niceties like a dropdown of layer names in the request builder.

I sketched out an implementation of this and the approach I took was to encode the input as a reference along the same lines as internal WFS/WCS calls are made. In XML it looks like this:

wps:Input
ows:Identifierinput</ows:Identifier>
<wps:Reference xlink:href=“http://geoserver/catalog/topp:states”/>
</wps:Input>

With that it required a PPIO and an InputProvider and then it worked like a charm.

Looking for feedback for this one. If folks think this is worth while I will happily prep a pull request. Although I know it’s close to feature freeze time which is fine. Happy to wait.

Thanks!

-Justin

I would make a new interface that allows access to the ResourceInfo …

interface AccessInfo {
ResoruceInfo toInfo();
}

And then wrap the GridCoverage in a proxy that also supports a AccesInfo:

class GridCoveageWrapper implements GridCoverage, AccessInfo {
GridCoveage delegate;
ResourceInfo info;

}

Or similar, if I was smarter I would make it as a DynamicProxy so I could clean apply InfoWrapper around any parameter … since you are a smart guy I bet you could make that work :slight_smile:

Jody

···

On 26 August 2015 at 08:51, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I have a scenario where I want to write a GeoServer WPS process that takes in raster input but then passes the raster off to another service to do the actual processing. The other service takes the path to the raster as it’s input. So my problem is that it seems the established way to write a raster process is to take a GridCoverage object as input, and from what I can tell there is no easy way to get the original source of the coverage (which makes sense). So right now to pull this off the grid coverage is written out to a temporary geotiff file which is then passed off. I would like to avoid this copy step and work on the original file directly if possible.

So I guess my first question is if folks have done anything like this and if so how they approached it. If not I was thinking of the following solution:

Rather than declare my input of type GridCoverage, declare it of type CoverageInfo. From there I can get to the store and then to the file backing it. I am also aware that a grid coverage could potentially come from a non-file source. For these cases I still plan to keep the “copy to geotiff” in place as a fallback.

Now I could just have my process take a string as input, interpret that as a layer name, and then look it up in the catalog. But I was thinking adding a bit of infrastructure for it (like what’s there for accepting GridCoverage and FeatureCollection) could be of general use. Also with that approach I lose some niceties like a dropdown of layer names in the request builder.

I sketched out an implementation of this and the approach I took was to encode the input as a reference along the same lines as internal WFS/WCS calls are made. In XML it looks like this:

wps:Input
ows:Identifierinput</ows:Identifier>
<wps:Reference xlink:href=“http://geoserver/catalog/topp:states”/>
</wps:Input>

With that it required a PPIO and an InputProvider and then it worked like a charm.

Looking for feedback for this one. If folks think this is worth while I will happily prep a pull request. Although I know it’s close to feature freeze time which is fine. Happy to wait.

Thanks!

-Justin



Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Jody Garnett

I see your approach, when planning “to keep the ‘copy to geotif’ in place as a fallback” do you end up publishing the process twice or something - I guess I am not clear on how that shows up as a ProcessDescription. The process would end up taking either a GridCoverage or a CoverageInfo reference right?

It seems a bit odd (or at least non-transparent) to advertise the requirement for a CoverageInfo (rather than a grid coverage) for what is essentially an optimization (avoiding writing out a temporary GeoTIFF).

I do like the idea of allowing Processes to work with CoverageInfo or similar when they are specific about working with the GeoServer configuration (we already have publish process steps). So if you want to power through and submit a pull request before the cut-off will be happy to review.

···

On 26 August 2015 at 08:51, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I have a scenario where I want to write a GeoServer WPS process that takes in raster input but then passes the raster off to another service to do the actual processing. The other service takes the path to the raster as it’s input. So my problem is that it seems the established way to write a raster process is to take a GridCoverage object as input, and from what I can tell there is no easy way to get the original source of the coverage (which makes sense). So right now to pull this off the grid coverage is written out to a temporary geotiff file which is then passed off. I would like to avoid this copy step and work on the original file directly if possible.

So I guess my first question is if folks have done anything like this and if so how they approached it. If not I was thinking of the following solution:

Rather than declare my input of type GridCoverage, declare it of type CoverageInfo. From there I can get to the store and then to the file backing it. I am also aware that a grid coverage could potentially come from a non-file source. For these cases I still plan to keep the “copy to geotiff” in place as a fallback.

Now I could just have my process take a string as input, interpret that as a layer name, and then look it up in the catalog. But I was thinking adding a bit of infrastructure for it (like what’s there for accepting GridCoverage and FeatureCollection) could be of general use. Also with that approach I lose some niceties like a dropdown of layer names in the request builder.

I sketched out an implementation of this and the approach I took was to encode the input as a reference along the same lines as internal WFS/WCS calls are made. In XML it looks like this:

wps:Input
ows:Identifierinput</ows:Identifier>
<wps:Reference xlink:href=“http://geoserver/catalog/topp:states”/>
</wps:Input>

With that it required a PPIO and an InputProvider and then it worked like a charm.

Looking for feedback for this one. If folks think this is worth while I will happily prep a pull request. Although I know it’s close to feature freeze time which is fine. Happy to wait.

Thanks!

-Justin



Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Jody Garnett

On Wed, Aug 26, 2015 at 10:28 AM, Jody Garnett <jody.garnett@anonymised.com>
wrote:

I would make a new interface that allows access to the ResourceInfo ....

interface AccessInfo {
   ResoruceInfo toInfo();
}

And then wrap the GridCoverage in a proxy that also supports a AccesInfo:

class GridCoveageWrapper implements GridCoverage, AccessInfo {
   GridCoveage delegate;
   ResourceInfo info;
   ...
}

Or similar, if I was smarter I would make it as a DynamicProxy so I could
clean apply InfoWrapper around any parameter ... since you are a smart guy
I bet you could make that work :slight_smile:

Sorry but what does this buy me over the approach I thought of? The
CoverageInfo already provides access to the grid coverage... so this seems
like just adding a superfluous interface?

Jody

--
Jody Garnett

On 26 August 2015 at 08:51, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I have a scenario where I want to write a GeoServer WPS process that
takes in raster input but then passes the raster off to another service to
do the actual processing. The other service takes the path to the raster as
it's input. So my problem is that it seems the established way to write a
raster process is to take a GridCoverage object as input, and from what I
can tell there is no easy way to get the original source of the coverage
(which makes sense). So right now to pull this off the grid coverage is
written out to a temporary geotiff file which is then passed off. I would
like to avoid this copy step and work on the original file directly if
possible.

So I guess my first question is if folks have done anything like this and
if so how they approached it. If not I was thinking of the following
solution:

Rather than declare my input of type GridCoverage, declare it of type
CoverageInfo. From there I can get to the store and then to the file
backing it. I am also aware that a grid coverage could potentially come
from a non-file source. For these cases I still plan to keep the "copy to
geotiff" in place as a fallback.

Now I could just have my process take a string as input, interpret that
as a layer name, and then look it up in the catalog. But I was thinking
adding a bit of infrastructure for it (like what's there for accepting
GridCoverage and FeatureCollection) could be of general use. Also with that
approach I lose some niceties like a dropdown of layer names in the request
builder.

I sketched out an implementation of this and the approach I took was to
encode the input as a reference along the same lines as internal WFS/WCS
calls are made. In XML it looks like this:

  <wps:Input>
     <ows:Identifier>input</ows:Identifier>
     <wps:Reference xlink:href="http://geoserver/catalog/topp:states&quot;/&gt;
  </wps:Input>

With that it required a PPIO and an InputProvider and then it worked like
a charm.

Looking for feedback for this one. If folks think this is worth while I
will happily prep a pull request. Although I know it's close to feature
freeze time which is fine. Happy to wait.

Thanks!

-Justin

------------------------------------------------------------------------------

_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Hi Justin,
the approach makes sense to me.
So, in this case, it will be up to the process to decide whether to dump to tiff or keep
the original file.

I guess this would make a lot of sense for wrappers to stuff such as gdal_translate… the downside is
that you cannot chain the process anymore with other processes that generate coverages, or perform
selections that WCS could do for you (bbox, bands).

If you want to go for a more general approach, you could either add two parameters, or create a new
type of input that can return either file or GridCoverage, and see if setting up converters will allow
for process chaining.

Anyways, just thinking out loud here

Cheers
Andrea

···

On Wed, Aug 26, 2015 at 5:51 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I have a scenario where I want to write a GeoServer WPS process that takes in raster input but then passes the raster off to another service to do the actual processing. The other service takes the path to the raster as it’s input. So my problem is that it seems the established way to write a raster process is to take a GridCoverage object as input, and from what I can tell there is no easy way to get the original source of the coverage (which makes sense). So right now to pull this off the grid coverage is written out to a temporary geotiff file which is then passed off. I would like to avoid this copy step and work on the original file directly if possible.

So I guess my first question is if folks have done anything like this and if so how they approached it. If not I was thinking of the following solution:

Rather than declare my input of type GridCoverage, declare it of type CoverageInfo. From there I can get to the store and then to the file backing it. I am also aware that a grid coverage could potentially come from a non-file source. For these cases I still plan to keep the “copy to geotiff” in place as a fallback.

Now I could just have my process take a string as input, interpret that as a layer name, and then look it up in the catalog. But I was thinking adding a bit of infrastructure for it (like what’s there for accepting GridCoverage and FeatureCollection) could be of general use. Also with that approach I lose some niceties like a dropdown of layer names in the request builder.

I sketched out an implementation of this and the approach I took was to encode the input as a reference along the same lines as internal WFS/WCS calls are made. In XML it looks like this:

wps:Input
ows:Identifierinput</ows:Identifier>
<wps:Reference xlink:href=“http://geoserver/catalog/topp:states”/>
</wps:Input>

With that it required a PPIO and an InputProvider and then it worked like a charm.

Looking for feedback for this one. If folks think this is worth while I will happily prep a pull request. Although I know it’s close to feature freeze time which is fine. Happy to wait.

Thanks!

-Justin



Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Wed, Aug 26, 2015 at 10:35 AM, Jody Garnett <jody.garnett@anonymised.com>
wrote:

I see your approach, when planning "to keep the 'copy to geotif' in place
as a fallback" do you end up publishing the process twice or something - I
guess I am not clear on how that shows up as a ProcessDescription. The
process would end up taking either a GridCoverage or a CoverageInfo
reference right?

Right. I guess it depends on the need. If we can assume the input will
always be a GeoServer layer then CoverageInfo alone is good enough. But if
not then I guess yeah, two inputs would be another way to do it. Now your
suggestion of another interface or a proxy makes more sense, I wasn't
following it before.

It seems a bit odd (or at least non-transparent) to advertise the
requirement for a CoverageInfo (rather than a grid coverage) for what is
essentially an optimization (avoiding writing out a temporary GeoTIFF).

I can think of other reasons for wanting the CoverageInfo like getting at
the other metadata attached to it that isn't easily accessible from the
GridCoverage.

I do *like* the idea of allowing Processes to work with CoverageInfo or
similar when they are specific about working with the GeoServer
configuration (we already have publish process steps). So if you want to
power through and submit a pull request before the cut-off will be happy to
review.

--
Jody Garnett

On 26 August 2015 at 08:51, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I have a scenario where I want to write a GeoServer WPS process that
takes in raster input but then passes the raster off to another service to
do the actual processing. The other service takes the path to the raster as
it's input. So my problem is that it seems the established way to write a
raster process is to take a GridCoverage object as input, and from what I
can tell there is no easy way to get the original source of the coverage
(which makes sense). So right now to pull this off the grid coverage is
written out to a temporary geotiff file which is then passed off. I would
like to avoid this copy step and work on the original file directly if
possible.

So I guess my first question is if folks have done anything like this and
if so how they approached it. If not I was thinking of the following
solution:

Rather than declare my input of type GridCoverage, declare it of type
CoverageInfo. From there I can get to the store and then to the file
backing it. I am also aware that a grid coverage could potentially come
from a non-file source. For these cases I still plan to keep the "copy to
geotiff" in place as a fallback.

Now I could just have my process take a string as input, interpret that
as a layer name, and then look it up in the catalog. But I was thinking
adding a bit of infrastructure for it (like what's there for accepting
GridCoverage and FeatureCollection) could be of general use. Also with that
approach I lose some niceties like a dropdown of layer names in the request
builder.

I sketched out an implementation of this and the approach I took was to
encode the input as a reference along the same lines as internal WFS/WCS
calls are made. In XML it looks like this:

  <wps:Input>
     <ows:Identifier>input</ows:Identifier>
     <wps:Reference xlink:href="http://geoserver/catalog/topp:states&quot;/&gt;
  </wps:Input>

With that it required a PPIO and an InputProvider and then it worked like
a charm.

Looking for feedback for this one. If folks think this is worth while I
will happily prep a pull request. Although I know it's close to feature
freeze time which is fine. Happy to wait.

Thanks!

-Justin

------------------------------------------------------------------------------

_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Hey Andrea,

···

On Wed, Aug 26, 2015 at 11:03 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi Justin,
the approach makes sense to me.
So, in this case, it will be up to the process to decide whether to dump to tiff or keep
the original file.

Correct.

I guess this would make a lot of sense for wrappers to stuff such as gdal_translate… the downside is
that you cannot chain the process anymore with other processes that generate coverages, or perform
selections that WCS could do for you (bbox, bands).

Yeah, which would definitely be nice to maintain cleanly if possible.

If you want to go for a more general approach, you could either add two parameters, or create a new
type of input that can return either file or GridCoverage, and see if setting up converters will allow
for process chaining.

Yeah, the latter is kind of along the same lines of what Jody is suggesting. Since GridCoverage is an interface we could get fancy at implementing it in a way that would allow the process writer to check if it is of a certain type that could provide access to the coverage. Something like.

GridCoverage input;
CoverageInfo info;
if (input instanceof WWWGridCoverage) {
info = ((WWWGridCoverage)input).coverageInfo()
}

Also thinking out loud…

Anyways, just thinking out loud here

Cheers
Andrea

On Wed, Aug 26, 2015 at 5:51 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I have a scenario where I want to write a GeoServer WPS process that takes in raster input but then passes the raster off to another service to do the actual processing. The other service takes the path to the raster as it’s input. So my problem is that it seems the established way to write a raster process is to take a GridCoverage object as input, and from what I can tell there is no easy way to get the original source of the coverage (which makes sense). So right now to pull this off the grid coverage is written out to a temporary geotiff file which is then passed off. I would like to avoid this copy step and work on the original file directly if possible.

So I guess my first question is if folks have done anything like this and if so how they approached it. If not I was thinking of the following solution:

Rather than declare my input of type GridCoverage, declare it of type CoverageInfo. From there I can get to the store and then to the file backing it. I am also aware that a grid coverage could potentially come from a non-file source. For these cases I still plan to keep the “copy to geotiff” in place as a fallback.

Now I could just have my process take a string as input, interpret that as a layer name, and then look it up in the catalog. But I was thinking adding a bit of infrastructure for it (like what’s there for accepting GridCoverage and FeatureCollection) could be of general use. Also with that approach I lose some niceties like a dropdown of layer names in the request builder.

I sketched out an implementation of this and the approach I took was to encode the input as a reference along the same lines as internal WFS/WCS calls are made. In XML it looks like this:

wps:Input
ows:Identifierinput</ows:Identifier>
<wps:Reference xlink:href=“http://geoserver/catalog/topp:states”/>
</wps:Input>

With that it required a PPIO and an InputProvider and then it worked like a charm.

Looking for feedback for this one. If folks think this is worth while I will happily prep a pull request. Although I know it’s close to feature freeze time which is fine. Happy to wait.

Thanks!

-Justin



Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Wed, Aug 26, 2015 at 7:14 PM, Justin Deoliveira <jdeolive@anonymised.com>
wrote:

Yeah, the latter is kind of along the same lines of what Jody is

suggesting. Since GridCoverage is an interface we could get fancy at
implementing it in a way that would allow the process writer to check if it
is of a certain type that could provide access to the coverage. Something
like.

GridCoverage input;
CoverageInfo info;
if (input instanceof WWWGridCoverage) {
   info = ((WWWGridCoverage)input).coverageInfo()
}

Also thinking out loud...

I believe that it might work, one just needs to make sure the raster in it
is delay-loaded
until actually needed. Most of the information attached to the coverage
could be derived
from the reader directly, calling the various getOriginal* methods, without
actually
loading the raster data

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

On Wed, Aug 26, 2015 at 11:17 AM, Andrea Aime <andrea.aime@anonymised.com>
wrote:

On Wed, Aug 26, 2015 at 7:14 PM, Justin Deoliveira <jdeolive@anonymised.com>
wrote:

Yeah, the latter is kind of along the same lines of what Jody is

suggesting. Since GridCoverage is an interface we could get fancy at
implementing it in a way that would allow the process writer to check if it
is of a certain type that could provide access to the coverage. Something
like.

GridCoverage input;
CoverageInfo info;
if (input instanceof WWWGridCoverage) {
   info = ((WWWGridCoverage)input).coverageInfo()
}

Also thinking out loud...

I believe that it might work, one just needs to make sure the raster in it
is delay-loaded
until actually needed. Most of the information attached to the coverage
could be derived
from the reader directly, calling the various getOriginal* methods,
without actually
loading the raster data

Right. I might experiment with this option... i'll take a look and see how

hairy it gets and report back :slight_smile:

Thanks a lot for the quick feedback Jody and Andrea! Much appreciated.

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

Sorry if my example was not clear,

The answer is that a GridCoverage would be sent in, and you could do and instance of check to access to a ResourceInfo if needed.

String name name( GridCoverage coverage ){
if( coverage instanceof AccessInfo ){
ResourceInfo info = ((AccessInfo)coverage).toInfo();
return info.getName();
}
return “unnamed”;
}

···

On 26 August 2015 at 09:52, Justin Deoliveira <jdeolive@anonymised.com> wrote:


Jody Garnett

On Wed, Aug 26, 2015 at 10:28 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

I would make a new interface that allows access to the ResourceInfo …

interface AccessInfo {
ResoruceInfo toInfo();
}

And then wrap the GridCoverage in a proxy that also supports a AccesInfo:

class GridCoveageWrapper implements GridCoverage, AccessInfo {
GridCoveage delegate;
ResourceInfo info;

}

Or similar, if I was smarter I would make it as a DynamicProxy so I could clean apply InfoWrapper around any parameter … since you are a smart guy I bet you could make that work :slight_smile:

Sorry but what does this buy me over the approach I thought of? The CoverageInfo already provides access to the grid coverage… so this seems like just adding a superfluous interface?

Jody


Jody Garnett

On 26 August 2015 at 08:51, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I have a scenario where I want to write a GeoServer WPS process that takes in raster input but then passes the raster off to another service to do the actual processing. The other service takes the path to the raster as it’s input. So my problem is that it seems the established way to write a raster process is to take a GridCoverage object as input, and from what I can tell there is no easy way to get the original source of the coverage (which makes sense). So right now to pull this off the grid coverage is written out to a temporary geotiff file which is then passed off. I would like to avoid this copy step and work on the original file directly if possible.

So I guess my first question is if folks have done anything like this and if so how they approached it. If not I was thinking of the following solution:

Rather than declare my input of type GridCoverage, declare it of type CoverageInfo. From there I can get to the store and then to the file backing it. I am also aware that a grid coverage could potentially come from a non-file source. For these cases I still plan to keep the “copy to geotiff” in place as a fallback.

Now I could just have my process take a string as input, interpret that as a layer name, and then look it up in the catalog. But I was thinking adding a bit of infrastructure for it (like what’s there for accepting GridCoverage and FeatureCollection) could be of general use. Also with that approach I lose some niceties like a dropdown of layer names in the request builder.

I sketched out an implementation of this and the approach I took was to encode the input as a reference along the same lines as internal WFS/WCS calls are made. In XML it looks like this:

wps:Input
ows:Identifierinput</ows:Identifier>
<wps:Reference xlink:href=“http://geoserver/catalog/topp:states”/>
</wps:Input>

With that it required a PPIO and an InputProvider and then it worked like a charm.

Looking for feedback for this one. If folks think this is worth while I will happily prep a pull request. Although I know it’s close to feature freeze time which is fine. Happy to wait.

Thanks!

-Justin



Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Hey guys,

I took another look at this trying to implement what Jody suggested… and I don’t think it will quite work with a wrapper or a proxy. Reason being that most raster processes actually take GridCoverage2D rather than GridCoverage. I think trying to wrap that class will be a bit messy and proxying a class would require a CGLIB dependency… all in all getting to be a bit hairy. Staying along the same line though I think introducing a new interface in GeoTools, something named CoverageSupplier (modelled after guava/java8 supplier interface) would work. Basic idea would look like:

interface CoverageSupplier extends Supplier;

With that it would be easy enough for GeoServer to plug in an implementation that provides access to the catalog metadata when it is around. Although I am not sure how to do this and make the process GeoServer agnostic… another class or interface I guess to represent all of that same metadata… maybe just a simple map structure would do:

interface CoverageSupplier extends Supplier {
Map<String,Object> metadata();
}

This would give me what I need in a general way. If folks think that is a good idea I can continue down that path (and btw I would want to add the equivalent supplier interface for FeatureCollection). If not though I think I may just fall back to my original approach which is to be able to declare an input directly on CoverageInfo.

Thanks!

-Justin

···

On Wed, Aug 26, 2015 at 12:23 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Sorry if my example was not clear,

The answer is that a GridCoverage would be sent in, and you could do and instance of check to access to a ResourceInfo if needed.

String name name( GridCoverage coverage ){
if( coverage instanceof AccessInfo ){
ResourceInfo info = ((AccessInfo)coverage).toInfo();
return info.getName();
}
return “unnamed”;
}


Jody Garnett

On 26 August 2015 at 09:52, Justin Deoliveira <jdeolive@anonymised.com> wrote:

On Wed, Aug 26, 2015 at 10:28 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

I would make a new interface that allows access to the ResourceInfo …

interface AccessInfo {
ResoruceInfo toInfo();
}

And then wrap the GridCoverage in a proxy that also supports a AccesInfo:

class GridCoveageWrapper implements GridCoverage, AccessInfo {
GridCoveage delegate;
ResourceInfo info;

}

Or similar, if I was smarter I would make it as a DynamicProxy so I could clean apply InfoWrapper around any parameter … since you are a smart guy I bet you could make that work :slight_smile:

Sorry but what does this buy me over the approach I thought of? The CoverageInfo already provides access to the grid coverage… so this seems like just adding a superfluous interface?

Jody


Jody Garnett

On 26 August 2015 at 08:51, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I have a scenario where I want to write a GeoServer WPS process that takes in raster input but then passes the raster off to another service to do the actual processing. The other service takes the path to the raster as it’s input. So my problem is that it seems the established way to write a raster process is to take a GridCoverage object as input, and from what I can tell there is no easy way to get the original source of the coverage (which makes sense). So right now to pull this off the grid coverage is written out to a temporary geotiff file which is then passed off. I would like to avoid this copy step and work on the original file directly if possible.

So I guess my first question is if folks have done anything like this and if so how they approached it. If not I was thinking of the following solution:

Rather than declare my input of type GridCoverage, declare it of type CoverageInfo. From there I can get to the store and then to the file backing it. I am also aware that a grid coverage could potentially come from a non-file source. For these cases I still plan to keep the “copy to geotiff” in place as a fallback.

Now I could just have my process take a string as input, interpret that as a layer name, and then look it up in the catalog. But I was thinking adding a bit of infrastructure for it (like what’s there for accepting GridCoverage and FeatureCollection) could be of general use. Also with that approach I lose some niceties like a dropdown of layer names in the request builder.

I sketched out an implementation of this and the approach I took was to encode the input as a reference along the same lines as internal WFS/WCS calls are made. In XML it looks like this:

wps:Input
ows:Identifierinput</ows:Identifier>
<wps:Reference xlink:href=“http://geoserver/catalog/topp:states”/>
</wps:Input>

With that it required a PPIO and an InputProvider and then it worked like a charm.

Looking for feedback for this one. If folks think this is worth while I will happily prep a pull request. Although I know it’s close to feature freeze time which is fine. Happy to wait.

Thanks!

-Justin



Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Thanks for looking at it Justin, my bad forgetting that we had swapped these over to a class rather than an interface. The joys of java having two type systems.

Sorry it took me a while to respond - here are my notes/questions/research:

Q: Is it possible to make a fake CoverageInfo around a chained GridCoverage?

Prevent defining the same process twice when defining processes in GeoServer. You could fill in the getGridCoverage(ProgressListener listener, Hints hints) method etc…

This would not help geotools at all, but would prevent you having to introduce CoverageSupplier interfaces.

So this is a quick/clean alternative if you want it. Not great due to the number of CoverageInfo methods.

Q: Do you need to define Supplier / CoverageSupplier at the GeoTools level?

Yep, alternative is wrapper CoverageInfo as described above.

Approach is similar to the Provider interfaces/classes out of uDig:

uDig API is largely the same as you propose, takes some effort to account for blocking, parameters, and so on not sure if you want to consider such things at the moment.

Q: Can we use Property to smuggle extra data in here to avoid duplicating similar data structures?

Not really, even though api is not stable it is entrenched.

Compare:

Parameter is already used to document Map<String,Object> (i.e. key/value) parameters for both datastore factories, functions, and processes.

The kicker here is that we want a more capable data structure to pass values into a processes, similar to have feature model works with Attribute values (each has a getUserData map).

May be worth

Q: Can we use Supplier to get a handle on some of the magic book keeping GeoServer does?

GeoServer tracks resources consumed during process execution using thread locals, in addition to being complex, it is rather opaque to developers writing processes.

We may indeed be able to, so a process defined as taking a supplier …

public static Supplier floor( Supplier dem, Polygon area, ProgressListener progress ) …

Would give a process a chance to register any resources (say file) that needs to be cleaned in the returned metadata map.

Aside: Although Supplier seems to be more complicated then our quick annotations allow for, may be better to keep it for hand created processes.

Q: We already have resource wrappers in GeoServer can we combine forces with Supplier ?

See:

I see no reason why these could not implement Supplier allowing GeoServer to smoothly hand the objects over to a process. Currently they are all kept behind the scenes, used to stage data prior to calling the process.

···

On 1 September 2015 at 06:33, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hey guys,

I took another look at this trying to implement what Jody suggested… and I don’t think it will quite work with a wrapper or a proxy. Reason being that most raster processes actually take GridCoverage2D rather than GridCoverage. I think trying to wrap that class will be a bit messy and proxying a class would require a CGLIB dependency… all in all getting to be a bit hairy. Staying along the same line though I think introducing a new interface in GeoTools, something named CoverageSupplier (modelled after guava/java8 supplier interface) would work. Basic idea would look like:

interface CoverageSupplier extends Supplier;

With that it would be easy enough for GeoServer to plug in an implementation that provides access to the catalog metadata when it is around. Although I am not sure how to do this and make the process GeoServer agnostic… another class or interface I guess to represent all of that same metadata… maybe just a simple map structure would do:

interface CoverageSupplier extends Supplier {
Map<String,Object> metadata();
}

This would give me what I need in a general way. If folks think that is a good idea I can continue down that path (and btw I would want to add the equivalent supplier interface for FeatureCollection). If not though I think I may just fall back to my original approach which is to be able to declare an input directly on CoverageInfo.

Thanks!

-Justin


Jody Garnett

On Wed, Aug 26, 2015 at 12:23 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Sorry if my example was not clear,

The answer is that a GridCoverage would be sent in, and you could do and instance of check to access to a ResourceInfo if needed.

String name name( GridCoverage coverage ){
if( coverage instanceof AccessInfo ){
ResourceInfo info = ((AccessInfo)coverage).toInfo();
return info.getName();
}
return “unnamed”;
}


Jody Garnett

On 26 August 2015 at 09:52, Justin Deoliveira <jdeolive@anonymised.com> wrote:

On Wed, Aug 26, 2015 at 10:28 AM, Jody Garnett <jody.garnett@anonymised.com…> wrote:

I would make a new interface that allows access to the ResourceInfo …

interface AccessInfo {
ResoruceInfo toInfo();
}

And then wrap the GridCoverage in a proxy that also supports a AccesInfo:

class GridCoveageWrapper implements GridCoverage, AccessInfo {
GridCoveage delegate;
ResourceInfo info;

}

Or similar, if I was smarter I would make it as a DynamicProxy so I could clean apply InfoWrapper around any parameter … since you are a smart guy I bet you could make that work :slight_smile:

Sorry but what does this buy me over the approach I thought of? The CoverageInfo already provides access to the grid coverage… so this seems like just adding a superfluous interface?

Jody


Jody Garnett

On 26 August 2015 at 08:51, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I have a scenario where I want to write a GeoServer WPS process that takes in raster input but then passes the raster off to another service to do the actual processing. The other service takes the path to the raster as it’s input. So my problem is that it seems the established way to write a raster process is to take a GridCoverage object as input, and from what I can tell there is no easy way to get the original source of the coverage (which makes sense). So right now to pull this off the grid coverage is written out to a temporary geotiff file which is then passed off. I would like to avoid this copy step and work on the original file directly if possible.

So I guess my first question is if folks have done anything like this and if so how they approached it. If not I was thinking of the following solution:

Rather than declare my input of type GridCoverage, declare it of type CoverageInfo. From there I can get to the store and then to the file backing it. I am also aware that a grid coverage could potentially come from a non-file source. For these cases I still plan to keep the “copy to geotiff” in place as a fallback.

Now I could just have my process take a string as input, interpret that as a layer name, and then look it up in the catalog. But I was thinking adding a bit of infrastructure for it (like what’s there for accepting GridCoverage and FeatureCollection) could be of general use. Also with that approach I lose some niceties like a dropdown of layer names in the request builder.

I sketched out an implementation of this and the approach I took was to encode the input as a reference along the same lines as internal WFS/WCS calls are made. In XML it looks like this:

wps:Input
ows:Identifierinput</ows:Identifier>
<wps:Reference xlink:href=“http://geoserver/catalog/topp:states”/>
</wps:Input>

With that it required a PPIO and an InputProvider and then it worked like a charm.

Looking for feedback for this one. If folks think this is worth while I will happily prep a pull request. Although I know it’s close to feature freeze time which is fine. Happy to wait.

Thanks!

-Justin



Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

On Tue, Sep 1, 2015 at 3:02 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Thanks for looking at it Justin, my bad forgetting that we had swapped
these over to a class rather than an interface. The joys of java having two
type systems.

Sorry it took me a while to respond - here are my notes/questions/research:

*Q: Is it possible to make a fake CoverageInfo around a chained
GridCoverage?*

Prevent defining the same process twice when defining processes in
GeoServer. You could fill in the getGridCoverage(ProgressListener listener,
Hints hints) method etc...

Same process twice?

This would not help geotools at all, but would prevent you having to
introduce CoverageSupplier interfaces.

So this is a quick/clean alternative if you want it. Not great due to the
number of CoverageInfo methods.

Sorry... not following you here.

*Q: Do you need to define Supplier / CoverageSupplier at the GeoTools
level?*

Yep, alternative is wrapper CoverageInfo as described above.

Approach is similar to the Provider interfaces/classes out of uDig:
* IBlockingProvider
<https://github.com/locationtech/udig-platform/blob/00ab09fc89760c7ad1069872840e621d3e4dbf4b/plugins/org.locationtech.udig.core/src/org/locationtech/udig/core/IBlockingProvider.java&gt;
* IProvider
<https://github.com/locationtech/udig-platform/blob/00ab09fc89760c7ad1069872840e621d3e4dbf4b/plugins/org.locationtech.udig.core/src/org/locationtech/udig/core/IProvider.java&gt;
* StaticProvider
<https://github.com/locationtech/udig-platform/blob/00ab09fc89760c7ad1069872840e621d3e4dbf4b/plugins/org.locationtech.udig.core/src/org/locationtech/udig/core/StaticProvider.java&gt;

uDig API is largely the same as you propose, takes some effort to account
for blocking, parameters, and so on not sure if you want to consider such
things at the moment.

I hadn't considered blocking or other I/O concerns...

*Q: Can we use Property to smuggle extra data in here to avoid duplicating
similar data structures?*

Not really, even though api is not stable it is entrenched.

Compare:

* Parameter
<http://docs.geotools.org/latest/javadocs/org/opengis/parameter/Parameter.html&gt;
/ Param
<http://docs.geotools.org/latest/javadocs/org/geotools/data/DataAccessFactory.Param.html&gt;
* Property
<http://docs.geotools.org/latest/javadocs/org/opengis/feature/Property.html&gt; /
Attribute
<http://docs.geotools.org/latest/javadocs/org/opengis/feature/Attribute.html&gt;

Parameter is already used to document Map<String,Object> (i.e. key/value)
parameters for both datastore factories, functions, and processes.

The kicker here is that we want a more capable data structure to pass
values into a processes, similar to have feature model works with Attribute
values (each has a getUserData map).

May be worth

Looks like you didn't finish the thought here... are you suggesting that we
could use Attribute to relay this metadata?

*Q: Can we use Supplier to get a handle on some of the magic book keeping
GeoServer does?*

GeoServer tracks resources consumed during process execution using thread
locals, in addition to being complex, it is rather opaque to developers
writing processes.

We may indeed be able to, so a process defined as taking a supplier ...

public static Supplier<GridCoverage> floor( Supplier<GridCoverageReader2D>
dem, Polygon area, ProgressListener progress ) ...

Would give a process a chance to register any resources (say file) that
needs to be cleaned in the returned metadata map.

Aside: Although Supplier<T> seems to be more complicated then our quick
annotations allow for, may be better to keep it for hand created processes.

Sorry... this is all going over my head. It is hard to follow what appear
to be a brain dump as notes.

Q: We already have resource wrappers in GeoServer can we combine forces
with Supplier<T> ?

See:

* WPSResource
<https://github.com/geoserver/geoserver/blob/master/src/extension/wps/wps-core/src/main/java/org/geoserver/wps/resource/WPSResource.java&gt;
/ CoverageResource
<https://github.com/geoserver/geoserver/blob/master/src/extension/wps/wps-core/src/main/java/org/geoserver/wps/resource/GridCoverageResource.java&gt;

I see no reason why these could not implement Supplier<Covearge> allowing
GeoServer to smoothly hand the objects over to a process. Currently they
are all kept behind the scenes, used to stage data prior to calling the
process.

I see... I wasn't aware of these... looks like these are currently used to
stash created during a process execution that need to be cleaned up after
the process chain is complete. Are you suggesting we repurpose them as
input/output parameters.

--
Jody Garnett

On 1 September 2015 at 06:33, Justin Deoliveira <jdeolive@anonymised.com>
wrote:

Hey guys,

I took another look at this trying to implement what Jody suggested...
and I don't think it will quite work with a wrapper or a proxy. Reason
being that most raster processes actually take GridCoverage2D rather than
GridCoverage. I think trying to wrap that class will be a bit messy and
proxying a class would require a CGLIB dependency... all in all getting to
be a bit hairy. Staying along the same line though I think introducing a
new interface in GeoTools, something named CoverageSupplier (modelled after
guava/java8 supplier interface) would work. Basic idea would look like:

  interface CoverageSupplier extends Supplier<GridCoverage>;

With that it would be easy enough for GeoServer to plug in an
implementation that provides access to the catalog metadata when it is
around. Although I am not sure how to do this and make the process
GeoServer agnostic... another class or interface I guess to represent all
of that same metadata... maybe just a simple map structure would do:

  interface CoverageSupplier extends Supplier<GridCoverage> {
     Map<String,Object> metadata();
  }

This would give me what I need in a general way. If folks think that is a
good idea I can continue down that path (and btw I would want to add the
equivalent supplier interface for FeatureCollection). If not though I think
I may just fall back to my original approach which is to be able to declare
an input directly on CoverageInfo.

Thanks!

-Justin

On Wed, Aug 26, 2015 at 12:23 PM, Jody Garnett <jody.garnett@anonymised.com>
wrote:

Sorry if my example was not clear,

The answer is that a GridCoverage would be sent in, and you could do and
instance of check to access to a ResourceInfo if needed.

String name name( GridCoverage coverage ){
     if( coverage instanceof AccessInfo ){
        ResourceInfo info = ((AccessInfo)coverage).toInfo();
        return info.getName();
     }
     return "unnamed";
}

--
Jody Garnett

On 26 August 2015 at 09:52, Justin Deoliveira <jdeolive@anonymised.com>
wrote:

On Wed, Aug 26, 2015 at 10:28 AM, Jody Garnett <jody.garnett@anonymised.com>
wrote:

I would make a new interface that allows access to the ResourceInfo
....

interface AccessInfo {
   ResoruceInfo toInfo();
}

And then wrap the GridCoverage in a proxy that also supports a
AccesInfo:

class GridCoveageWrapper implements GridCoverage, AccessInfo {
   GridCoveage delegate;
   ResourceInfo info;
   ...
}

Or similar, if I was smarter I would make it as a DynamicProxy so I
could clean apply InfoWrapper around any parameter ... since you are a
smart guy I bet you could make that work :slight_smile:

Sorry but what does this buy me over the approach I thought of? The
CoverageInfo already provides access to the grid coverage... so this seems
like just adding a superfluous interface?

Jody

--
Jody Garnett

On 26 August 2015 at 08:51, Justin Deoliveira <jdeolive@anonymised.com>
wrote:

Hi folks,

I have a scenario where I want to write a GeoServer WPS process that
takes in raster input but then passes the raster off to another service to
do the actual processing. The other service takes the path to the raster as
it's input. So my problem is that it seems the established way to write a
raster process is to take a GridCoverage object as input, and from what I
can tell there is no easy way to get the original source of the coverage
(which makes sense). So right now to pull this off the grid coverage is
written out to a temporary geotiff file which is then passed off. I would
like to avoid this copy step and work on the original file directly if
possible.

So I guess my first question is if folks have done anything like this
and if so how they approached it. If not I was thinking of the following
solution:

Rather than declare my input of type GridCoverage, declare it of type
CoverageInfo. From there I can get to the store and then to the file
backing it. I am also aware that a grid coverage could potentially come
from a non-file source. For these cases I still plan to keep the "copy to
geotiff" in place as a fallback.

Now I could just have my process take a string as input, interpret
that as a layer name, and then look it up in the catalog. But I was
thinking adding a bit of infrastructure for it (like what's there for
accepting GridCoverage and FeatureCollection) could be of general use. Also
with that approach I lose some niceties like a dropdown of layer names in
the request builder.

I sketched out an implementation of this and the approach I took was
to encode the input as a reference along the same lines as internal WFS/WCS
calls are made. In XML it looks like this:

  <wps:Input>
     <ows:Identifier>input</ows:Identifier>
     <wps:Reference xlink:href="http://geoserver/catalog/topp:states
"/>
  </wps:Input>

With that it required a PPIO and an InputProvider and then it worked
like a charm.

Looking for feedback for this one. If folks think this is worth while
I will happily prep a pull request. Although I know it's close to feature
freeze time which is fine. Happy to wait.

Thanks!

-Justin

------------------------------------------------------------------------------

_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Sorry about the bulk notes, figure it was better share what I had rather than keep you waiting.

Comments inline:

···

Want to avoid defining the same process twice - once to work with a GridCoverage (the result of a processing chain) and a second time with CoverageInfo.

If you wanted to make a ConverageInfo wrapper around a GidCoverage (result of a processing chain) you can do so. It is not a perfect fit, but it would be a GeoServer specific alternative to introducing a CoverageSupplier.

We can let this alternative drop.

I considered if it was suitable (it is a value wrapper with a user data map).

The missing sentence - suggests using getUserData() (rather than getMetadata()) for consistency.

Considering two alternatives:

  • if you wanted to use Supplier in a method signature - and define the process using reflection.
  • if you want to save the Supplier technique for processes that are defined as a distinct class without reflection

Yes.

As I understand it WPSResource / CoverageResource has the job of producing an input/output value currently. The engine uses these to stage the data before calling the process.

Letting these WPSResource implementations act directly as a CoverageSupplier (and be passed into a process that cares) seems better than introducing another set of indirection/wrappers.

As a bonus we may end up with something more clear than the use of ThreadLocals to smuggle data into and out of a process.


Jody Garnett

Same process twice?

So this is a quick/clean alternative if you want it. Not great due to the number of CoverageInfo methods.

Sorry… not following you here.

Looks like you didn’t finish the thought here… are you suggesting that we could use Attribute to relay this metadata?

Sorry… this is all going over my head. It is hard to follow what appear to be a brain dump as notes.

I see no reason why these could not implement Supplier allowing GeoServer to smoothly hand the objects over to a process. Currently they are all kept behind the scenes, used to stage data prior to calling the process.

I see… I wasn’t aware of these… looks like these are currently used to stash created during a process execution that need to be cleaned up after the process chain is complete. Are you suggesting we repurpose them as input/output parameters.

Hi Justin,
sorry for sending you down the rabbit hole for nothing :slight_smile:

I’m happy with both approaches you suggested

Cheers
Andrea

···

On Tue, Sep 1, 2015 at 3:33 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hey guys,

I took another look at this trying to implement what Jody suggested… and I don’t think it will quite work with a wrapper or a proxy. Reason being that most raster processes actually take GridCoverage2D rather than GridCoverage. I think trying to wrap that class will be a bit messy and proxying a class would require a CGLIB dependency… all in all getting to be a bit hairy. Staying along the same line though I think introducing a new interface in GeoTools, something named CoverageSupplier (modelled after guava/java8 supplier interface) would work. Basic idea would look like:

interface CoverageSupplier extends Supplier;

With that it would be easy enough for GeoServer to plug in an implementation that provides access to the catalog metadata when it is around. Although I am not sure how to do this and make the process GeoServer agnostic… another class or interface I guess to represent all of that same metadata… maybe just a simple map structure would do:

interface CoverageSupplier extends Supplier {
Map<String,Object> metadata();
}

This would give me what I need in a general way. If folks think that is a good idea I can continue down that path (and btw I would want to add the equivalent supplier interface for FeatureCollection). If not though I think I may just fall back to my original approach which is to be able to declare an input directly on CoverageInfo.

Thanks!

-Justin



Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

On Wed, Aug 26, 2015 at 12:23 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Sorry if my example was not clear,

The answer is that a GridCoverage would be sent in, and you could do and instance of check to access to a ResourceInfo if needed.

String name name( GridCoverage coverage ){
if( coverage instanceof AccessInfo ){
ResourceInfo info = ((AccessInfo)coverage).toInfo();
return info.getName();
}
return “unnamed”;
}


Jody Garnett

On 26 August 2015 at 09:52, Justin Deoliveira <jdeolive@anonymised.com> wrote:

On Wed, Aug 26, 2015 at 10:28 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

I would make a new interface that allows access to the ResourceInfo …

interface AccessInfo {
ResoruceInfo toInfo();
}

And then wrap the GridCoverage in a proxy that also supports a AccesInfo:

class GridCoveageWrapper implements GridCoverage, AccessInfo {
GridCoveage delegate;
ResourceInfo info;

}

Or similar, if I was smarter I would make it as a DynamicProxy so I could clean apply InfoWrapper around any parameter … since you are a smart guy I bet you could make that work :slight_smile:

Sorry but what does this buy me over the approach I thought of? The CoverageInfo already provides access to the grid coverage… so this seems like just adding a superfluous interface?

Jody


Jody Garnett

On 26 August 2015 at 08:51, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I have a scenario where I want to write a GeoServer WPS process that takes in raster input but then passes the raster off to another service to do the actual processing. The other service takes the path to the raster as it’s input. So my problem is that it seems the established way to write a raster process is to take a GridCoverage object as input, and from what I can tell there is no easy way to get the original source of the coverage (which makes sense). So right now to pull this off the grid coverage is written out to a temporary geotiff file which is then passed off. I would like to avoid this copy step and work on the original file directly if possible.

So I guess my first question is if folks have done anything like this and if so how they approached it. If not I was thinking of the following solution:

Rather than declare my input of type GridCoverage, declare it of type CoverageInfo. From there I can get to the store and then to the file backing it. I am also aware that a grid coverage could potentially come from a non-file source. For these cases I still plan to keep the “copy to geotiff” in place as a fallback.

Now I could just have my process take a string as input, interpret that as a layer name, and then look it up in the catalog. But I was thinking adding a bit of infrastructure for it (like what’s there for accepting GridCoverage and FeatureCollection) could be of general use. Also with that approach I lose some niceties like a dropdown of layer names in the request builder.

I sketched out an implementation of this and the approach I took was to encode the input as a reference along the same lines as internal WFS/WCS calls are made. In XML it looks like this:

wps:Input
ows:Identifierinput</ows:Identifier>
<wps:Reference xlink:href=“http://geoserver/catalog/topp:states”/>
</wps:Input>

With that it required a PPIO and an InputProvider and then it worked like a charm.

Looking for feedback for this one. If folks think this is worth while I will happily prep a pull request. Although I know it’s close to feature freeze time which is fine. Happy to wait.

Thanks!

-Justin



Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


No worries, I really just wanted to explore a few alternatives and gather up some feedback so no worries. Since this is just an optimization it’s not on the critical path at the moment. When it becomes more of an issue I’ll try to sum up what we found here and restart the conversation. Thanks for the feedback guys.

···

On Fri, Sep 4, 2015 at 9:38 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi Justin,
sorry for sending you down the rabbit hole for nothing :slight_smile:

I’m happy with both approaches you suggested

Cheers

Andrea

On Tue, Sep 1, 2015 at 3:33 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hey guys,

I took another look at this trying to implement what Jody suggested… and I don’t think it will quite work with a wrapper or a proxy. Reason being that most raster processes actually take GridCoverage2D rather than GridCoverage. I think trying to wrap that class will be a bit messy and proxying a class would require a CGLIB dependency… all in all getting to be a bit hairy. Staying along the same line though I think introducing a new interface in GeoTools, something named CoverageSupplier (modelled after guava/java8 supplier interface) would work. Basic idea would look like:

interface CoverageSupplier extends Supplier;

With that it would be easy enough for GeoServer to plug in an implementation that provides access to the catalog metadata when it is around. Although I am not sure how to do this and make the process GeoServer agnostic… another class or interface I guess to represent all of that same metadata… maybe just a simple map structure would do:

interface CoverageSupplier extends Supplier {
Map<String,Object> metadata();
}

This would give me what I need in a general way. If folks think that is a good idea I can continue down that path (and btw I would want to add the equivalent supplier interface for FeatureCollection). If not though I think I may just fall back to my original approach which is to be able to declare an input directly on CoverageInfo.

Thanks!

-Justin



Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Wed, Aug 26, 2015 at 12:23 PM, Jody Garnett <jody.garnett@…403…> wrote:

Sorry if my example was not clear,

The answer is that a GridCoverage would be sent in, and you could do and instance of check to access to a ResourceInfo if needed.

String name name( GridCoverage coverage ){
if( coverage instanceof AccessInfo ){
ResourceInfo info = ((AccessInfo)coverage).toInfo();
return info.getName();
}
return “unnamed”;
}


Jody Garnett

On 26 August 2015 at 09:52, Justin Deoliveira <jdeolive@anonymised.com> wrote:

On Wed, Aug 26, 2015 at 10:28 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

I would make a new interface that allows access to the ResourceInfo …

interface AccessInfo {
ResoruceInfo toInfo();
}

And then wrap the GridCoverage in a proxy that also supports a AccesInfo:

class GridCoveageWrapper implements GridCoverage, AccessInfo {
GridCoveage delegate;
ResourceInfo info;

}

Or similar, if I was smarter I would make it as a DynamicProxy so I could clean apply InfoWrapper around any parameter … since you are a smart guy I bet you could make that work :slight_smile:

Sorry but what does this buy me over the approach I thought of? The CoverageInfo already provides access to the grid coverage… so this seems like just adding a superfluous interface?

Jody


Jody Garnett

On 26 August 2015 at 08:51, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I have a scenario where I want to write a GeoServer WPS process that takes in raster input but then passes the raster off to another service to do the actual processing. The other service takes the path to the raster as it’s input. So my problem is that it seems the established way to write a raster process is to take a GridCoverage object as input, and from what I can tell there is no easy way to get the original source of the coverage (which makes sense). So right now to pull this off the grid coverage is written out to a temporary geotiff file which is then passed off. I would like to avoid this copy step and work on the original file directly if possible.

So I guess my first question is if folks have done anything like this and if so how they approached it. If not I was thinking of the following solution:

Rather than declare my input of type GridCoverage, declare it of type CoverageInfo. From there I can get to the store and then to the file backing it. I am also aware that a grid coverage could potentially come from a non-file source. For these cases I still plan to keep the “copy to geotiff” in place as a fallback.

Now I could just have my process take a string as input, interpret that as a layer name, and then look it up in the catalog. But I was thinking adding a bit of infrastructure for it (like what’s there for accepting GridCoverage and FeatureCollection) could be of general use. Also with that approach I lose some niceties like a dropdown of layer names in the request builder.

I sketched out an implementation of this and the approach I took was to encode the input as a reference along the same lines as internal WFS/WCS calls are made. In XML it looks like this:

wps:Input
ows:Identifierinput</ows:Identifier>
<wps:Reference xlink:href=“http://geoserver/catalog/topp:states”/>
</wps:Input>

With that it required a PPIO and an InputProvider and then it worked like a charm.

Looking for feedback for this one. If folks think this is worth while I will happily prep a pull request. Although I know it’s close to feature freeze time which is fine. Happy to wait.

Thanks!

-Justin



Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel