[Geoserver-devel] [GeoShaver] Re: Plans for remainder of GeoShaver project

Ciao Jon,
sending this email just to throw some information in the mix about
what we have been doing on the netCDF side of the world. I am going to
split things in levels, from imageio to geoserver

<IMAGEIO>
- Daniel, Alessio and partly me have been working at the lowest
imageio level trying to implement imagio readers with no geospatial
notion for
      -- netcdf-cf (pure java)
      -- grib1 (soon grib2 as well) pure java
      -- various hdf profiles, using the NCSA java bindings, which we
have rewritten from scratch since we were having all sort of failures

Note that for the moment we refer to these plugins as the "flat" plugins.

This work has been recently (after a long discussion with a client :slight_smile:
) moved to the spike directory of the imageio-ext project here:
https://imageio-ext.dev.java.net/source/browse/imageio-ext/spike/nd-work/
The project is open source and there is a decent extent of
documentation. We have plans to work on these plugins until the end of
next summer adding other formats as well like more HDF profiles,
HDF-EOS and, depending on some NDAs I have signed JPEG2000.

Note that all this is independent from the gdal-based plugins.

- We have tried to cooperate with the GeoMatys guys on the definition
of a set of spatiotemporal metadata in order to write geospatial
enabled Imageio plugins.
This work is captured in this google document
ttp://docs.google.com/Doc?id=dgcvg3gc_34cpxtgkkv&hl=it

<GeoTools>
This is where we are mostly working right now:

- Jody Garnett together Daniele (and some limited feedback from
myself) is working on implementing a replacement for the old coverarge
I/O API, namely the GridCoverageEchange API, which have been
deprecated since ages. We might want to put together a proposal to the
GeoApi project afterwards in order to replace the old GCE.
This work is going on under the geotools spike directory in the svn
(geotools trunk branch)

-Daniele and Alessio are working on implementing what we call "smart"
Imageio readers for netcdf-cf, hdf-avhrr and grib1. A smart reader is
simply a wrapper for a flat reader which add geospatial metadata to
it. We are still talking about 2D raster data but with geospatial
metadata describing detp/elevation, CRS, bbox, time, etc... This in
the future should reside inside GeoTools since it would depend on
other Geotools core modules
This work is actually on our private svn, but we could make it available.

- Jody Garnett and Daniele are working on building new geotools
plugins which would leverage on the above mentioned smart readers in
order to build coverages out of 2D rasters. For the first deliverable
we should just produce 2D coverage but in the future we are looking
into going a step further (iso 19123 or something similar I guess)
This work is actually on our private svn, but we could make it available.

<GeoServer>
- Alessio has already implemented and committed the EMF bindings to
allow GeoServer 1.7.x to parse
BAND,TIME,DEPTH/ELEVATION params in the GeoServer. The work in under
revision and should be committed soon.
Note that for the TIME parsing we reused the temporal work done byt GeoMatys.

- Alessio is working his way through refactoring the UI and
configuration of GeoServer using stubs of the new CoverageIo
interfaces which jody is defining in order to make possible to serve
2D coverages out of multidimensional ones for netcdf-cf, grib1 and
hdf-avhrr.
We are targeting mid sept to have a working demo for WMS 1.1.1 and WCS
1.0. Afterwards we will work on WMS 1.3 and WCS 1.1. The end goal is
to have the GeoServer able to do animations from WMS and return
multidim coverages (netcdf-cf as an instance) from WCS, but for this
are talking about next year much probably.

Ciao,
Simone.

On Wed, Aug 13, 2008 at 12:48 PM, Jon Blower <jdb@anonymised.com> wrote:

Hi Martin,

Would it help you if we worked together on some of these problems to
try to produce a NetCDF reader that meets all of our needs? Also, as
a separate question, do you plan on working on the configuration of
GeoServer itself (to register NetCDF datasets in the catalog) or is
all your work at the GeoTools level?

Cheers, Jon

On Wed, Jul 16, 2008 at 2:24 PM, Martin Desruisseaux
<martin.desruisseaux@anonymised.com> wrote:

Hello all

This is true that the current NetCDF image reader has limitation regarding its
generality. We are aware of them and are slowly working toward some fixes. The
current work (which I believe is a necessary step before we can improve NetCDF-
like readers) is to get a good structure of GeographicMetadata for coverage,
using "GML in JPEG 2000" as an inspiration. Cedric worked on that recently (one
or two week ago) together with Daniele from Geosolution, but we are not yet done.

       Martin

>

--
--------------------------------------------------------------
Dr Jon Blower Tel: +44 118 378 5213 (direct line)
Technical Director Tel: +44 118 378 8741 (ESSC)
Reading e-Science Centre Fax: +44 118 378 6413
ESSC Email: j.d.blower@anonymised.com
University of Reading
3 Earley Gate
Reading RG6 6AL, UK
--------------------------------------------------------------

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "GeoShaver" group.
To post to this group, send email to geoshaver@anonymised.com
To unsubscribe from this group, send email to geoshaver+unsubscribe@anonymised.com
For more options, visit this group at http://groups.google.co.uk/group/geoshaver?hl=en-GB
-~----------~----~----~----~------~----~------~--~---

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions S.A.S.
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it

-------------------------------------------------------

Ciao Simone,

Thanks very much for this. Can I check that I've understood a few
things correctly?

* Do I understand correctly that you have three potential ways to read NetCDF:
- Flat plugins, no geospatial metadata (is this what GeoServer uses
right now?)
- "Smart" plugins, which wrap flat plugins, providing geospatial metadata
- "Geospatial-enabled" plugins, which requires the definition of
some imageio metadata
What is the practical difference between the "smart" and
"geospatial-enabled" plugins?

* Are your "flat" plugins for NetCDF different from the NetCDF image
readers implemented by Geomatys? Do you use the Unidata Java NetCDF
libraries, which can read NetCDF, GRIB1/2 and HDF identically?

Is there any particular area in which we can help (perhaps extending
some of these things to nD, or perhaps testing that this new stuff
works with our data)?

Cheers, Jon

On Thu, Aug 14, 2008 at 11:54 AM, Simone Giannecchini
<simboss1@anonymised.com> wrote:

Ciao Jon,
sending this email just to throw some information in the mix about
what we have been doing on the netCDF side of the world. I am going to
split things in levels, from imageio to geoserver

<IMAGEIO>
- Daniel, Alessio and partly me have been working at the lowest
imageio level trying to implement imagio readers with no geospatial
notion for
     -- netcdf-cf (pure java)
     -- grib1 (soon grib2 as well) pure java
     -- various hdf profiles, using the NCSA java bindings, which we
have rewritten from scratch since we were having all sort of failures

Note that for the moment we refer to these plugins as the "flat" plugins.

This work has been recently (after a long discussion with a client :slight_smile:
) moved to the spike directory of the imageio-ext project here:
https://imageio-ext.dev.java.net/source/browse/imageio-ext/spike/nd-work/
The project is open source and there is a decent extent of
documentation. We have plans to work on these plugins until the end of
next summer adding other formats as well like more HDF profiles,
HDF-EOS and, depending on some NDAs I have signed JPEG2000.

Note that all this is independent from the gdal-based plugins.

- We have tried to cooperate with the GeoMatys guys on the definition
of a set of spatiotemporal metadata in order to write geospatial
enabled Imageio plugins.
This work is captured in this google document
ttp://docs.google.com/Doc?id=dgcvg3gc_34cpxtgkkv&hl=it

<GeoTools>
This is where we are mostly working right now:

- Jody Garnett together Daniele (and some limited feedback from
myself) is working on implementing a replacement for the old coverarge
I/O API, namely the GridCoverageEchange API, which have been
deprecated since ages. We might want to put together a proposal to the
GeoApi project afterwards in order to replace the old GCE.
This work is going on under the geotools spike directory in the svn
(geotools trunk branch)

-Daniele and Alessio are working on implementing what we call "smart"
Imageio readers for netcdf-cf, hdf-avhrr and grib1. A smart reader is
simply a wrapper for a flat reader which add geospatial metadata to
it. We are still talking about 2D raster data but with geospatial
metadata describing detp/elevation, CRS, bbox, time, etc... This in
the future should reside inside GeoTools since it would depend on
other Geotools core modules
This work is actually on our private svn, but we could make it available.

- Jody Garnett and Daniele are working on building new geotools
plugins which would leverage on the above mentioned smart readers in
order to build coverages out of 2D rasters. For the first deliverable
we should just produce 2D coverage but in the future we are looking
into going a step further (iso 19123 or something similar I guess)
This work is actually on our private svn, but we could make it available.

<GeoServer>
- Alessio has already implemented and committed the EMF bindings to
allow GeoServer 1.7.x to parse
BAND,TIME,DEPTH/ELEVATION params in the GeoServer. The work in under
revision and should be committed soon.
Note that for the TIME parsing we reused the temporal work done byt GeoMatys.

- Alessio is working his way through refactoring the UI and
configuration of GeoServer using stubs of the new CoverageIo
interfaces which jody is defining in order to make possible to serve
2D coverages out of multidimensional ones for netcdf-cf, grib1 and
hdf-avhrr.
We are targeting mid sept to have a working demo for WMS 1.1.1 and WCS
1.0. Afterwards we will work on WMS 1.3 and WCS 1.1. The end goal is
to have the GeoServer able to do animations from WMS and return
multidim coverages (netcdf-cf as an instance) from WCS, but for this
are talking about next year much probably.

Ciao,
Simone.

--
--------------------------------------------------------------
Dr Jon Blower Tel: +44 118 378 5213 (direct line)
Technical Director Tel: +44 118 378 8741 (ESSC)
Reading e-Science Centre Fax: +44 118 378 6413
ESSC Email: j.d.blower@anonymised.com
University of Reading
3 Earley Gate
Reading RG6 6AL, UK
--------------------------------------------------------------

--
--------------------------------------------------------------
Dr Jon Blower Tel: +44 118 378 5213 (direct line)
Technical Director Tel: +44 118 378 8741 (ESSC)
Reading e-Science Centre Fax: +44 118 378 6413
ESSC Email: j.d.blower@anonymised.com
University of Reading
3 Earley Gate
Reading RG6 6AL, UK
--------------------------------------------------------------

Ciao Jon,
sorry about this late answer, no excuses.
Please, read below...

On Fri, Aug 15, 2008 at 2:38 PM, Jon Blower <jdb@anonymised.com> wrote:

Ciao Simone,

Thanks very much for this. Can I check that I've understood a few
things correctly?

* Do I understand correctly that you have three potential ways to read NetCDF:
- Flat plugins, no geospatial metadata (is this what GeoServer uses
right now?)
- "Smart" plugins, which wrap flat plugins, providing geospatial metadata
- "Geospatial-enabled" plugins, which requires the definition of
some imageio metadata
What is the practical difference between the "smart" and
"geospatial-enabled" plugins?

* Are your "flat" plugins for NetCDF different from the NetCDF image
readers implemented by Geomatys? Do you use the Unidata Java NetCDF
libraries, which can read NetCDF, GRIB1/2 and HDF identically?

We are trying to separate concerns for the different capabilities
needed (quoting your words):

- flat plugins (better name needed!) focus on basic metadata and
raster space subsetting and decimation. No notion of geo* at this
level, reading 2D slices out of a netcdf-cf file (imageio-ext
spike/ndplugin directory)
- smart plugins (we are calling them spatiotemporal plugins now, smart
seemed bad to me), wrap flat plugins, providing geospatial metadata
"Geospatial-enabled" plugins, which requires the definition of some
imageio metadata. They should reside within geotools since they
describe each 2D slice in spatiotemporal temporal terms allowing to
regroup them later as (Grid)Coverages. (this is in our svn since a
work in progress, but as I said we can open that up for you).

A few clarifications:
- we using netcdf lib for netcdf only, We have our own lib for grib1
and we are using the JNI bindings for HDF.
- We won't change this at least for a while, netcdf 4.0 is not mature as a lib

Is there any particular area in which we can help (perhaps extending
some of these things to nD, or perhaps testing that this new stuff
works with our data)?

Intensive work is being carried out at three levels.
- defining new api for covera-io (see gt-trunk spike dir) [me,
daniele, aaime, jgarnett]
- refining a first version of the spatiotemporal readers [alessio,daniele]
- implementing geotools plugins on top of the spatiotemporal reader
and porting old geotiff [me, daniele, jgarnett]
- refactoring geoserver 1.7.x [alessio, me ]

It might be a bit difficult to catch-up, but if you are interested I
can open up for you the parts that are in our private svn.
You can check how/where things are and then decide whether or not to
join the effort.

Ciao,
Simone.

Cheers, Jon

On Thu, Aug 14, 2008 at 11:54 AM, Simone Giannecchini
<simboss1@anonymised.com> wrote:

Ciao Jon,
sending this email just to throw some information in the mix about
what we have been doing on the netCDF side of the world. I am going to
split things in levels, from imageio to geoserver

<IMAGEIO>
- Daniel, Alessio and partly me have been working at the lowest
imageio level trying to implement imagio readers with no geospatial
notion for
     -- netcdf-cf (pure java)
     -- grib1 (soon grib2 as well) pure java
     -- various hdf profiles, using the NCSA java bindings, which we
have rewritten from scratch since we were having all sort of failures

Note that for the moment we refer to these plugins as the "flat" plugins.

This work has been recently (after a long discussion with a client :slight_smile:
) moved to the spike directory of the imageio-ext project here:
https://imageio-ext.dev.java.net/source/browse/imageio-ext/spike/nd-work/
The project is open source and there is a decent extent of
documentation. We have plans to work on these plugins until the end of
next summer adding other formats as well like more HDF profiles,
HDF-EOS and, depending on some NDAs I have signed JPEG2000.

Note that all this is independent from the gdal-based plugins.

- We have tried to cooperate with the GeoMatys guys on the definition
of a set of spatiotemporal metadata in order to write geospatial
enabled Imageio plugins.
This work is captured in this google document
ttp://docs.google.com/Doc?id=dgcvg3gc_34cpxtgkkv&hl=it

<GeoTools>
This is where we are mostly working right now:

- Jody Garnett together Daniele (and some limited feedback from
myself) is working on implementing a replacement for the old coverarge
I/O API, namely the GridCoverageEchange API, which have been
deprecated since ages. We might want to put together a proposal to the
GeoApi project afterwards in order to replace the old GCE.
This work is going on under the geotools spike directory in the svn
(geotools trunk branch)

-Daniele and Alessio are working on implementing what we call "smart"
Imageio readers for netcdf-cf, hdf-avhrr and grib1. A smart reader is
simply a wrapper for a flat reader which add geospatial metadata to
it. We are still talking about 2D raster data but with geospatial
metadata describing detp/elevation, CRS, bbox, time, etc... This in
the future should reside inside GeoTools since it would depend on
other Geotools core modules
This work is actually on our private svn, but we could make it available.

- Jody Garnett and Daniele are working on building new geotools
plugins which would leverage on the above mentioned smart readers in
order to build coverages out of 2D rasters. For the first deliverable
we should just produce 2D coverage but in the future we are looking
into going a step further (iso 19123 or something similar I guess)
This work is actually on our private svn, but we could make it available.

<GeoServer>
- Alessio has already implemented and committed the EMF bindings to
allow GeoServer 1.7.x to parse
BAND,TIME,DEPTH/ELEVATION params in the GeoServer. The work in under
revision and should be committed soon.
Note that for the TIME parsing we reused the temporal work done byt GeoMatys.

- Alessio is working his way through refactoring the UI and
configuration of GeoServer using stubs of the new CoverageIo
interfaces which jody is defining in order to make possible to serve
2D coverages out of multidimensional ones for netcdf-cf, grib1 and
hdf-avhrr.
We are targeting mid sept to have a working demo for WMS 1.1.1 and WCS
1.0. Afterwards we will work on WMS 1.3 and WCS 1.1. The end goal is
to have the GeoServer able to do animations from WMS and return
multidim coverages (netcdf-cf as an instance) from WCS, but for this
are talking about next year much probably.

Ciao,
Simone.

--
--------------------------------------------------------------
Dr Jon Blower Tel: +44 118 378 5213 (direct line)
Technical Director Tel: +44 118 378 8741 (ESSC)
Reading e-Science Centre Fax: +44 118 378 6413
ESSC Email: j.d.blower@anonymised.com
University of Reading
3 Earley Gate
Reading RG6 6AL, UK
--------------------------------------------------------------

--
--------------------------------------------------------------
Dr Jon Blower Tel: +44 118 378 5213 (direct line)
Technical Director Tel: +44 118 378 8741 (ESSC)
Reading e-Science Centre Fax: +44 118 378 6413
ESSC Email: j.d.blower@anonymised.com
University of Reading
3 Earley Gate
Reading RG6 6AL, UK
--------------------------------------------------------------

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "GeoShaver" group.
To post to this group, send email to geoshaver@anonymised.com
To unsubscribe from this group, send email to geoshaver+unsubscribe@anonymised.com
For more options, visit this group at http://groups.google.co.uk/group/geoshaver?hl=en-GB
-~----------~----~----~----~------~----~------~--~---

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions S.A.S.
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it

-------------------------------------------------------