[Geoserver-devel] Java bindings for GDAL (was [Gvsig_desarrolladores] JMRSID)

GeoServer GeoServer Developer
,
1

Hi guys,
I'm writing this in english since I guess It may be of interest for other
projects, so I'm cc'ing this message to the appropriate lists.

First, I want to adhere to Alvaro's congratulations for the great job you're
doing with gvSig, and raise up some questions.

FYI: the gvSig team has managed to include Java bindings for GDAL inside its
product.

As it become obvious, java access to gdal may be of great interest for the
GeoServer WMS/WCS implementations. My questions are:
- do you know if access to gdal is thread safe? I guess I have read from the
gdal creator that it is not still thread safe. Do you know if it would be
suitable for a server usage?
- do you have plans (if not already done) to commit and maintain the Java
bindings for GDAL at the GDAL source tree? I ask because it seems like a long
waited contribution on the GDAL site and forums.

Geoserver WCS guys: are you interested in accessing rasters through GDAL? if
so, are you interested in coordinating with gvSig team to inspect the
possibilities in the short term? or may be you are focused on higher level
stuff right now?

I guess it could be a huge capability for geoserver. If you're not interested
may be I could arrange some 'spare' time to dedicate, but it would be really
fantastic if you're interesting at least in investigating the possibilities
of integration.

Alvaro: are you working on an OGC map server or a custom one? if the former,
would you be interested in adding gdal support to GeoServer WMS?

Best regards and thanks to all of you for the great stuff you're working on.

Gabriel.

On Tuesday 27 September 2005 09:15, Alvaro Zabala wrote:

Hola a todos,

En primer lugar quería felicitar al equipo de gvSIG por su gran labor.
En mi opinión, gvSIG es mucho más que un herramienta final de usuario,
y gracias a su diseño modular ha puesto a disposición del resto de la
comunidad GIS muchas funcionalidades hasta ahora no cubiertas, y
perfectamente reutilizables con relativo poco esfuerzo.

En el caso que nos ocupa, en mi organización estamos implantando un
servidor de mapas basado en sw libre, y para no tener que convertir
toda la imaginería raster (en formato Mr.SID, unos 12 DVDs de doble
cara y capa) a ECW hemos incluido soporte para Mr.SID gracias a la
librería de gvSIG jmrsid.

Ahora vienen mis dudas. Las primeras pruebas funcionales las pasa sin
problemas, pero al estar basado en una DLL-gdal-, querría saber si se
han detectado problemas en el manejo de la librería. Por poner un
ejemplo, la librería ECW de ER-Mapper en este sentido es una
calamidad. La DLL debe mantener una caché de tiles, y desde el API
Java no hay manera de liberarla. Esto obliga a tener que implementar
truquitos, como cerrar el fichero cada X imágenes procesadas, o
controlar externamente la resolución de las imágenes que se quieren
obtener de la librería, pues esta no permite remuestrear.

¿Es jmrsid-gdal- estable en este sentido? ¿Se arreglaron los problemas
que daba gdal en el manejo de MrSID a principios del 2005?

Un saludo a todos, y gracias de antemano.

--
Alvaro Zabala Ordóñez
Miembro fundador de AGIL
Asociación para la promoción del GIS Libre
www.agiles.org

Funcionario del Cuerpo de Gestión de Sistemas e Informática de la
Administración General del Estado.
Confederación Hidrográfica del Guadalquivir.
Teléfono: 954939523
Plaza de España, sector II.
SEVILLA

_______________________________________________
gvSIG_desarrolladores mailing list
gvSIG_desarrolladores@anonymised.com
http://runas.cap.gva.es/mailman/listinfo/gvsig_desarrolladores

2

Hi Gabriel!
Its welcome to have news from you,

Alvaro: are you working on an OGC map server or a custom one? if the former,
would you be interested in adding gdal support to GeoServer WMS?

Usually in a project learning curve, stability, usability and fast
time to get results are more important than standards and design.
Althought we think that Geotools -and Geotools based like Udig and
Geoserver- are the most serious projects in open source java based GIS
-with gvSIG permission, for me, gvSIG is as good or even better than
uDIG-, we found difficult to start with it.

Geotools is a very good designed project, but very unstable-always
changing-and it prefer good desing over good application working. For
that reason, we are working with a custom map server -by now we neednt
OGC XML WMS communications, only an intranet web client-so our code
would not map good with geotools/geoserver APIs. I dont have the time
necessary to study Geoserver APIs and to migrate our code to it, but I
would help to do that work by answering mails.

Best regards, and sorry for not to help more.

--
Alvaro Zabala Ordóñez
Miembro fundador de AGIL
Asociación para la promoción del GIS Libre
www.agiles.org

Funcionario del Cuerpo de Gestión de Sistemas e Informática de la
Administración General del Estado.
Confederación Hidrográfica del Guadalquivir.
Teléfono: 954939523
Plaza de España, sector II.
SEVILLA

3

- do you know if access to gdal is thread safe? I guess I have read from the
gdal creator that it is not still thread safe. Do you know if it would be
suitable for a server usage?

After some probes, Im pretty sure that gdal isnt thread safe.

I had to synchronize my dao class' access method (which calls
jmrsidnative class methods) to avoid problems with concurrent request
of a same image.

Best regards.

--
Alvaro Zabala Ordóñez
Miembro fundador de AGIL
Asociación para la promoción del GIS Libre
www.agiles.org

Funcionario del Cuerpo de Gestión de Sistemas e Informática de la
Administración General del Estado.
Confederación Hidrográfica del Guadalquivir.
Teléfono: 954939523
Plaza de España, sector II.
SEVILLA

4

Alvaro Zabala wrote:

- do you know if access to gdal is thread safe? I guess I have read from the
gdal creator that it is not still thread safe. Do you know if it would be
suitable for a server usage?
   
After some probes, Im pretty sure that gdal isnt thread safe.

I had to synchronize my dao class' access method (which calls
jmrsidnative class methods) to avoid problems with concurrent request
of a same image.

Best regards.

--
Alvaro Zabala Ordóñez
Miembro fundador de AGIL
Asociación para la promoción del GIS Libre
www.agiles.org

Funcionario del Cuerpo de Gestión de Sistemas e Informática de la
Administración General del Estado.
Confederación Hidrográfica del Guadalquivir.
Teléfono: 954939523
Plaza de España, sector II.
SEVILLA

_______________________________________________
gvSIG_usuarios mailing list
gvSIG_usuarios@anonymised.com
http://runas.cap.gva.es/mailman/listinfo/gvsig_usuarios

--
  Luis W. Sevilla
  Equipo de desarrollo gvSIG
  Conselleria de Infraestructuras y Transporte
  Generalitat Valenciana
  Valencia - Spain
  http://www.gvsig.gva.es

5

Alvaro Zabala wrote:

- do you know if access to gdal is thread safe? I guess I have read from the
gdal creator that it is not still thread safe. Do you know if it would be
suitable for a server usage?
   
After some probes, Im pretty sure that gdal isnt thread safe.

I had to synchronize my dao class' access method (which calls
jmrsidnative class methods) to avoid problems with concurrent request
of a same image.

There are anythig we can do in a wrapper development to do a C/C++ underliying library threadsafe. If
Frank saids gdal isn't thread safe, so jgdal can´t be too.

Besides this gdal it's at the bottom of mapserver, so it may be with GeoServer, isn't it?

Greetings

(sorry for cross-posting, and for empty-replying)

Best regards.

--
Alvaro Zabala Ordóñez
Miembro fundador de AGIL
Asociación para la promoción del GIS Libre
www.agiles.org

Funcionario del Cuerpo de Gestión de Sistemas e Informática de la
Administración General del Estado.
Confederación Hidrográfica del Guadalquivir.
Teléfono: 954939523
Plaza de España, sector II.
SEVILLA

_______________________________________________
gvSIG_usuarios mailing list
gvSIG_usuarios@anonymised.com
http://runas.cap.gva.es/mailman/listinfo/gvsig_usuarios

--
  Luis W. Sevilla
  Equipo de desarrollo gvSIG
  Conselleria de Infraestructuras y Transporte
  Generalitat Valenciana
  Valencia - Spain
  http://www.gvsig.gva.es

6

On 9/27/05, Gabriel Roldán <groldan@anonymised.com> wrote:

FYI: the gvSig team has managed to include Java bindings for GDAL inside its
product.

As it become obvious, java access to gdal may be of great interest for the
GeoServer WMS/WCS implementations. My questions are:
- do you know if access to gdal is thread safe? I guess I have read from the
gdal creator that it is not still thread safe. Do you know if it would be
suitable for a server usage?

Gabriel,

There has been significant work in recent months to make GDAL thread-safe
under fairly specific circumstances. These limitations include that read-only
is thread safe, but writing is not, and that any given dataset be accessed only
from one thread at a time.

- do you have plans (if not already done) to commit and maintain the Java
bindings for GDAL at the GDAL source tree? I ask because it seems like a long
waited contribution on the GDAL site and forums.

The gvSig team has offered their Java bindings for GDAL back to me, but
my understanding is that they were built "ad hoc". Kevin Ruland and Howard
Butler have been working on "next generation swig bindings" for GDAL and
my hope is that someone will be able to pick up the Java aspect of this work
to produce Java bindings that are closely aligned with other NG SWIG
bindings. These would then be the "official" supported GDAL bindings.

Of course, from my point of view anyone is welcome to use the gvSig
GDAL bindings but they just won't be "officially supported".

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@anonymised.com..
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent

7

ok,
I'm under the impression that GDAL could still be good for WMS.

On Tuesday 27 September 2005 15:57, Frank Warmerdam wrote:

Butler have been working on "next generation swig bindings" for GDAL and
my hope is that someone will be able to pick up the Java aspect of this
work to produce Java bindings that are closely aligned with other NG SWIG
bindings. These would then be the "official" supported GDAL bindings.

Oh dear, you can't imagine since how many time ago I'm willing to find the
time to look at the C# swig bindings and try to adapt it to java. Hopefully
I'll find the time or someone else do it in the while :slight_smile:

thanks to all for your replies.

sincerelly,

Gabriel.

8

Gabriel Roldán wrote:

Hi guys,
I'm writing this in english since I guess It may be of interest for other projects, so I'm cc'ing this message to the appropriate lists.

First, I want to adhere to Alvaro's congratulations for the great job you're doing with gvSig, and raise up some questions.

Thanks, it's very apreciated and motivating your support.

FYI: the gvSig team has managed to include Java bindings for GDAL inside its product.

As it become obvious, java access to gdal may be of great interest for the GeoServer WMS/WCS implementations. My questions are:
- do you know if access to gdal is thread safe? I guess I have read from the gdal creator that it is not still thread safe. Do you know if it would be suitable for a server usage?

Well, Frank has answered by itself. I think server usage must be evaluated.

- do you have plans (if not already done) to commit and maintain the Java bindings for GDAL at the GDAL source tree? I ask because it seems like a long waited contribution on the GDAL site and forums.

Well, our point for jgdal development was usability from java, and fidelity to object model that python wrapper
seems to suggest.
When we aproximate java wrapper we saw than some of the more critical work must be hand made, and SWING
needed his own learning curve. So we decided to have a more 'hand made' approach, that Frank Warmerdam didn't
liked and he doesn't seem to agree with us for doing official support.

Geoserver WCS guys: are you interested in accessing rasters through GDAL? if so, are you interested in coordinating with gvSig team to inspect the possibilities in the short term? or may be you are focused on higher level stuff right now?

I guess it could be a huge capability for geoserver. If you're not interested may be I could arrange some 'spare' time to dedicate, but it would be really fantastic if you're interesting at least in investigating the possibilities of integration.

We can work together on this. I offered some collaboration to simone on this aspect, and I'm waiting from some
answer about the topic.

Alvaro: are you working on an OGC map server or a custom one? if the former, would you be interested in adding gdal support to GeoServer WMS?

shoudn't be too dificult to add. Even add full jcms underlying support may be a good idea.

Best regards and thanks to all of you for the great stuff you're working on.

Same to you

Gabriel.

On Tuesday 27 September 2005 09:15, Alvaro Zabala wrote:

Hola a todos,

En primer lugar quería felicitar al equipo de gvSIG por su gran labor.
En mi opinión, gvSIG es mucho más que un herramienta final de usuario,
y gracias a su diseño modular ha puesto a disposición del resto de la
comunidad GIS muchas funcionalidades hasta ahora no cubiertas, y
perfectamente reutilizables con relativo poco esfuerzo.

En el caso que nos ocupa, en mi organización estamos implantando un
servidor de mapas basado en sw libre, y para no tener que convertir
toda la imaginería raster (en formato Mr.SID, unos 12 DVDs de doble
cara y capa) a ECW hemos incluido soporte para Mr.SID gracias a la
librería de gvSIG jmrsid.

Ahora vienen mis dudas. Las primeras pruebas funcionales las pasa sin
problemas, pero al estar basado en una DLL-gdal-, querría saber si se
han detectado problemas en el manejo de la librería. Por poner un
ejemplo, la librería ECW de ER-Mapper en este sentido es una
calamidad. La DLL debe mantener una caché de tiles, y desde el API
Java no hay manera de liberarla. Esto obliga a tener que implementar
truquitos, como cerrar el fichero cada X imágenes procesadas, o
controlar externamente la resolución de las imágenes que se quieren
obtener de la librería, pues esta no permite remuestrear.

¿Es jmrsid-gdal- estable en este sentido? ¿Se arreglaron los problemas
que daba gdal en el manejo de MrSID a principios del 2005?

Un saludo a todos, y gracias de antemano.

--
Alvaro Zabala Ordóñez
Miembro fundador de AGIL
Asociación para la promoción del GIS Libre
www.agiles.org

Funcionario del Cuerpo de Gestión de Sistemas e Informática de la
Administración General del Estado.
Confederación Hidrográfica del Guadalquivir.
Teléfono: 954939523
Plaza de España, sector II.
SEVILLA

_______________________________________________
gvSIG_desarrolladores mailing list
gvSIG_desarrolladores@anonymised.com
http://runas.cap.gva.es/mailman/listinfo/gvsig_desarrolladores
   

_______________________________________________
OpenSDI mailing list
OpenSDI@anonymised.com
http://lists.eogeo.org/mailman/listinfo/opensdi

--
  Luis W. Sevilla
  Equipo de desarrollo gvSIG
  Conselleria de Infraestructuras y Transporte
  Generalitat Valenciana
  Valencia - Spain
  http://www.gvsig.gva.es