[Geoserver-users] Tomcat hangs

GeoServer GeoServer User
1

Hi,

this has probably nothing to do with Geoserver, so please just ignore this, if you haven’t got any knowledge about it. I will propably passed it over to tomcat mailing list anyway. Just trying my luck here.

TomCat 6.0.14, GeoServer (no matter what version), Centos 4.6, java sdk 1.6.0_04

Apache Tomcat/6.0.14 1.6.0_04-b12 Sun Microsystems Inc. Linux 2.6.9-67.0.1.ELsmp i386

JVM: Free memory: 1935.00 MB Total memory: 2032.31 MB Max memory: 3072.18 MB

I am running quite heavy raster data sets in it (in order to print them out with 300 dpi) and it from time to time hangs, usually few minutes, sometimes for ever. Usually there are enough RAM left and processor is almost idling. Sometimes processor is doing 100% and won’t get idling (garbage collection?) . I guess it’s about TomCat tuning and has nothing to do with Geoserver so forgive me this enquiry on this list.

But on the other hand, if you have any ideas what to do, please tell me. I can’t publish a web service which hangs already with one user.

  • mika -
2

Lehtonen, Mika ha scritto:

Hi,

this has probably nothing to do with Geoserver, so please just ignore this, if you haven't got any knowledge about it. I will propably passed it over to tomcat mailing list anyway. Just trying my luck here.

TomCat 6.0.14, GeoServer (no matter what version), Centos 4.6, java sdk 1.6.0_04
Apache Tomcat/6.0.14 1.6.0_04-b12 Sun Microsystems Inc. Linux 2.6.9-67.0.1.ELsmp i386

JVM: Free memory: 1935.00 MB Total memory: 2032.31 MB Max memory: 3072.18 MB

I am running quite heavy raster data sets in it (in order to print them out with 300 dpi) and it from time to time hangs, usually few minutes, sometimes for ever. Usually there are enough RAM left and processor is almost idling. Sometimes processor is doing 100% and won't get idling (garbage collection?) . I guess it's about TomCat tuning and has nothing to do with Geoserver so forgive me this enquiry on this list.

Hum, what kind of process architecture do you have on that PC?
If it's not at least dual core, the "jvm ergonomics" (http://java.sun.com/j2se/1.5.0/docs/guide/vm/gc-ergonomics.html) won't enable automatically and you may want to add the (-XX:+UseParallelGC)
to the jvm options.

Besides that, for sure serving a very large coverage is not going to
help performance. I wish I could point you to PDF as an alternative
for printing, but unfortunately this bug prevents PDF encoding
of coverages http://jira.codehaus.org/browse/GEOS-1461

Cheers
Andrea

3

The processor is Intel Xeon E5420 quad-core (Dell PE 1950 III)

- mika -

Andrea Aime kirjoitti:

Lehtonen, Mika ha scritto:

Hi,

this has probably nothing to do with Geoserver, so please just ignore this, if you haven't got any knowledge about it. I will propably passed it over to tomcat mailing list anyway. Just trying my luck here.

TomCat 6.0.14, GeoServer (no matter what version), Centos 4.6, java sdk 1.6.0_04
Apache Tomcat/6.0.14 1.6.0_04-b12 Sun Microsystems Inc. Linux 2.6.9-67.0.1.ELsmp i386

JVM: Free memory: 1935.00 MB Total memory: 2032.31 MB Max memory: 3072.18 MB

I am running quite heavy raster data sets in it (in order to print them out with 300 dpi) and it from time to time hangs, usually few minutes, sometimes for ever. Usually there are enough RAM left and processor is almost idling. Sometimes processor is doing 100% and won't get idling (garbage collection?) . I guess it's about TomCat tuning and has nothing to do with Geoserver so forgive me this enquiry on this list.

Hum, what kind of process architecture do you have on that PC?
If it's not at least dual core, the "jvm ergonomics" (http://java.sun.com/j2se/1.5.0/docs/guide/vm/gc-ergonomics.html) won't enable automatically and you may want to add the (-XX:+UseParallelGC)
to the jvm options.

Besides that, for sure serving a very large coverage is not going to
help performance. I wish I could point you to PDF as an alternative
for printing, but unfortunately this bug prevents PDF encoding
of coverages http://jira.codehaus.org/browse/GEOS-1461

Cheers
Andrea

--
Sähköpostiosoitteeni on vaihtunut. Uusi osoite on mika@anonymised.com
Vanhaan osoitteeseen tulevat postit kääntyvät uuteen 25.4. asti.

My e-mail address has been changed. New address is mika@anonymised.com
Mails arriving in the old address will be forwarded to the new one till 25th April.

Mika Lehtonen
XML-Scanning Littoinen Ky
Lankakatu 2 E 13, 20660 Littoinen
mika@anonymised.com
mbl +358 (0)44 2908259

4

Lehtonen, Mika ha scritto:

The processor is Intel Xeon E5420 quad-core (Dell PE 1950 III)

Hmm, with such configuration the jvm should auto configure
for best performance. Are the original coverages optimized
for good performance? That is, are the inner tiled geotiffs
with overviews?

Some hints on how to create one is provided here:
http://docs.codehaus.org/display/GEOSDOC/High+performance+coverage+serving

Cheers
Andrea

5

Ok,

I changed, as for a test, the garbage collection / application throughput ratio to little bigger and increase the maximum heap near to maximum RAM. I am not counting this would help, but we will see. I had similar problems with a virtual server I used earlier. It had only 512 MB RAM so I thought that was the reason for hanging. But now with 4 GB RAM there are same kind of symtoms.

Yeah, I should make those overviews. Tiling I am using. Here is the command line I used for Geotiff converting. I guess the LZW is not a good idea?

gdal_translate -of GTiff -co "PROFILE=GeoTIFF" -co "INTERLEAVE=PIXEL" -co "COMPRESS=LZW" -co "TILED=YES" -a_srs EPSG:2393 source.tif destination.tif

- mika -

Andrea Aime kirjoitti:

Lehtonen, Mika ha scritto:

The processor is Intel Xeon E5420 quad-core (Dell PE 1950 III)

Hmm, with such configuration the jvm should auto configure
for best performance. Are the original coverages optimized
for good performance? That is, are the inner tiled geotiffs
with overviews?

Some hints on how to create one is provided here:
http://docs.codehaus.org/display/GEOSDOC/High+performance+coverage+serving

Cheers
Andrea

6

Lehtonen, Mika ha scritto:

Ok,

I changed, as for a test, the garbage collection / application throughput ratio to little bigger and increase the maximum heap near to maximum RAM. I am not counting this would help, but we will see. I had similar problems with a virtual server I used earlier. It had only 512 MB RAM so I thought that was the reason for hanging. But now with 4 GB RAM there are same kind of symtoms.

Yeah, I should make those overviews. Tiling I am using. Here is the command line I used for Geotiff converting. I guess the LZW is not a good idea?

gdal_translate -of GTiff -co "PROFILE=GeoTIFF" -co "INTERLEAVE=PIXEL" -co "COMPRESS=LZW" -co "TILED=YES" -a_srs EPSG:2393 source.tif destination.tif

If you have fast disks, it may be better to leave tiles uncompressed,
otherwise try alternate compressions.
For examples, we have good results with JPEG compression, since
decompression is fast. Of course you can use it only if you can
accept a slight degradation in image quality, but I think you can
control the compression factor to make it acceptable.

Cheers
Andrea

7

Yep, disks are fast (15000 rpm SAS).

I prefer not to use JPG. I have several bi-level tifs which I have to put on top of each other. So they are "line art" natively but in order to get them as I described, I have to merge them into one color or grayscale file. Jpeg would ruin the whole thing. There is no more clever way to put multiple rasters into Geoserver? There are no transparency option for coverages?

mika

Andrea Aime kirjoitti:

Lehtonen, Mika ha scritto:

Ok,

I changed, as for a test, the garbage collection / application throughput ratio to little bigger and increase the maximum heap near to maximum RAM. I am not counting this would help, but we will see. I had similar problems with a virtual server I used earlier. It had only 512 MB RAM so I thought that was the reason for hanging. But now with 4 GB RAM there are same kind of symtoms.

Yeah, I should make those overviews. Tiling I am using. Here is the command line I used for Geotiff converting. I guess the LZW is not a good idea?

gdal_translate -of GTiff -co "PROFILE=GeoTIFF" -co "INTERLEAVE=PIXEL" -co "COMPRESS=LZW" -co "TILED=YES" -a_srs EPSG:2393 source.tif destination.tif

If you have fast disks, it may be better to leave tiles uncompressed,
otherwise try alternate compressions.
For examples, we have good results with JPEG compression, since
decompression is fast. Of course you can use it only if you can
accept a slight degradation in image quality, but I think you can
control the compression factor to make it acceptable.

Cheers
Andrea

8

Lehtonen, Mika ha scritto:

Yep, disks are fast (15000 rpm SAS).

I prefer not to use JPG. I have several bi-level tifs which I have to put on top of each other. So they are "line art" natively but in order to get them as I described, I have to merge them into one color or grayscale file. Jpeg would ruin the whole thing. There is no more clever way to put multiple rasters into Geoserver?

What do you mean? More clever than what?

There are no transparency option for coverages?

GeoServer should use transparency if the coverage
is using:
* 4byte ARGB pixels with transparency
* indexed palette with a declared transparent color
Not sure there is a way to tell one pixel is transparent in binary images thought.

If you're in one of these cases and GeoServer is not respecting
transparency can you open a jira issue at jira.codehaus.org and
attach a sample image (a small one would do)

Cheers
Andrea

9

I meant that transparency thing. I haven't got it working. And because I use lots of bilevel files, life is getting unpleasent. Overlapping images can't be used and no overlaying. Switching opacity in SLD won't solve anything but can be used while testing.

Has anybody else experience of transparent coverages, that is rasters?

Besides these problems, coverage store is working fine and wms request with coverage datasets are handled extremely fast, though wms from e.g. shp-files has been more unbelieveble. Producing huge rasters out of shape happens in no time at all.

- mika -

Andrea Aime kirjoitti:

Lehtonen, Mika ha scritto:

Yep, disks are fast (15000 rpm SAS).

I prefer not to use JPG. I have several bi-level tifs which I have to put on top of each other. So they are "line art" natively but in order to get them as I described, I have to merge them into one color or grayscale file. Jpeg would ruin the whole thing. There is no more clever way to put multiple rasters into Geoserver?

What do you mean? More clever than what?

There are no transparency option for coverages?

GeoServer should use transparency if the coverage
is using:
* 4byte ARGB pixels with transparency
* indexed palette with a declared transparent color
Not sure there is a way to tell one pixel is transparent in binary images thought.

If you're in one of these cases and GeoServer is not respecting
transparency can you open a jira issue at jira.codehaus.org and
attach a sample image (a small one would do)

Cheers
Andrea

10

Ciao Mika,
can you copy and paste a gdalinfo on one of these coverages?

A consideration: If your coverages are quite big and you use them
without overviews you are going to end up with doing a lot of in
memory resampling. Depending on the size of the original coverage this
might cause some jvm freezes when the garbage collector kicks in
(parallel GC should help but looking a little bit at the gc dump would
probably help you spot better configs).

Simone.

On Fri, Feb 15, 2008 at 11:18 AM, Lehtonen, Mika <mika@anonymised.com> wrote:

Ok,

I changed, as for a test, the garbage collection / application
throughput ratio to little bigger and increase the maximum heap near to
maximum RAM. I am not counting this would help, but we will see. I had
similar problems with a virtual server I used earlier. It had only 512
MB RAM so I thought that was the reason for hanging. But now with 4 GB
RAM there are same kind of symtoms.

Yeah, I should make those overviews. Tiling I am using. Here is the
command line I used for Geotiff converting. I guess the LZW is not a
good idea?

gdal_translate -of GTiff -co "PROFILE=GeoTIFF" -co "INTERLEAVE=PIXEL"
-co "COMPRESS=LZW" -co "TILED=YES" -a_srs EPSG:2393 source.tif
destination.tif

- mika -

Andrea Aime kirjoitti:

> Lehtonen, Mika ha scritto:
>> The processor is Intel Xeon E5420 quad-core (Dell PE 1950 III)
>
> Hmm, with such configuration the jvm should auto configure
> for best performance. Are the original coverages optimized
> for good performance? That is, are the inner tiled geotiffs
> with overviews?
>
> Some hints on how to create one is provided here:
> http://docs.codehaus.org/display/GEOSDOC/High+performance+coverage+serving
>
>
> Cheers
> Andrea

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions S.A.S.
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it

-------------------------------------------------------

11

Lehtonen, Mika ha scritto:

I meant that transparency thing. I haven't got it working. And because I use lots of bilevel files, life is getting unpleasent. Overlapping images can't be used and no overlaying. Switching opacity in SLD won't solve anything but can be used while testing.

Has anybody else experience of transparent coverages, that is rasters?

I've tried to force a transaprency in a binary image I have with the following command:
gdal_translate -a_srs myfile.prj -a_nodata 0 myfile.tif output.tif -co "NBITS=1" -co "TILED=YES" -co "COMPRESS=DEFLATE"

and gdal_info on the output looks like:

Driver: GTiff/GeoTIFF
Files: cn.tif
Size is 4972, 10953
Coordinate System is:
PROJCS["Monte Mario / Italy zone 1",
     GEOGCS["Monte Mario",
         DATUM["Monte_Mario",
             SPHEROID["International 1924",6378388,297.0000000000014,
                 AUTHORITY["EPSG","7022"]],
             AUTHORITY["EPSG","6265"]],
         PRIMEM["Greenwich",0],
         UNIT["degree",0.0174532925199433],
         AUTHORITY["EPSG","4265"]],
     PROJECTION["Transverse_Mercator"],
     PARAMETER["latitude_of_origin",0],
     PARAMETER["central_meridian",9],
     PARAMETER["scale_factor",0.9996],
     PARAMETER["false_easting",1500000],
     PARAMETER["false_northing",0],
     UNIT["metre",1,
         AUTHORITY["EPSG","9001"]],
     AUTHORITY["EPSG","3003"]]
GeoTransform =
   1402796.794050454, -5.270943e-006, 0.112851313689
   5000064.879096299, 0.112851313599, 5.270973e-006
Metadata:
   AREA_OR_POINT=Area
   TIFFTAG_DATETIME=2006:12:19 10:21:07
   TIFFTAG_XRESOLUTION=450
   TIFFTAG_YRESOLUTION=450
   TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)
Image Structure Metadata:
   COMPRESSION=PACKBITS
   INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 1402796.794, 5000064.879) ( 7d45'48.95"E, 45d 8'47.50"N)
Lower Left ( 1404032.854, 5000064.937) ( 7d46'45.54"E, 45d 8'48.11"N)
Upper Right ( 1402796.768, 5000625.976) ( 7d45'48.56"E, 45d 9'5.68"N)
Lower Right ( 1404032.828, 5000626.034) ( 7d46'45.15"E, 45d 9'6.29"N)
Center ( 1403414.811, 5000345.456) ( 7d46'17.05"E, 45d 8'56.90"N)
Band 1 Block=256x256 Type=Byte, ColorInterp=Palette
   NoData Value=0
   Image Structure Metadata:
     NBITS=1
   Color Table (RGB with 2 entries)
     0: 0,0,0,255
     1: 255,255,255,255

As you can see the NoData information is retained, but apparently
the GeoTiff reader is not using it to create a transparency in
the data.

Simone, cc'ed, is the mantainer of the geotiff reader. Simone, any
idea?

Cheers
Andrea

12

Terve Simone!

Here you are. I haven’t yet used overviews, but I tried today. Unfortunately I managed to get Tomcat broken. It won’t response any more to any http requests.

That GS stuff is little too complicated for me but if you can give easy hints what to do, I would appreciate that a lot. I just can’t figure out why the server hangs while I still got a lot of free memory and a quadcore processor. I just don’t sound reasonable. But on the other hand, I really don’t understand what happens uder hood.

I am usin jsvc script to start Tom and usually when it gets stuck, jsvc process is reserving one core entirely and won’t let it free until I kill the whole process like: killall -9 jsvc. What would you consider of that?

with best regards

  • mika -

Simone Giannecchini kirjoitti:

ukko_kapo_gdalinfo.txt (1.31 KB)

···
13

Hi Andrea, Simone and the others!

Testing raster data with geotiff overviews (2,4,8) was quite promising. I lifted the heap sizes up to 2,5g min and 3,5g max, and added the line which defines the garbage collection versus application processing ratio to be 1/49 (application gets 50 times more time than gc, default is 100). At least outputting big rasters (12000x12000x24) with wms worked and old tom was stable after that, willing to cooperate with other duties. Even though I managed to get out of memory message with even more demanding requests, tom returned to its stable status after occasion and put out less demanding jobs. So far so good.

Of what I am a little bit worried is the rendering of these lineart rasters. Using overviews makes zoom out view even worse. Take a look at the image I attached. I know this is a difficult job and there are only few lineart raster users like me I guess but still there might be some work to do. I am using fixed resolutions which may not be a good idea? Would it be better to use autoresolution? If the overview resolution would fit into the screen resolution, the outcome would be much better, isn't that right?

And finally about that transparency. The 1-bit raster shouldn't be obstacle of getting transparency working. It can always be converted to any other depth, right? I haven't been able to get any raster transparent. If I remember right, I even tried gif or some other format with native transparency option and couldn't get it work. I would very much like to get some insructions how it should be done.

reg. mika

Andrea Aime kirjoitti:

Lehtonen, Mika ha scritto:

I meant that transparency thing. I haven't got it working. And because I use lots of bilevel files, life is getting unpleasent. Overlapping images can't be used and no overlaying. Switching opacity in SLD won't solve anything but can be used while testing.

Has anybody else experience of transparent coverages, that is rasters?

I've tried to force a transaprency in a binary image I have with the following command:
gdal_translate -a_srs myfile.prj -a_nodata 0 myfile.tif output.tif -co "NBITS=1" -co "TILED=YES" -co "COMPRESS=DEFLATE"

and gdal_info on the output looks like:

Driver: GTiff/GeoTIFF
Files: cn.tif
Size is 4972, 10953
Coordinate System is:
PROJCS["Monte Mario / Italy zone 1",
    GEOGCS["Monte Mario",
        DATUM["Monte_Mario",
            SPHEROID["International 1924",6378388,297.0000000000014,
                AUTHORITY["EPSG","7022"]],
            AUTHORITY["EPSG","6265"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433],
        AUTHORITY["EPSG","4265"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",9],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",1500000],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AUTHORITY["EPSG","3003"]]
GeoTransform =
  1402796.794050454, -5.270943e-006, 0.112851313689
  5000064.879096299, 0.112851313599, 5.270973e-006
Metadata:
  AREA_OR_POINT=Area
  TIFFTAG_DATETIME=2006:12:19 10:21:07
  TIFFTAG_XRESOLUTION=450
  TIFFTAG_YRESOLUTION=450
  TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)
Image Structure Metadata:
  COMPRESSION=PACKBITS
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 1402796.794, 5000064.879) ( 7d45'48.95"E, 45d 8'47.50"N)
Lower Left ( 1404032.854, 5000064.937) ( 7d46'45.54"E, 45d 8'48.11"N)
Upper Right ( 1402796.768, 5000625.976) ( 7d45'48.56"E, 45d 9'5.68"N)
Lower Right ( 1404032.828, 5000626.034) ( 7d46'45.15"E, 45d 9'6.29"N)
Center ( 1403414.811, 5000345.456) ( 7d46'17.05"E, 45d 8'56.90"N)
Band 1 Block=256x256 Type=Byte, ColorInterp=Palette
  NoData Value=0
  Image Structure Metadata:
    NBITS=1
  Color Table (RGB with 2 entries)
    0: 0,0,0,255
    1: 255,255,255,255

As you can see the NoData information is retained, but apparently
the GeoTiff reader is not using it to create a transparency in
the data.

Simone, cc'ed, is the mantainer of the geotiff reader. Simone, any
idea?

Cheers
Andrea

rendering_demo.png

14

Hi Andrea and Simone,

I sent a message (yesterday evening) which concerned these raster issues. But it had too big attachment (~400 k) so it stuck for waiting moderator's approval. So if you have a possibility, please take a look at it.

Thanks
mika

Andrea Aime kirjoitti:

Lehtonen, Mika ha scritto:

I meant that transparency thing. I haven't got it working. And because I use lots of bilevel files, life is getting unpleasent. Overlapping images can't be used and no overlaying. Switching opacity in SLD won't solve anything but can be used while testing.

Has anybody else experience of transparent coverages, that is rasters?

I've tried to force a transaprency in a binary image I have with the following command:
gdal_translate -a_srs myfile.prj -a_nodata 0 myfile.tif output.tif -co "NBITS=1" -co "TILED=YES" -co "COMPRESS=DEFLATE"

and gdal_info on the output looks like:

Driver: GTiff/GeoTIFF
Files: cn.tif
Size is 4972, 10953
Coordinate System is:
PROJCS["Monte Mario / Italy zone 1",
    GEOGCS["Monte Mario",
        DATUM["Monte_Mario",
            SPHEROID["International 1924",6378388,297.0000000000014,
                AUTHORITY["EPSG","7022"]],
            AUTHORITY["EPSG","6265"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433],
        AUTHORITY["EPSG","4265"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",9],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",1500000],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AUTHORITY["EPSG","3003"]]
GeoTransform =
  1402796.794050454, -5.270943e-006, 0.112851313689
  5000064.879096299, 0.112851313599, 5.270973e-006
Metadata:
  AREA_OR_POINT=Area
  TIFFTAG_DATETIME=2006:12:19 10:21:07
  TIFFTAG_XRESOLUTION=450
  TIFFTAG_YRESOLUTION=450
  TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)
Image Structure Metadata:
  COMPRESSION=PACKBITS
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 1402796.794, 5000064.879) ( 7d45'48.95"E, 45d 8'47.50"N)
Lower Left ( 1404032.854, 5000064.937) ( 7d46'45.54"E, 45d 8'48.11"N)
Upper Right ( 1402796.768, 5000625.976) ( 7d45'48.56"E, 45d 9'5.68"N)
Lower Right ( 1404032.828, 5000626.034) ( 7d46'45.15"E, 45d 9'6.29"N)
Center ( 1403414.811, 5000345.456) ( 7d46'17.05"E, 45d 8'56.90"N)
Band 1 Block=256x256 Type=Byte, ColorInterp=Palette
  NoData Value=0
  Image Structure Metadata:
    NBITS=1
  Color Table (RGB with 2 entries)
    0: 0,0,0,255
    1: 255,255,255,255

As you can see the NoData information is retained, but apparently
the GeoTiff reader is not using it to create a transparency in
the data.

Simone, cc'ed, is the mantainer of the geotiff reader. Simone, any
idea?

Cheers
Andrea

--
Sähköpostiosoitteeni on vaihtunut. Uusi osoite on mika@anonymised.com
Vanhaan osoitteeseen tulevat postit kääntyvät uuteen 25.4. asti.

My e-mail address has been changed. New address is mika@anonymised.com
Mails arriving in the old address will be forwarded to the new one till 25th April.

Mika Lehtonen
XML-Scanning Littoinen Ky
Lankakatu 2 E 13, 20660 Littoinen
mika@anonymised.com
mbl +358 (0)44 2908259

15

Hi,

  Have you thought about using the original line work rather than
bilevel tiffs? Most vector will be much smaller than the associated tiff
especially if you build pyramid layers of detail. The WFS outputs svg, gml,
kml, and the svg can easily be adapted for xaml. Svg and xaml can nicely
stack multiple raster and vector layers in the same client browser. This
would remove the bilevel transparency issue as well as size issue.

  I suspect that downsize sampling pyramids for bilevel would make
lines disappear sporadically at higher zoom out levels. This wouldn't happen
for vectors.

randy

-----Original Message-----
From: geoserver-users-bounces@lists.sourceforge.net
[mailto:geoserver-users-bounces@lists.sourceforge.net] On Behalf Of
Lehtonen, Mika
Sent: Sunday, February 17, 2008 2:55 PM
To: Andrea Aime
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] Tomcat hangs

Hi Andrea, Simone and the others!

Testing raster data with geotiff overviews (2,4,8) was quite promising.
I lifted the heap sizes up to 2,5g min and 3,5g max, and added the line
which defines the garbage collection versus application processing ratio to
be 1/49 (application gets 50 times more time than gc, default is 100). At
least outputting big rasters (12000x12000x24) with wms worked and old tom
was stable after that, willing to cooperate with other duties. Even though I
managed to get out of memory message with even more demanding requests, tom
returned to its stable status after occasion and put out less demanding
jobs. So far so good.

Of what I am a little bit worried is the rendering of these lineart rasters.
Using overviews makes zoom out view even worse. Take a look at the image I
attached. I know this is a difficult job and there are only few lineart
raster users like me I guess but still there might be some work to do. I am
using fixed resolutions which may not be a good idea?
Would it be better to use autoresolution? If the overview resolution would
fit into the screen resolution, the outcome would be much better, isn't that
right?

And finally about that transparency. The 1-bit raster shouldn't be obstacle
of getting transparency working. It can always be converted to any other
depth, right? I haven't been able to get any raster transparent. If I
remember right, I even tried gif or some other format with native
transparency option and couldn't get it work. I would very much like to get
some insructions how it should be done.

reg. mika

Andrea Aime kirjoitti:

Lehtonen, Mika ha scritto:

I meant that transparency thing. I haven't got it working. And
because I use lots of bilevel files, life is getting unpleasent.
Overlapping images can't be used and no overlaying. Switching opacity
in SLD won't solve anything but can be used while testing.

Has anybody else experience of transparent coverages, that is rasters?

I've tried to force a transaprency in a binary image I have with the
following command:
gdal_translate -a_srs myfile.prj -a_nodata 0 myfile.tif output.tif -co
"NBITS=1" -co "TILED=YES" -co "COMPRESS=DEFLATE"

and gdal_info on the output looks like:

Driver: GTiff/GeoTIFF
Files: cn.tif
Size is 4972, 10953
Coordinate System is:
PROJCS["Monte Mario / Italy zone 1",
    GEOGCS["Monte Mario",
        DATUM["Monte_Mario",
            SPHEROID["International 1924",6378388,297.0000000000014,
                AUTHORITY["EPSG","7022"]],
            AUTHORITY["EPSG","6265"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433],
        AUTHORITY["EPSG","4265"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",9],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",1500000],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AUTHORITY["EPSG","3003"]]
GeoTransform =
  1402796.794050454, -5.270943e-006, 0.112851313689
  5000064.879096299, 0.112851313599, 5.270973e-006
Metadata:
  AREA_OR_POINT=Area
  TIFFTAG_DATETIME=2006:12:19 10:21:07
  TIFFTAG_XRESOLUTION=450
  TIFFTAG_YRESOLUTION=450
  TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch) Image Structure Metadata:
  COMPRESSION=PACKBITS
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 1402796.794, 5000064.879) ( 7d45'48.95"E, 45d
8'47.50"N) Lower Left ( 1404032.854, 5000064.937) ( 7d46'45.54"E,
45d 8'48.11"N) Upper Right ( 1402796.768, 5000625.976) (
7d45'48.56"E, 45d 9'5.68"N) Lower Right ( 1404032.828, 5000626.034) (

7d46'45.15"E, 45d 9'6.29"N)

Center ( 1403414.811, 5000345.456) ( 7d46'17.05"E, 45d 8'56.90"N)
Band 1 Block=256x256 Type=Byte, ColorInterp=Palette
  NoData Value=0
  Image Structure Metadata:
    NBITS=1
  Color Table (RGB with 2 entries)
    0: 0,0,0,255
    1: 255,255,255,255

As you can see the NoData information is retained, but apparently the
GeoTiff reader is not using it to create a transparency in the data.

Simone, cc'ed, is the mantainer of the geotiff reader. Simone, any
idea?

Cheers
Andrea

16

Lehtonen, Mika ha scritto:

Hi Andrea and Simone,

I sent a message (yesterday evening) which concerned these raster issues. But it had too big attachment (~400 k) so it stuck for waiting moderator's approval. So if you have a possibility, please take a look at it.

Mika,
mi impression is that GeoServer is taking the fast route for rendering,
that is, subsampling from the last overview, and you don't have enough overviews (try gdalinfo on that file, the smallest overview should still
very big, something like 2000x2000).
I'd suggest ou ramp up the overviews up to 64 (2 4 8 16 32 64) and
the rendered quality should improve.

Cheers
Andrea

17

Hi Randy,

I would gladly use vectors if they’d exist. These are hand drawn maps, then scanned, warped, overlayed and plotted out as raster. There is no way to get them as a vector. These are “zone planning” maps (don’t know the exact word in English). For example automatic vector conversion can’t be seriously counted as a method of make them usable.
Trust me, the only reasonable way is use them as a raster. I still appreciate your effort, thanks.

reg mika

Randy George kirjoitti:

18

Hi,

first of all, the original issue; I got it solved, I want to believe so anyway. I am using apache cocoon (fop/batik) for pdf generation and received valuable information from cocoon-users list. Big rasters come out at least 14000x14000 px size through wms (4G RAM). And it seems that tommy is stable and won't hang anymore.

Second issue: in order to keep overview that big (2000x2000) I can only use parameters like 2 3 4 6 8 in gdaladdo. Using something like 64 makes smallest overview from 15000 px to be 234 pix. Anyway the outcome is bad when zooming out. These overviews should be filtered with blur, soften or whatever one calls it, filter. That is the answer I believe. But how to do it without using any GUI raster manipulator? Gdaladdo won't do it; average and nearest method are the only parameters which can be applied.
Can Geoserver use a separate overview file? It would be easier to filter overviews separately; since we are talking about multipage tif, right? Any idea? Please remember, that I don't want the "original" raster to be filtered. The best printing is made from sharp original; softening mixed with a printers rasterizing won't produce the best result.

And finally transparency; any new information about it?

with best regards
mika

Andrea Aime kirjoitti:

Lehtonen, Mika ha scritto:

Hi Andrea and Simone,

I sent a message (yesterday evening) which concerned these raster issues. But it had too big attachment (~400 k) so it stuck for waiting moderator's approval. So if you have a possibility, please take a look at it.

Mika,
mi impression is that GeoServer is taking the fast route for rendering,
that is, subsampling from the last overview, and you don't have enough overviews (try gdalinfo on that file, the smallest overview should still
very big, something like 2000x2000).
I'd suggest ou ramp up the overviews up to 64 (2 4 8 16 32 64) and
the rendered quality should improve.

Cheers
Andrea

19

Lehtonen, Mika ha scritto:

Hi,

first of all, the original issue; I got it solved, I want to believe so anyway. I am using apache cocoon (fop/batik) for pdf generation and received valuable information from cocoon-users list. Big rasters come out at least 14000x14000 px size through wms (4G RAM). And it seems that tommy is stable and won't hang anymore.

Second issue: in order to keep overview that big (2000x2000) I can only use parameters like 2 3 4 6 8 in gdaladdo. Using something like 64 makes smallest overview from 15000 px to be 234 pix.

Using non power of 2 overviews level is useless. Try 2 4 8 16 32.

Anyway the outcome is bad when zooming out. These overviews should be filtered with blur, soften or whatever one calls it, filter. That is the answer I believe. But how to do it without using any GUI raster manipulator? Gdaladdo won't do it; average and nearest method are the only parameters which can be applied.
Can Geoserver use a separate overview file? It would be easier to filter overviews separately; since we are talking about multipage tif, right? Any idea? Please remember, that I don't want the "original" raster to be filtered. The best printing is made from sharp original; softening mixed with a printers rasterizing won't produce the best result.

No, GeoServer cannot use external overviews. But I guess you might try
something like this:
* create scaled down versions of your data, by powers of 2, using a
   professional photo editing software and then convert them to geotiff
   using gdal_translate
* associate each one with an SLD that has a strict scale rendering rule,
   something like

   <Rule>
     <MinScaleDenominator>1000</MinScaleDenominator>
     <MaxScaleDenominator>2000</MaxScaleDenominator>
     <RasterSymbolizer>
       ....

* make a GetMap request that lists all the "oveview" layers one
   on top of the other, if the scale rules are properly set, only
   one will be drawn (you can also set up a layer group to allow
   using just one layer)

And finally transparency; any new information about it?

None, I'm too busy to work on it right now and Simone did not answer either. If you're in a hurry you may want to consider commercial support, try to contact either Chris Holmes (cholmes at openplans dot org) or Simone Giannecchini (simboss1 at gmail dot com) for a quote.

Cheers
Andrea

20

Andrea Aime ha scritto:

Anyway the outcome is bad when zooming out. These overviews should be filtered with blur, soften or whatever one calls it, filter. That is the answer I believe. But how to do it without using any GUI raster manipulator? Gdaladdo won't do it; average and nearest method are the only parameters which can be applied.
Can Geoserver use a separate overview file? It would be easier to filter overviews separately; since we are talking about multipage tif, right? Any idea? Please remember, that I don't want the "original" raster to be filtered. The best printing is made from sharp original; softening mixed with a printers rasterizing won't produce the best result.

Oh, forgot one thing. Did you enable the bilinear interpolation in the
WMS configuration panel?
Cheers
Andrea