[Geoserver-users] Memory usage and Geotiff

GeoServer GeoServer User
1

Hello

I'm working with some big raster files and I'm finding some problems related
to memory allocation.
The tiff I need to publish are quite a lot but in this first try I'm working
with only two of them. They are 1 bit images with a dimension of 11653 X
8995 pixels. The original files was tiff with world file associated. I used
gadl_translate to export in geotiff format.
Then I put them in my Geoserver 1.6.1 instance running on a Tomcat 6.0.14
with JVM 1.6.0_03. The initial memory pool is 128 Mbytes and the Maximum
memory pool is 512 Mbytes.
After cataloging in Geoserver I tried them with the open layer demo included
in Geoserver. I got the images but after a lot of zoom or pan the memory
allocation of tomcat6.exe process is raising up to the limit and the
computer show a substantial decrease of performance.
I tried to quit my client requests and wait for a memory release but nothing
happens and the only solution is to restart tomcat service.
Of course I'm getting wrong in something step of the process but I can't see
which.
Thanks a lot for your help

2

Hi Stefano,

I am also working quite often with 1-bit images, so maybe we could exchange our information and experience if needed, I mean also in the future.

I guess you are having problems with Tomcat, not with geoserver. Try the following link. It has some very good sublinks of how to tune the Tommy (jvm).

http://blogs.sun.com/watt/resource/jvm-options-list.html

I would recommended you to increase your heap near to maximum possible and also tune the initial heap near to that. That's a good starting point. You also have to probably check your garbage collection parameters and so on. But with the information you provided, it is difficult to say more.

If you'd like to chat about Tomcat tuning, you can send me a mail directly to mika@anonymised.com Like I said, I don't believe this is a Geoserver issue.

regards
mika

Stefano Iacovella kirjoitti:

Hello

I'm working with some big raster files and I'm finding some problems related
to memory allocation.
The tiff I need to publish are quite a lot but in this first try I'm working
with only two of them. They are 1 bit images with a dimension of 11653 X
8995 pixels. The original files was tiff with world file associated. I used
gadl_translate to export in geotiff format.
Then I put them in my Geoserver 1.6.1 instance running on a Tomcat 6.0.14
with JVM 1.6.0_03. The initial memory pool is 128 Mbytes and the Maximum
memory pool is 512 Mbytes.
After cataloging in Geoserver I tried them with the open layer demo included
in Geoserver. I got the images but after a lot of zoom or pan the memory
allocation of tomcat6.exe process is raising up to the limit and the
computer show a substantial decrease of performance.
I tried to quit my client requests and wait for a memory release but nothing
happens and the only solution is to restart tomcat service.
Of course I'm getting wrong in something step of the process but I can't see
which.
Thanks a lot for your help

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
  

3

Even if it's not a geoserver issue it'd be great to put notes up on the GeoServer wiki, in the user section, as this is obviously a problem that multiple users are hitting. I can direct you on where to put it in the wiki.

Chris

Lehtonen, Mika wrote:

Hi Stefano,

I am also working quite often with 1-bit images, so maybe we could exchange our information and experience if needed, I mean also in the future.

I guess you are having problems with Tomcat, not with geoserver. Try the following link. It has some very good sublinks of how to tune the Tommy (jvm).

http://blogs.sun.com/watt/resource/jvm-options-list.html

I would recommended you to increase your heap near to maximum possible and also tune the initial heap near to that. That's a good starting point. You also have to probably check your garbage collection parameters and so on. But with the information you provided, it is difficult to say more.

If you'd like to chat about Tomcat tuning, you can send me a mail directly to mika@anonymised.com Like I said, I don't believe this is a Geoserver issue.

regards
mika

Stefano Iacovella kirjoitti:

Hello

I'm working with some big raster files and I'm finding some problems related
to memory allocation.
The tiff I need to publish are quite a lot but in this first try I'm working
with only two of them. They are 1 bit images with a dimension of 11653 X
8995 pixels. The original files was tiff with world file associated. I used
gadl_translate to export in geotiff format.
Then I put them in my Geoserver 1.6.1 instance running on a Tomcat 6.0.14
with JVM 1.6.0_03. The initial memory pool is 128 Mbytes and the Maximum
memory pool is 512 Mbytes.
After cataloging in Geoserver I tried them with the open layer demo included
in Geoserver. I got the images but after a lot of zoom or pan the memory
allocation of tomcat6.exe process is raising up to the limit and the
computer show a substantial decrease of performance.
I tried to quit my client requests and wait for a memory release but nothing
happens and the only solution is to restart tomcat service.
Of course I'm getting wrong in something step of the process but I can't see
which.
Thanks a lot for your help

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
  
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:4005,47cffda1139702143011171!

4

Sure,

why not, fine by me.

Probably there are many like me, testing and testing and testing without a proper knowledge and knowhow of the details. I guess something useful will come out of it eventually; you know by browsing internet and putting things together (and testing).

Large raster datasets is in any case the situation which isn't taken into account in many instructions concerning the JVM tuning. So maybe we could come out with something "fresh" by investigating it. I myself, have been driving my server into its limits by printing large datasets into pdf (with Apache FOP). I haven't been running into any problems with Geoserver wms. And I am using a heap size of over 3g and planning to still increase it after I got more RAM.

- mika -

Chris Holmes kirjoitti:

Even if it's not a geoserver issue it'd be great to put notes up on the GeoServer wiki, in the user section, as this is obviously a problem that multiple users are hitting. I can direct you on where to put it in the wiki.

Chris

Lehtonen, Mika wrote:

Hi Stefano,

I am also working quite often with 1-bit images, so maybe we could exchange our information and experience if needed, I mean also in the future.

I guess you are having problems with Tomcat, not with geoserver. Try the following link. It has some very good sublinks of how to tune the Tommy (jvm).

http://blogs.sun.com/watt/resource/jvm-options-list.html

I would recommended you to increase your heap near to maximum possible and also tune the initial heap near to that. That's a good starting point. You also have to probably check your garbage collection parameters and so on. But with the information you provided, it is difficult to say more.

If you'd like to chat about Tomcat tuning, you can send me a mail directly to mika@anonymised.com Like I said, I don't believe this is a Geoserver issue.

regards
mika

Stefano Iacovella kirjoitti:

Hello

I'm working with some big raster files and I'm finding some problems related
to memory allocation.
The tiff I need to publish are quite a lot but in this first try I'm working
with only two of them. They are 1 bit images with a dimension of 11653 X
8995 pixels. The original files was tiff with world file associated. I used
gadl_translate to export in geotiff format.
Then I put them in my Geoserver 1.6.1 instance running on a Tomcat 6.0.14
with JVM 1.6.0_03. The initial memory pool is 128 Mbytes and the Maximum
memory pool is 512 Mbytes.
After cataloging in Geoserver I tried them with the open layer demo included
in Geoserver. I got the images but after a lot of zoom or pan the memory
allocation of tomcat6.exe process is raising up to the limit and the
computer show a substantial decrease of performance.
I tried to quit my client requests and wait for a memory release but nothing
happens and the only solution is to restart tomcat service.
Of course I'm getting wrong in something step of the process but I can't see
which.
Thanks a lot for your help

-------------------------------------------------------------------------

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
  
-------------------------------------------------------------------------

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:4005,47cffda1139702143011171!

5

Stefano Iacovella ha scritto:

Hello

I'm working with some big raster files and I'm finding some problems related
to memory allocation.
The tiff I need to publish are quite a lot but in this first try I'm working
with only two of them. They are 1 bit images with a dimension of 11653 X
8995 pixels. The original files was tiff with world file associated. I used
gadl_translate to export in geotiff format.
Then I put them in my Geoserver 1.6.1 instance running on a Tomcat 6.0.14
with JVM 1.6.0_03. The initial memory pool is 128 Mbytes and the Maximum
memory pool is 512 Mbytes.
After cataloging in Geoserver I tried them with the open layer demo included
in Geoserver. I got the images but after a lot of zoom or pan the memory
allocation of tomcat6.exe process is raising up to the limit and the
computer show a substantial decrease of performance.
I tried to quit my client requests and wait for a memory release but nothing
happens and the only solution is to restart tomcat service.

A few remarks:
* once the java vm grabs memory it won't give it back, no way to make it
   do so... but high memory usage reported by the task manager or "top"
   does not mean the heap is full. Try going into the admin panel and hit
   "free memory", does the actual heap usage stay high?
* is the big image tiled? If not I'd suggest to use gdal_translate and
   tile it, otherwise geoserver will be forced to read full rows
   out of the tiff or, if the tiff is compressed, to read it all tother,
   even if only a small part of it is needed

Let me konw if this helps
Cheers
Andrea

6

Hello all,

I am performing some try on coverages with raster layers of France organised like this:

  • layer for 1/4,000,000 : 1 file of 474k.
  • layer for 1/1,000,000 : 1 file of 4747k.
  • layer for 1/500,000 : 1 file of 16164k.
  • layer for 1/250,000 : 132 files, size ranging from 73k to 1051k. Total size: 61 Mo
  • layer for 1/100,000 : 528 files, size ranging from 112k to 1734k.Total size : 280 Mo
  • layer for 1/50,000 : 1053 files, size ranging from 71k to 1373k.Total size : 465 Mo.
  • layer for 1/20,000 : 1600 files, size ranging from 277k to 4565k.Total size : 1.8 Go.
  • layer for 1/5,000 : 23194 files, size ranging from 208k to 2761k.Total size : 13.1 Go.

The files are GeoTiff files and I use the mosaic plugin.

If I only uses the layers from 1/4,000,000 to 1/100,000, GeoServer works well : the wms service replies fast and the memory used by GeoServer remains acceptable.

Adding the 1/50,000 layer decreases dramatically the performance of the wms service and GeoServer starts using a lot of memory. I don’t mention the use of all the layers.

In fact, the tomcat uses so much memory than I can not use the admin interface and free the memory. Thus, I generally kill my tomcat instance.

So I would like to know what I can do to improve these:

  • Should I try the pyramid plugin ? Since my files are not so big, I am not sure this will help.
  • Will the use of a cache service like GeoWebCache or TileCache improve something ? I think not.

Have you any ideas ?

For my tests, I use a Windows XP, with 2 Go of RAM and a Athlon 64 x2 Dual.

Sincerelly,

Alexandre

On Thu, Mar 6, 2008 at 6:35 PM, Andrea Aime <aaime@anonymised.com> wrote:

Stefano Iacovella ha scritto:

Hello

I’m working with some big raster files and I’m finding some problems related
to memory allocation.
The tiff I need to publish are quite a lot but in this first try I’m working
with only two of them. They are 1 bit images with a dimension of 11653 X
8995 pixels. The original files was tiff with world file associated. I used
gadl_translate to export in geotiff format.
Then I put them in my Geoserver 1.6.1 instance running on a Tomcat 6.0.14
with JVM 1.6.0_03. The initial memory pool is 128 Mbytes and the Maximum
memory pool is 512 Mbytes.
After cataloging in Geoserver I tried them with the open layer demo included
in Geoserver. I got the images but after a lot of zoom or pan the memory
allocation of tomcat6.exe process is raising up to the limit and the
computer show a substantial decrease of performance.
I tried to quit my client requests and wait for a memory release but nothing
happens and the only solution is to restart tomcat service.

A few remarks:

  • once the java vm grabs memory it won’t give it back, no way to make it
    do so… but high memory usage reported by the task manager or “top”
    does not mean the heap is full. Try going into the admin panel and hit
    “free memory”, does the actual heap usage stay high?
  • is the big image tiled? If not I’d suggest to use gdal_translate and
    tile it, otherwise geoserver will be forced to read full rows
    out of the tiff or, if the tiff is compressed, to read it all tother,
    even if only a small part of it is needed

Let me konw if this helps
Cheers
Andrea

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

7

Hi,

I fear I can’t give you best possible help because I am mostly MapServer user when it comes to imagery. However there might be some analogy.

Can you tell what scale are you using when accessing the 1/50000 layers and beyond? Actually, could you calculate how many physical image files lieing there behind the tileindex shapefile you must actually open with your request? When the request is sent what happens first is that the tileindex shapefile if queried in order to find out what physical files must be opened. Large shapefiles are slower to query but using indexed shapefiles might help a bit. But if you have zoomed out very far so that the system needs to open tens or hundreds physical image files before it can render the requested window then it will be slow for sure. What helps here is to create external overview images with lower resolution and covering larger area, and make another layer out of those. Then you need to configure the system to use first the overview images and just after zooming in close enough to switch the high resolution images on. The aim might be that no more than 10 physical files needs to be opened for one request ever.

-Jukka Rahkonen-

Lähettäjä: geoserver-users-bounces@lists.sourceforge.net [mailto:geoserver-users-bounces@lists.sourceforge.net] Puolesta Alexandre Gacon
Lähetetty: 7. maaliskuuta 2008 10:20
Vastaanottaja: geoserver-users
Aihe: Re: [Geoserver-users] Memory usage and Geotiff

Hello all,

I am performing some try on coverages with raster layers of France organised like this:

  • layer for 1/4,000,000 : 1 file of 474k.
  • layer for 1/1,000,000 : 1 file of 4747k.
  • layer for 1/500,000 : 1 file of 16164k.
  • layer for 1/250,000 : 132 files, size ranging from 73k to 1051k. Total size: 61 Mo
  • layer for 1/100,000 : 528 files, size ranging from 112k to 1734k.Total size : 280 Mo
  • layer for 1/50,000 : 1053 files, size ranging from 71k to 1373k.Total size : 465 Mo.
  • layer for 1/20,000 : 1600 files, size ranging from 277k to 4565k.Total size : 1.8 Go.
  • layer for 1/5,000 : 23194 files, size ranging from 208k to 2761k.Total size : 13.1 Go.

The files are GeoTiff files and I use the mosaic plugin.

If I only uses the layers from 1/4,000,000 to 1/100,000, GeoServer works well : the wms service replies fast and the memory used by GeoServer remains acceptable.

Adding the 1/50,000 layer decreases dramatically the performance of the wms service and GeoServer starts using a lot of memory. I don’t mention the use of all the layers.

In fact, the tomcat uses so much memory than I can not use the admin interface and free the memory. Thus, I generally kill my tomcat instance.

So I would like to know what I can do to improve these:

  • Should I try the pyramid plugin ? Since my files are not so big, I am not sure this will help.
  • Will the use of a cache service like GeoWebCache or TileCache improve something ? I think not.

Have you any ideas ?

For my tests, I use a Windows XP, with 2 Go of RAM and a Athlon 64 x2 Dual.

Sincerelly,

Alexandre

On Thu, Mar 6, 2008 at 6:35 PM, Andrea Aime <aaime@anonymised.com> wrote:

Stefano Iacovella ha scritto:

Hello

I’m working with some big raster files and I’m finding some problems related
to memory allocation.
The tiff I need to publish are quite a lot but in this first try I’m working
with only two of them. They are 1 bit images with a dimension of 11653 X
8995 pixels. The original files was tiff with world file associated. I used
gadl_translate to export in geotiff format.
Then I put them in my Geoserver 1.6.1 instance running on a Tomcat 6.0.14
with JVM 1.6.0_03. The initial memory pool is 128 Mbytes and the Maximum
memory pool is 512 Mbytes.
After cataloging in Geoserver I tried them with the open layer demo included
in Geoserver. I got the images but after a lot of zoom or pan the memory
allocation of tomcat6.exe process is raising up to the limit and the
computer show a substantial decrease of performance.
I tried to quit my client requests and wait for a memory release but nothing
happens and the only solution is to restart tomcat service.

A few remarks:

  • once the java vm grabs memory it won’t give it back, no way to make it
    do so… but high memory usage reported by the task manager or “top”
    does not mean the heap is full. Try going into the admin panel and hit
    “free memory”, does the actual heap usage stay high?
  • is the big image tiled? If not I’d suggest to use gdal_translate and
    tile it, otherwise geoserver will be forced to read full rows
    out of the tiff or, if the tiff is compressed, to read it all tother,
    even if only a small part of it is needed

Let me konw if this helps
Cheers
Andrea

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Geoserver-users mailing list
Geoserver-users@anonymised.comists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

8

Hi,

After reading your mail, I follow the behavior of my wms service with attention and I found that one of the layer was configured with the default raster renderer. Thus, it was always loaded !
Using the good renderer halves the memory used. It remains not fast enough but it is a better start to look for improvements.

Do you advise to use MapServer for displaying large raster datasets ? Should I generate some pyramids to decrease the size of the images ? Should I use an other output format ?

Sincerelly,

Alexandre Gacon

On Fri, Mar 7, 2008 at 1:05 PM, Rahkonen Jukka <Jukka.Rahkonen@anonymised.com> wrote:

Hi,

I fear I can’t give you best possible help because I am mostly MapServer user when it comes to imagery. However there might be some analogy.

Can you tell what scale are you using when accessing the 1/50000 layers and beyond? Actually, could you calculate how many physical image files lieing there behind the tileindex shapefile you must actually open with your request? When the request is sent what happens first is that the tileindex shapefile if queried in order to find out what physical files must be opened. Large shapefiles are slower to query but using indexed shapefiles might help a bit. But if you have zoomed out very far so that the system needs to open tens or hundreds physical image files before it can render the requested window then it will be slow for sure. What helps here is to create external overview images with lower resolution and covering larger area, and make another layer out of those. Then you need to configure the system to use first the overview images and just after zooming in close enough to switch the high resolution images on. The aim might be that no more than 10 physical files needs to be opened for one request ever.

-Jukka Rahkonen-

Lähettäjä: geoserver-users-bounces@lists.sourceforge.net [mailto:geoserver-users-bounces@lists.sourceforge.net] Puolesta Alexandre Gacon
Lähetetty: 7. maaliskuuta 2008 10:20
Vastaanottaja: geoserver-users
Aihe: Re: [Geoserver-users] Memory usage and Geotiff

Hello all,

I am performing some try on coverages with raster layers of France organised like this:

  • layer for 1/4,000,000 : 1 file of 474k.
  • layer for 1/1,000,000 : 1 file of 4747k.
  • layer for 1/500,000 : 1 file of 16164k.
  • layer for 1/250,000 : 132 files, size ranging from 73k to 1051k. Total size: 61 Mo
  • layer for 1/100,000 : 528 files, size ranging from 112k to 1734k.Total size : 280 Mo
  • layer for 1/50,000 : 1053 files, size ranging from 71k to 1373k.Total size : 465 Mo.
  • layer for 1/20,000 : 1600 files, size ranging from 277k to 4565k.Total size : 1.8 Go.
  • layer for 1/5,000 : 23194 files, size ranging from 208k to 2761k.Total size : 13.1 Go.

The files are GeoTiff files and I use the mosaic plugin.

If I only uses the layers from 1/4,000,000 to 1/100,000, GeoServer works well : the wms service replies fast and the memory used by GeoServer remains acceptable.

Adding the 1/50,000 layer decreases dramatically the performance of the wms service and GeoServer starts using a lot of memory. I don’t mention the use of all the layers.

In fact, the tomcat uses so much memory than I can not use the admin interface and free the memory. Thus, I generally kill my tomcat instance.

So I would like to know what I can do to improve these:

  • Should I try the pyramid plugin ? Since my files are not so big, I am not sure this will help.
  • Will the use of a cache service like GeoWebCache or TileCache improve something ? I think not.

Have you any ideas ?

For my tests, I use a Windows XP, with 2 Go of RAM and a Athlon 64 x2 Dual.

Sincerelly,

Alexandre

On Thu, Mar 6, 2008 at 6:35 PM, Andrea Aime <aaime@anonymised.com> wrote:

Stefano Iacovella ha scritto:

Hello

I’m working with some big raster files and I’m finding some problems related
to memory allocation.
The tiff I need to publish are quite a lot but in this first try I’m working
with only two of them. They are 1 bit images with a dimension of 11653 X
8995 pixels. The original files was tiff with world file associated. I used
gadl_translate to export in geotiff format.
Then I put them in my Geoserver 1.6.1 instance running on a Tomcat 6.0.14
with JVM 1.6.0_03. The initial memory pool is 128 Mbytes and the Maximum
memory pool is 512 Mbytes.
After cataloging in Geoserver I tried them with the open layer demo included
in Geoserver. I got the images but after a lot of zoom or pan the memory
allocation of tomcat6.exe process is raising up to the limit and the
computer show a substantial decrease of performance.
I tried to quit my client requests and wait for a memory release but nothing
happens and the only solution is to restart tomcat service.

A few remarks:

  • once the java vm grabs memory it won’t give it back, no way to make it
    do so… but high memory usage reported by the task manager or “top”
    does not mean the heap is full. Try going into the admin panel and hit
    “free memory”, does the actual heap usage stay high?
  • is the big image tiled? If not I’d suggest to use gdal_translate and
    tile it, otherwise geoserver will be forced to read full rows
    out of the tiff or, if the tiff is compressed, to read it all tother,
    even if only a small part of it is needed

Let me konw if this helps
Cheers
Andrea

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

9

Hi,

I do not believe it is the question of which one is better but about how to get the best out of your Geoserver. What I usually do is to create one mosaiced quick look image with 5 meter pixel size (GeoTIFF with tiles and internal overview layers) from about 100 original aerial photos with 0.5 meter pixel size. Perhaps if the area is very large then another with 50 m pixel sixe from the 5 meter images. Overviews are used in far-away situation until one image pixel equals one pixel on screen. That strategy will for sure make sense with any server.

-Jukka Rahkonen-

Lähettäjä: geoserver-users-bounces@lists.sourceforge.net [mailto:geoserver-users-bounces@lists.sourceforge.net] Puolesta Alexandre Gacon
Lähetetty: 7. maaliskuuta 2008 15:02
Vastaanottaja: geoserver-users
Aihe: Re: [Geoserver-users] Memory usage and Geotiff

Hi,

After reading your mail, I follow the behavior of my wms service with attention and I found that one of the layer was configured with the default raster renderer. Thus, it was always loaded !
Using the good renderer halves the memory used. It remains not fast enough but it is a better start to look for improvements.

Do you advise to use MapServer for displaying large raster datasets ? Should I generate some pyramids to decrease the size of the images ? Should I use an other output format ?

10

Thank you for all these advice.

I will try to prepare other layers in order to have such overviews.

Alexandre

On Fri, Mar 7, 2008 at 2:14 PM, Rahkonen Jukka <Jukka.Rahkonen@anonymised.com> wrote:

Hi,

I do not believe it is the question of which one is better but about how to get the best out of your Geoserver. What I usually do is to create one mosaiced quick look image with 5 meter pixel size (GeoTIFF with tiles and internal overview layers) from about 100 original aerial photos with 0.5 meter pixel size. Perhaps if the area is very large then another with 50 m pixel sixe from the 5 meter images. Overviews are used in far-away situation until one image pixel equals one pixel on screen. That strategy will for sure make sense with any server.

-Jukka Rahkonen-

Lähettäjä: geoserver-users-bounces@lists.sourceforge.net [mailto:geoserver-users-bounces@lists.sourceforge.net] Puolesta Alexandre Gacon

Lähetetty: 7. maaliskuuta 2008 15:02

Vastaanottaja: geoserver-users
Aihe: Re: [Geoserver-users] Memory usage and Geotiff

Hi,

After reading your mail, I follow the behavior of my wms service with attention and I found that one of the layer was configured with the default raster renderer. Thus, it was always loaded !
Using the good renderer halves the memory used. It remains not fast enough but it is a better start to look for improvements.

Do you advise to use MapServer for displaying large raster datasets ? Should I generate some pyramids to decrease the size of the images ? Should I use an other output format ?

11

Alexandre Gacon ha scritto:

Hi,

After reading your mail, I follow the behavior of my wms service with attention and I found that one of the layer was configured with the default raster renderer. Thus, it was always loaded !
Using the good renderer halves the memory used. It remains not fast enough but it is a better start to look for improvements.

raster renderer, good renderer? I'm lost, GeoServer is using
a single rendering subsystem, there is no choice... can you
give us a little more details?

Cheers
Andrea

12

Hi,

In fact, I want to render my raster layers only if my map is in the good scale interval. For example if I look my map at a scale of 1/2,000,000, I must display only the 1/4,000,000 raster layer. So I configured as much styles as I have raster datasets and I set each coverage to use the prepared style. I just forgot to change the 1/50,000 layer which used the default raster style (always visible). Thus GeoServer loaded about 500 Mo of geotiff files : my Tomcat container didn’t appreciate.

Is it clear ?

Alexandre

On Fri, Mar 7, 2008 at 2:37 PM, Andrea Aime <aaime@anonymised.com> wrote:

Alexandre Gacon ha scritto:

Hi,

After reading your mail, I follow the behavior of my wms service with
attention and I found that one of the layer was configured with the
default raster renderer. Thus, it was always loaded !
Using the good renderer halves the memory used. It remains not fast
enough but it is a better start to look for improvements.

raster renderer, good renderer? I’m lost, GeoServer is using
a single rendering subsystem, there is no choice… can you
give us a little more details?

Cheers
Andrea

13

Alexandre Gacon ha scritto:
....

Do you advise to use MapServer for displaying large raster datasets ? Should I generate some pyramids to decrease the size of the images ? Should I use an other output format ?

I recently did a performance test using some geotiffs. If the
geotiff has no overviews GeoServer is quite a bit slower than
MapServer, but if you use gdaladdo to add overviews into the
geotiff files then they are more or less even (differences of less than
10%).

Generally speaking, the key to have fast raster serving in
GeoServer is to:
* tile your data
* add overviews
* experiment with compression (depending on your disk it may
   be better to keep the geotiff compressed, or not)
You can get there by using gdal_translate and gdaladdo:
http://docs.codehaus.org/display/GEOSDOC/High+performance+coverage+serving

Another key point is to have native JAI installed, the pure
java version that ships along with GeoServer is quite a bit
slower. If you are under windows or linux you can use
the installers provided here:
jai.dev.java.net
jai-imageio.dev.java.net/
If you prefer you can follow the instructions for manual
install provided here:
http://docs.codehaus.org/display/GEOT/Manual+JAI+Installation

Performance is also severely affected by the JDK you're
using and by the JVM settings, see here:
http://geoserver.org/display/GEOSDOC/6+GeoServer+in+Production+Environment

Hope this helps
Cheers
Andrea

14

I already have the native JAI and I have configured my JVM as indicated on the website.

I didn’t know about the geotiffs overview but I will look at the links you gave.

Sincerelly,

Alexandre

On Fri, Mar 7, 2008 at 2:45 PM, Andrea Aime <aaime@anonymised.com> wrote:

Alexandre Gacon ha scritto:
…

Do you advise to use MapServer for displaying large raster datasets ?
Should I generate some pyramids to decrease the size of the images ?
Should I use an other output format ?

I recently did a performance test using some geotiffs. If the
geotiff has no overviews GeoServer is quite a bit slower than
MapServer, but if you use gdaladdo to add overviews into the
geotiff files then they are more or less even (differences of less than
10%).

Generally speaking, the key to have fast raster serving in
GeoServer is to:

Another key point is to have native JAI installed, the pure
java version that ships along with GeoServer is quite a bit
slower. If you are under windows or linux you can use
the installers provided here:
jai.dev.java.net
jai-imageio.dev.java.net/
If you prefer you can follow the instructions for manual
install provided here:
http://docs.codehaus.org/display/GEOT/Manual+JAI+Installation

Performance is also severely affected by the JDK you’re
using and by the JVM settings, see here:
http://geoserver.org/display/GEOSDOC/6+GeoServer+in+Production+Environment

Hope this helps
Cheers
Andrea

15

Alexandre Gacon ha scritto:

Hi,

In fact, I want to render my raster layers only if my map is in the good scale interval. For example if I look my map at a scale of 1/2,000,000, I must display only the 1/4,000,000 raster layer. So I configured as much styles as I have raster datasets and I set each coverage to use the prepared style. I just forgot to change the 1/50,000 layer which used the default raster style (always visible). Thus GeoServer loaded about 500 Mo of geotiff files : my Tomcat container didn't appreciate.

Is it clear ?

Ah yes, it is.
Cheers
Andrea

16

In Geosevre 1.6.1, when I want add a datastore form Oracle spatial. I recive following error in log file:

2008-03-07 15:18:06,212 INFO [geoserver.filters] - 127.0.0.1 "POST /geoserver/config/data/storeSubmit.do" "Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" "http://localhost:8080/geoserver/config/data/storeSubmit.do&quot;

2008-03-07 15:18:06,212 WARN [geotools.data] - Could not acquire Oracle Spatial Database:java.lang.NoClassDefFoundError: org/geotools/geometry/JTS$ReferencedEnvelope

java.lang.NoClassDefFoundError: org/geotools/geometry/JTS$ReferencedEnvelope

            at org.geotools.data.oracle.OracleDataStoreFactory.createDataStore(OracleDataStoreFactory.java:177)

            at org.geotools.data.DataStoreFinder.getDataStore(DataStoreFinder.java:91)

            at org.vfny.geoserver.util.DataStoreUtils.getDataStore(DataStoreUtils.java:62)

            at org.vfny.geoserver.util.DataStoreUtils.acquireDataStore(DataStoreUtils.java:51)

            at org.vfny.geoserver.action.data.DataDataStoresEditorAction.execute(DataDataStoresEditorAction.java:134)

            at org.vfny.geoserver.action.ConfigAction.execute(ConfigAction.java:101)

            at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)

            at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)

            at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)

            at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)

            at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)

            at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)

            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)

            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

            at org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108)

            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)

            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

            at org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:69)

            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)

            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

            at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:41)

            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)

            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

            at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:264)

            at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)

            at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)

            at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)

            at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110)

            at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)

            at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)

            at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)

            at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)

            at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)

            at org.acegisecurity.wrapper.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:81)

            at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)

            at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:217)

            at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)

            at org.acegisecurity.ui.logout.LogoutFilter.doFilter(LogoutFilter.java:106)

            at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)

            at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:229)

            at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:274)

            at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:148)

            at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)

            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)

            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

            at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)

            at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)

            at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)

            at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)

            at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)

            at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:263)

            at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)

            at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:584)

            at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)

            at java.lang.Thread.run(Unknown Source)

I add plugin gt2-oracle-spatial.jar en ojdbc.jar.

Here is also melding in jsp:

I use same variable in geoserver 1.5.3, there is no problems:

Is something change in the new version?

Thanks,

Babak Dehghenpour library TU Delft

Mon-The-Wed-Thu-Fri ICT Developer

Room 2.32

* +3115-2789437 Prometheusplein 1

Fax +3115-2572060 P.O. Box 98

* b.dehghenpour@...1570... <mailto:b.dehghenpour@…1570…> 2628 ZC Delft

* www.library.tudelft.nl <http://www.library.tudelft.nl/&gt; The Netherland

image002.jpg

image003.jpg

17

Dehghenpour, Babak ha scritto:

In Geosevre 1.6.1, when I want add a datastore form Oracle spatial. I recive following error in log file:

2008-03-07 15:18:06,212 INFO [geoserver.filters] - 127.0.0.1 "POST /geoserver/config/data/storeSubmit.do" "Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12" "http://localhost:8080/geoserver/config/data/storeSubmit.do&quot;

2008-03-07 15:18:06,212 WARN [geotools.data] - Could not acquire Oracle Spatial Database:java.lang.NoClassDefFoundError: org/geotools/geometry/JTS$ReferencedEnvelope

java.lang.NoClassDefFoundError: org/geotools/geometry/JTS$ReferencedEnvelope

It seems to me you used the Oracle plugin for 1.5.x into 1.6.x?
Each version of GeoServer has its own plugins, they are not compatible
with a different one.
If this is the issue, try downloading the Oracle plugin for 1.6.1
and see if it works fine.

Let me know if it works for you
Cheers
Andrea

18

Andera,

Thank you Andrea now it ‘s working, but there is nothing here in documentation over difference download possibility.

http://geoserver.org/display/GEOSDOC/Oracle+DataStore

Thanks,

Babak Dehghenpour library TU Delft

Mon-The-Wed-Thu-Fri ICT Developer

Room 2.32

* +3115-2789437 Prometheusplein 1

Fax +3115-2572060 P.O. Box 98

* b.dehghenpour@...1570... <mailto:b.dehghenpour@…1570…> 2628 ZC Delft

* www.library.tudelft.nl <http://www.library.tudelft.nl/&gt; The Netherland

Dehghenpour, Babak ha scritto:

In Geosevre 1.6.1, when I want add a datastore form Oracle spatial. I

recive following error in log file:

2008-03-07 15:18:06,212 INFO [geoserver.filters] - 127.0.0.1 "POST

/geoserver/config/data/storeSubmit.do" "Mozilla/5.0 (Windows; U; Windows

NT 5.1; nl; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12"

"http://localhost:8080/geoserver/config/data/storeSubmit.do&quot;

2008-03-07 15:18:06,212 WARN [geotools.data] - Could not acquire Oracle

Spatial Database:java.lang.NoClassDefFoundError:

org/geotools/geometry/JTS$ReferencedEnvelope

java.lang.NoClassDefFoundError: org/geotools/geometry/JTS$ReferencedEnvelope

It seems to me you used the Oracle plugin for 1.5.x into 1.6.x?

Each version of GeoServer has its own plugins, they are not compatible

with a different one.

If this is the issue, try downloading the Oracle plugin for 1.6.1

and see if it works fine.

Let me know if it works for you

Cheers

Andrea

-------------------------------------------------------------------------

This SF.net email is sponsored by: Microsoft

Defy all challenges. Microsoft(R) Visual Studio 2008.

http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

_______________________________________________

Geoserver-users mailing list

Geoserver-users@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/geoserver-users