[Geoserver-users] Google Maps & WMS Layer

Hi all,

I'm using geoserver/google map to display a few geological layers.
This is software that I'm using in current setup.

Geoserver 1.4
Tomcat 5.5
Postgres 7.4
FreeBSD 6.1

I've just discover that every time I zoom in Geoserver opens a few new connections to the Postgres DB. Lower I go the more connections it opens until it reach maximum connection number allowed by Postgres. This cause dramatic reduction in performance. To illustrate I will post number of connections for a few zoom levels (with only one WMS layer enabled).

(I waited each time until layer is fully loaded and then zoom to the next level)
Zoom level 4:
No. of connections: 2

Zoom level 5:
No. of connections: 12

Zoom level 6:
No. of connections: 12

Zoom level 7:
No. of connections: 12

Zoom level 8:
No. of connections: 17

Zoom level 9:
No. of connections: 17

Zoom level 10:
No. of connections: 25

....

Following zoom level uses 30 connections and it takes a few minutes to display layer, even if it should display a single line.
Is this normal?
I'm aware that I can increase max number of connections in Postgres, but still if 100 users use map at the same time 3000 connections will be required.

Also when I zoom in to level 10 and then switch on WMS layer Geoserver opens only two connections and there is not performance degradation.

Any help or explanation will be greatly appreciated.

Sasha

Sasha ha scritto:

Hi all,

I'm using geoserver/google map to display a few geological layers.
This is software that I'm using in current setup.

Geoserver 1.4
Tomcat 5.5
Postgres 7.4
FreeBSD 6.1

I've just discover that every time I zoom in Geoserver opens a few new connections to the Postgres DB. Lower I go the more connections it opens until it reach maximum connection number allowed by Postgres. This cause dramatic reduction in performance.

Scary. Just to make sure, can you check what happens if you serve the same layer and zoom in/out using the map preview in the geoserver demo
section? Do the connection count raise as well?

Since you're using Google maps my wild guess would be that the KML
producer is leaving some feature reader open (each one uses a connection), but it's just a guess for the moment.

Cheers
Andrea Aime

Andrea Aime wrote:

Sasha ha scritto:

Hi all,

I'm using geoserver/google map to display a few geological layers.
This is software that I'm using in current setup.

Geoserver 1.4
Tomcat 5.5
Postgres 7.4
FreeBSD 6.1

I've just discover that every time I zoom in Geoserver opens a few new connections to the Postgres DB. Lower I go the more connections it opens until it reach maximum connection number allowed by Postgres. This cause dramatic reduction in performance.

Scary. Just to make sure, can you check what happens if you serve the same layer and zoom in/out using the map preview in the geoserver demo
section? Do the connection count raise as well?

Since you're using Google maps my wild guess would be that the KML
producer is leaving some feature reader open (each one uses a connection), but it's just a guess for the moment.

Cheers
Andrea Aime

Thanks for reply.

I've checked the same layer in the geoserver demo section and it opens only one connection for all zoom levels.
So your guess is probably correct, any idea of how I should solve this problem or where I can find more documentation related to this problem.

Cheers,
Sasha

Sasha ha scritto:

Andrea Aime wrote:

Sasha ha scritto:

I've checked the same layer in the geoserver demo section and it opens only one connection for all zoom levels.
So your guess is probably correct, any idea of how I should solve this problem or where I can find more documentation related to this problem.

The only way is to check Geoserver KML producer and try to locate
the unclosed reader. I've opened a Jira issue about it, if you're willing to hack and find the problem, that would be much appreciated :slight_smile:
KML generator is in the wms module, org.vfny.geoserver.wms.responses.map.kml package.

Cheers
Andrea Aime

PS: the jira issue is here:
http://jira.codehaus.org/browse/GEOS-773

Andrea Aime wrote:

Sasha ha scritto:

Andrea Aime wrote:

Sasha ha scritto:

I've checked the same layer in the geoserver demo section and it opens only one connection for all zoom levels.
So your guess is probably correct, any idea of how I should solve this problem or where I can find more documentation related to this problem.

The only way is to check Geoserver KML producer and try to locate
the unclosed reader. I've opened a Jira issue about it, if you're willing to hack and find the problem, that would be much appreciated :slight_smile:
KML generator is in the wms module, org.vfny.geoserver.wms.responses.map.kml package.

Actually for Google _maps_ we don't use the KML producer. We use the normal WMS producer

It's odd though, if you're letting it load all the way then it should be able to take back the connections, perhaps google maps is also loading things off screen?

I'd like to investigate it more, but in the meantime, Sasha, the best thing you can do is add caching. See: http://docs.codehaus.org/display/GEOSDOC/Caching

Google maps can take advantage of it since it makes the same request each time. You may have to go a bit slow in your first zooms, but after that the caching will kick in for each subsequent view.

best regards,

Chris

Cheers
Andrea Aime

PS: the jira issue is here:
http://jira.codehaus.org/browse/GEOS-773

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:1003,45406408241676309890654!

--
Chris Holmes
The Open Planning Project
http://topp.openplans.org

Chris Holmes ha scritto:

Actually for Google _maps_ we don't use the KML producer. We use the normal WMS producer

Hmm... Sasha, was that google maps or Google Earth? I don't see how
a google map WMS request is different that one done directly with the browser...

It's odd though, if you're letting it load all the way then it should be able to take back the connections, perhaps google maps is also loading things off screen?

I'd like to investigate it more, but in the meantime, Sasha, the best thing you can do is add caching. See: http://docs.codehaus.org/display/GEOSDOC/Caching

Wondering, would that cache KML too?

Cheers
Andrea

Andrea Aime wrote:

Chris Holmes ha scritto:

Actually for Google _maps_ we don't use the KML producer. We use the normal WMS producer

Hmm... Sasha, was that google maps or Google Earth? I don't see how
a google map WMS request is different that one done directly with the browser...

It's not. Except for the high number of requests - one for each tile. So it puts more load on GeoServer than a non-tiling wms web client

It's odd though, if you're letting it load all the way then it should be able to take back the connections, perhaps google maps is also loading things off screen?

I'd like to investigate it more, but in the meantime, Sasha, the best thing you can do is add caching. See: http://docs.codehaus.org/display/GEOSDOC/Caching

Wondering, would that cache KML too?

Yes it will, but the problem is we don't yet implement tiling for KML. The way Google Earth currently works is it just sends a request whenever the map stops zooming, so unless you happen to stop on exactly the same area that someone has viewed before you'll never get the advantages of the caching. The next big thing to do for our google earth support is 'super overlays' - http://earth.google.com/kml/kml_21tutorial.html#superoverlays Then we can divide the world up in to tiles and take advantage of caching.

Chris

Cheers
Andrea

!DSPAM:1003,4540c659282401219810056!

--
Chris Holmes
The Open Planning Project
http://topp.openplans.org

Hi,

I've already implemented caching and I'm using Google Maps.
This is the link to my web application if you want to see how it behaves.
http://occamtechnology.com.au/mapApp/map
If it stops responding it means that server run out of available connections.

Thanks for efforts, I really appreciate your time guys.

Sasha

Chris Holmes wrote:

Andrea Aime wrote:

Sasha ha scritto:

Andrea Aime wrote:

Sasha ha scritto:

I've checked the same layer in the geoserver demo section and it opens only one connection for all zoom levels.
So your guess is probably correct, any idea of how I should solve this problem or where I can find more documentation related to this problem.

The only way is to check Geoserver KML producer and try to locate
the unclosed reader. I've opened a Jira issue about it, if you're willing to hack and find the problem, that would be much appreciated :slight_smile:
KML generator is in the wms module, org.vfny.geoserver.wms.responses.map.kml package.

Actually for Google _maps_ we don't use the KML producer. We use the normal WMS producer

It's odd though, if you're letting it load all the way then it should be able to take back the connections, perhaps google maps is also loading things off screen?

I'd like to investigate it more, but in the meantime, Sasha, the best thing you can do is add caching. See: http://docs.codehaus.org/display/GEOSDOC/Caching

Google maps can take advantage of it since it makes the same request each time. You may have to go a bit slow in your first zooms, but after that the caching will kick in for each subsequent view.

best regards,

Chris

Cheers
Andrea Aime

PS: the jira issue is here:
http://jira.codehaus.org/browse/GEOS-773

-------------------------------------------------------------------------

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:1003,45406408241676309890654!

Sasha wrote:

Hi,

I've already implemented caching and I'm using Google Maps.
This is the link to my web application if you want to see how it behaves.
http://occamtechnology.com.au/mapApp/map
If it stops responding it means that server run out of available connections.

I got:

  Not Found
The requested URL /mapApp/map was not found on this server.

Apache/1.3.37 Server at occamtechnology.com.au Port 80

**
at that url.

If caching is implemented properly then there should be _no_ connections made after the the first time it's viewed.

Also what version of GeoServer are you using?

Chris

Thanks for efforts, I really appreciate your time guys.

Sasha

Chris Holmes wrote:

Andrea Aime wrote:

Sasha ha scritto:

Andrea Aime wrote:

Sasha ha scritto:

I've checked the same layer in the geoserver demo section and it opens only one connection for all zoom levels.
So your guess is probably correct, any idea of how I should solve this problem or where I can find more documentation related to this problem.

The only way is to check Geoserver KML producer and try to locate
the unclosed reader. I've opened a Jira issue about it, if you're willing to hack and find the problem, that would be much appreciated :slight_smile:
KML generator is in the wms module, org.vfny.geoserver.wms.responses.map.kml package.

Actually for Google _maps_ we don't use the KML producer. We use the normal WMS producer

It's odd though, if you're letting it load all the way then it should be able to take back the connections, perhaps google maps is also loading things off screen?

I'd like to investigate it more, but in the meantime, Sasha, the best thing you can do is add caching. See: http://docs.codehaus.org/display/GEOSDOC/Caching

Google maps can take advantage of it since it makes the same request each time. You may have to go a bit slow in your first zooms, but after that the caching will kick in for each subsequent view.

best regards,

Chris

Cheers
Andrea Aime

PS: the jira issue is here:
http://jira.codehaus.org/browse/GEOS-773

-------------------------------------------------------------------------

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:1003,45417ecd79711995013331!

--
Chris Holmes
The Open Planning Project
http://topp.openplans.org

Sasha wrote:

Chris Holmes wrote:

Sasha wrote:

Hi,

I've already implemented caching and I'm using Google Maps.
This is the link to my web application if you want to see how it behaves.
http://occamtechnology.com.au/mapApp/map
If it stops responding it means that server run out of available connections.

I got:

Not Found
The requested URL /mapApp/map was not found on this server.

Apache/1.3.37 Server at occamtechnology.com.au Port 80

**
at that url.

Link should be OK, just in the case try this one
http://occamtechnology.com.au/mapApp/maps/index.php

That worked.

If caching is implemented properly then there should be _no_ connections made after the the first time it's viewed.

Also what version of GeoServer are you using?

   geoserver 1.4.0
   If you want you can access it here
   http://150.101.230.62:8180/geoserver/welcome.do
   password and user name are default

You mean 1.4.0-RC1? Or an earlier 1.4.0?

I can zoom in fine, as long as I wait for everything to load. Your application is a pretty big google map, so that means it's asking for about 20 geoserver requests at once. If these aren't done at each zoom level, then they will just pile up.

Then I played with the application for longer, and it took me awhile to get it to finally fall over, even when doing a bunch of zooms. I think the caching appears to be working... Are you still seeing connections made when you're only zooming in to areas that have been seen before?

You're viewing a large areas all at once, so making tons of requests, especially when zooming in a bunch. To get it working without piling up connections the best thing to do is to pre-cache the map. You can do this by hand a bit by just looking at most of the map, but to do really well you'll need a pre-caching walker. Unfortunately I don't know of any for google maps, but I could give you hints on writing one. You could write a naive one by just using the current code and automatically move and zoom the map around with some javascript.

That said, I've seen this problem before, and there are fixes that could be done to GeoTools to use a better connection pooling library so it maybe won't completely fall over, you could limit the max connections in java and have it wait until an extra becomes available.

But I do think your server is pooling connections properly, it's just it's building up a ton of requests with google maps.

best regards,

Chris

Chris

Thanks for efforts, I really appreciate your time guys.

Sasha

Chris Holmes wrote:

Andrea Aime wrote:

Sasha ha scritto:

Andrea Aime wrote:

Sasha ha scritto:

I've checked the same layer in the geoserver demo section and it opens only one connection for all zoom levels.
So your guess is probably correct, any idea of how I should solve this problem or where I can find more documentation related to this problem.

The only way is to check Geoserver KML producer and try to locate
the unclosed reader. I've opened a Jira issue about it, if you're willing to hack and find the problem, that would be much appreciated :slight_smile:
KML generator is in the wms module, org.vfny.geoserver.wms.responses.map.kml package.

Actually for Google _maps_ we don't use the KML producer. We use the normal WMS producer

It's odd though, if you're letting it load all the way then it should be able to take back the connections, perhaps google maps is also loading things off screen?

I'd like to investigate it more, but in the meantime, Sasha, the best thing you can do is add caching. See: http://docs.codehaus.org/display/GEOSDOC/Caching

Google maps can take advantage of it since it makes the same request each time. You may have to go a bit slow in your first zooms, but after that the caching will kick in for each subsequent view.

best regards,

Chris

Cheers
Andrea Aime

PS: the jira issue is here:
http://jira.codehaus.org/browse/GEOS-773

-------------------------------------------------------------------------

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:1003,454181ee81036309890654!

--
Chris Holmes
The Open Planning Project
http://topp.openplans.org

Chris Holmes wrote:

Sasha wrote:

Chris Holmes wrote:

Sasha wrote:

Hi,

I've already implemented caching and I'm using Google Maps.
This is the link to my web application if you want to see how it behaves.
http://occamtechnology.com.au/mapApp/map
If it stops responding it means that server run out of available connections.

I got:

Not Found
The requested URL /mapApp/map was not found on this server.

Apache/1.3.37 Server at occamtechnology.com.au Port 80

**
at that url.

Link should be OK, just in the case try this one
http://occamtechnology.com.au/mapApp/maps/index.php

That worked.

If caching is implemented properly then there should be _no_ connections made after the the first time it's viewed.

Also what version of GeoServer are you using?

You mean 1.4.0-RC1? Or an earlier 1.4.0?

    sorry, it is 1.4.0-RC1

I can zoom in fine, as long as I wait for everything to load. Your application is a pretty big google map, so that means it's asking for about 20 geoserver requests at once. If these aren't done at each zoom level, then they will just pile up.

    I'll try to make map smaller and test results again.

Then I played with the application for longer, and it took me awhile to get it to finally fall over, even when doing a bunch of zooms. I think the caching appears to be working... Are you still seeing connections made when you're only zooming in to areas that have been seen before?

    No connections are made only when zooming to the areas that I haven't seen before.

You're viewing a large areas all at once, so making tons of requests, especially when zooming in a bunch. To get it working without piling up connections the best thing to do is to pre-cache the map. You can do this by hand a bit by just looking at most of the map, but to do really well you'll need a pre-caching walker. Unfortunately I don't know of any for google maps, but I could give you hints on writing one. You could write a naive one by just using the current code and automatically move and zoom the map around with some javascript.

That said, I've seen this problem before, and there are fixes that could be done to GeoTools to use a better connection pooling library so it maybe won't completely fall over, you could limit the max connections in java and have it wait until an extra becomes available.

But I do think your server is pooling connections properly, it's just it's building up a ton of requests with google maps.
best regards,

Thanks Chris, it was really useful. Now it is up to me to do more research. I will post my findings/results here.

Chris

Chris

Thanks for efforts, I really appreciate your time guys.

Sasha

Chris Holmes wrote:

Andrea Aime wrote:

Sasha ha scritto:

Andrea Aime wrote:

Sasha ha scritto:

I've checked the same layer in the geoserver demo section and it opens only one connection for all zoom levels.
So your guess is probably correct, any idea of how I should solve this problem or where I can find more documentation related to this problem.

The only way is to check Geoserver KML producer and try to locate
the unclosed reader. I've opened a Jira issue about it, if you're willing to hack and find the problem, that would be much appreciated :slight_smile:
KML generator is in the wms module, org.vfny.geoserver.wms.responses.map.kml package.

Actually for Google _maps_ we don't use the KML producer. We use the normal WMS producer

It's odd though, if you're letting it load all the way then it should be able to take back the connections, perhaps google maps is also loading things off screen?

I'd like to investigate it more, but in the meantime, Sasha, the best thing you can do is add caching. See: http://docs.codehaus.org/display/GEOSDOC/Caching

Google maps can take advantage of it since it makes the same request each time. You may have to go a bit slow in your first zooms, but after that the caching will kick in for each subsequent view.

best regards,

Chris

Cheers
Andrea Aime

PS: the jira issue is here:
http://jira.codehaus.org/browse/GEOS-773

-------------------------------------------------------------------------

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:1003,454181ee81036309890654!

Hi,

Could someone point me to the part of code that is responsable for closing/opening Postgres connections (for WMS layer display).
I've managed to stabilase my application, but still all opened connections are never closed, except when I restart geoserver or database.

Thanks a lot,
Sasa

P.S. I'm using 1.4.0-RC2 and 1.3.4

Chris Holmes wrote:

Sasha wrote:

Chris Holmes wrote:

Sasha wrote:

Hi,

I've already implemented caching and I'm using Google Maps.
This is the link to my web application if you want to see how it behaves.
http://occamtechnology.com.au/mapApp/map
If it stops responding it means that server run out of available connections.

I got:

Not Found
The requested URL /mapApp/map was not found on this server.

Apache/1.3.37 Server at occamtechnology.com.au Port 80

**
at that url.

Link should be OK, just in the case try this one
http://occamtechnology.com.au/mapApp/maps/index.php

That worked.

If caching is implemented properly then there should be _no_ connections made after the the first time it's viewed.

Also what version of GeoServer are you using?

   geoserver 1.4.0
   If you want you can access it here
   http://150.101.230.62:8180/geoserver/welcome.do
   password and user name are default

You mean 1.4.0-RC1? Or an earlier 1.4.0?

I can zoom in fine, as long as I wait for everything to load. Your application is a pretty big google map, so that means it's asking for about 20 geoserver requests at once. If these aren't done at each zoom level, then they will just pile up.

Then I played with the application for longer, and it took me awhile to get it to finally fall over, even when doing a bunch of zooms. I think the caching appears to be working... Are you still seeing connections made when you're only zooming in to areas that have been seen before?

You're viewing a large areas all at once, so making tons of requests, especially when zooming in a bunch. To get it working without piling up connections the best thing to do is to pre-cache the map. You can do this by hand a bit by just looking at most of the map, but to do really well you'll need a pre-caching walker. Unfortunately I don't know of any for google maps, but I could give you hints on writing one. You could write a naive one by just using the current code and automatically move and zoom the map around with some javascript.

That said, I've seen this problem before, and there are fixes that could be done to GeoTools to use a better connection pooling library so it maybe won't completely fall over, you could limit the max connections in java and have it wait until an extra becomes available.

But I do think your server is pooling connections properly, it's just it's building up a ton of requests with google maps.

best regards,

Chris

Chris

Thanks for efforts, I really appreciate your time guys.

Sasha

Chris Holmes wrote:

Andrea Aime wrote:

Sasha ha scritto:

Andrea Aime wrote:

Sasha ha scritto:

I've checked the same layer in the geoserver demo section and it opens only one connection for all zoom levels.
So your guess is probably correct, any idea of how I should solve this problem or where I can find more documentation related to this problem.

The only way is to check Geoserver KML producer and try to locate
the unclosed reader. I've opened a Jira issue about it, if you're willing to hack and find the problem, that would be much appreciated :slight_smile:
KML generator is in the wms module, org.vfny.geoserver.wms.responses.map.kml package.

Actually for Google _maps_ we don't use the KML producer. We use the normal WMS producer

It's odd though, if you're letting it load all the way then it should be able to take back the connections, perhaps google maps is also loading things off screen?

I'd like to investigate it more, but in the meantime, Sasha, the best thing you can do is add caching. See: http://docs.codehaus.org/display/GEOSDOC/Caching

Google maps can take advantage of it since it makes the same request each time. You may have to go a bit slow in your first zooms, but after that the caching will kick in for each subsequent view.

best regards,

Chris

Cheers
Andrea Aime

PS: the jira issue is here:
http://jira.codehaus.org/browse/GEOS-773

-------------------------------------------------------------------------

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:1003,454181ee81036309890654!

That bug should have been fixed already. I doubt it is 1.4 that is leaving the connections open, it might be the 1.3.4 instance you have running.
The postgis code is in geotools, under the plugin/postgis module.

Brent Owens
(The Open Planning Project)

Sasha wrote:

Hi,

Could someone point me to the part of code that is responsable for closing/opening Postgres connections (for WMS layer display).
I've managed to stabilase my application, but still all opened connections are never closed, except when I restart geoserver or database.

Thanks a lot,
Sasa

P.S. I'm using 1.4.0-RC2 and 1.3.4

Chris Holmes wrote:
  

Sasha wrote:
    

Chris Holmes wrote:
      

Sasha wrote:
        

Hi,

I've already implemented caching and I'm using Google Maps.
This is the link to my web application if you want to see how it behaves.
http://occamtechnology.com.au/mapApp/map
If it stops responding it means that server run out of available connections.
          

I got:

Not Found
The requested URL /mapApp/map was not found on this server.

Apache/1.3.37 Server at occamtechnology.com.au Port 80

**
at that url.

Link should be OK, just in the case try this one
http://occamtechnology.com.au/mapApp/maps/index.php
      

That worked.

If caching is implemented properly then there should be _no_ connections made after the the first time it's viewed.

Also what version of GeoServer are you using?

   geoserver 1.4.0
   If you want you can access it here
   http://150.101.230.62:8180/geoserver/welcome.do
   password and user name are default
      

You mean 1.4.0-RC1? Or an earlier 1.4.0?

I can zoom in fine, as long as I wait for everything to load. Your application is a pretty big google map, so that means it's asking for about 20 geoserver requests at once. If these aren't done at each zoom level, then they will just pile up.

Then I played with the application for longer, and it took me awhile to get it to finally fall over, even when doing a bunch of zooms. I think the caching appears to be working... Are you still seeing connections made when you're only zooming in to areas that have been seen before?

You're viewing a large areas all at once, so making tons of requests, especially when zooming in a bunch. To get it working without piling up connections the best thing to do is to pre-cache the map. You can do this by hand a bit by just looking at most of the map, but to do really well you'll need a pre-caching walker. Unfortunately I don't know of any for google maps, but I could give you hints on writing one. You could write a naive one by just using the current code and automatically move and zoom the map around with some javascript.

That said, I've seen this problem before, and there are fixes that could be done to GeoTools to use a better connection pooling library so it maybe won't completely fall over, you could limit the max connections in java and have it wait until an extra becomes available.

But I do think your server is pooling connections properly, it's just it's building up a ton of requests with google maps.

best regards,

Chris

Chris

Thanks for efforts, I really appreciate your time guys.

Sasha

Chris Holmes wrote:
          

Andrea Aime wrote:
            

Sasha ha scritto:
              

Andrea Aime wrote:
                

Sasha ha scritto:
                  

I've checked the same layer in the geoserver demo section and it opens only one connection for all zoom levels.
So your guess is probably correct, any idea of how I should solve this problem or where I can find more documentation related to this problem.
                

The only way is to check Geoserver KML producer and try to locate
the unclosed reader. I've opened a Jira issue about it, if you're willing to hack and find the problem, that would be much appreciated :slight_smile:
KML generator is in the wms module, org.vfny.geoserver.wms.responses.map.kml package.
              Actually for Google _maps_ we don't use the KML producer. We use the normal WMS producer

It's odd though, if you're letting it load all the way then it should be able to take back the connections, perhaps google maps is also loading things off screen?

I'd like to investigate it more, but in the meantime, Sasha, the best thing you can do is add caching. See: http://docs.codehaus.org/display/GEOSDOC/Caching

Google maps can take advantage of it since it makes the same request each time. You may have to go a bit slow in your first zooms, but after that the caching will kick in for each subsequent view.

best regards,

Chris

Cheers
Andrea Aime

PS: the jira issue is here:
http://jira.codehaus.org/browse/GEOS-773

-------------------------------------------------------------------------

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:1003,454181ee81036309890654!

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Thank for reply.
Unfortunately, connection problem is still there (even in 1.4) , but only when I display WMS layers on Google Maps (in Mapbuilder all connections are closed after layer has been displayed).
I'm not expert in this area, but I assume that opened connections are caused by Google Map then . Am I right? Or something else?
Help will be greatly appreciated because I'm struggling with this for almost 2 months and I'm not making any progress.

Cheers
Sasa

Brent Owens wrote:

That bug should have been fixed already. I doubt it is 1.4 that is leaving the connections open, it might be the 1.3.4 instance you have running.
The postgis code is in geotools, under the plugin/postgis module.

Brent Owens
(The Open Planning Project)

I've just discovered that even Mapbuilder leaves connections open, the only difference is that it opens only one connection while Google Maps opens more then one connection.

sasha 55570 0.0 0.2 10868 4516 ?? Ss 4:11PM 0:00.20 /usr/local/bin/postmaster -D ./d$
sasha 55571 0.0 0.2 7480 4488 ?? S 4:11PM 0:00.00 postmaster: stats buffer process$
sasha 55573 0.0 0.2 6500 4428 ?? S 4:11PM 0:00.00 postmaster: stats collector proc$
*sasha 55576 0.0 0.5 13948 10324 ?? S 4:13PM 0:01.19 postmaster: sasha geodata 150.10$
sasha 55577 0.0 0.5 13948 10324 ?? S 4:13PM 0:01.12 postmaster: sasha geodata 150.10$*
sasha 63534 0.0 0.5 14128 9780 ?? I 12:29PM 0:00.20 postmaster: sasha geodata 150.10$
sasha 63552 0.0 0.5 14220 9764 ?? I 12:33PM 0:00.25 postmaster: sasha geodata 150.10$
sasha 63623 0.0 0.4 13948 9224 ?? S 12:54PM 0:00.11 postmaster: sasha geodata 150.10$
sasha 63624 0.0 0.4 13948 9224 ?? S 12:54PM 0:00.08 postmaster: sasha geodata 150.10$
sasha 63625 0.0 0.4 13948 9224 ?? S 12:54PM 0:00.07 postmaster: sasha geodata 150.10$
sasha 63626 0.0 0.4 13948 9224 ?? S 12:54PM 0:00.08 postmaster: sasha geodata 150.10$
root 63628 0.0 0.0 600 480 p0 R+ 12:54PM 0:00.00 grep postmaster

Bold lines are connections opened yesterday around 4pm.

Cheers,
Sasa

Sasha wrote:

Thank for reply.
Unfortunately, connection problem is still there (even in 1.4) , but only when I display WMS layers on Google Maps (in Mapbuilder all connections are closed after layer has been displayed).
I'm not expert in this area, but I assume that opened connections are caused by Google Map then . Am I right? Or something else?
Help will be greatly appreciated because I'm struggling with this for almost 2 months and I'm not making any progress.

Cheers
Sasa

Brent Owens wrote:

That bug should have been fixed already. I doubt it is 1.4 that is leaving the connections open, it might be the 1.3.4 instance you have running.
The postgis code is in geotools, under the plugin/postgis module.

Brent Owens
(The Open Planning Project)

Thanks Sasha
We will look into it some more.

I know there are some issues with Google Maps opening up several connections, one for each tile. But they should definitely be closed once the request is finished.
I will reopen this bug.

Brent Owens
(The Open Planning Project)

Sasha wrote:

I've just discovered that even Mapbuilder leaves connections open, the only difference is that it opens only one connection while Google Maps opens more then one connection.

sasha 55570 0.0 0.2 10868 4516 ?? Ss 4:11PM 0:00.20 /usr/local/bin/postmaster -D ./d$
sasha 55571 0.0 0.2 7480 4488 ?? S 4:11PM 0:00.00 postmaster: stats buffer process$
sasha 55573 0.0 0.2 6500 4428 ?? S 4:11PM 0:00.00 postmaster: stats collector proc$
*sasha 55576 0.0 0.5 13948 10324 ?? S 4:13PM 0:01.19 postmaster: sasha geodata 150.10$
sasha 55577 0.0 0.5 13948 10324 ?? S 4:13PM 0:01.12 postmaster: sasha geodata 150.10$*
sasha 63534 0.0 0.5 14128 9780 ?? I 12:29PM 0:00.20 postmaster: sasha geodata 150.10$
sasha 63552 0.0 0.5 14220 9764 ?? I 12:33PM 0:00.25 postmaster: sasha geodata 150.10$
sasha 63623 0.0 0.4 13948 9224 ?? S 12:54PM 0:00.11 postmaster: sasha geodata 150.10$
sasha 63624 0.0 0.4 13948 9224 ?? S 12:54PM 0:00.08 postmaster: sasha geodata 150.10$
sasha 63625 0.0 0.4 13948 9224 ?? S 12:54PM 0:00.07 postmaster: sasha geodata 150.10$
sasha 63626 0.0 0.4 13948 9224 ?? S 12:54PM 0:00.08 postmaster: sasha geodata 150.10$
root 63628 0.0 0.0 600 480 p0 R+ 12:54PM 0:00.00 grep postmaster

Bold lines are connections opened yesterday around 4pm.

Cheers,
Sasa

Sasha wrote:

Thank for reply.
Unfortunately, connection problem is still there (even in 1.4) , but only when I display WMS layers on Google Maps (in Mapbuilder all connections are closed after layer has been displayed).
I'm not expert in this area, but I assume that opened connections are caused by Google Map then . Am I right? Or something else?
Help will be greatly appreciated because I'm struggling with this for almost 2 months and I'm not making any progress.

Cheers
Sasa

Brent Owens wrote:

That bug should have been fixed already. I doubt it is 1.4 that is leaving the connections open, it might be the 1.3.4 instance you have running.
The postgis code is in geotools, under the plugin/postgis module.

Brent Owens
(The Open Planning Project)

Brent Owens ha scritto:

Thanks Sasha
We will look into it some more.

I know there are some issues with Google Maps opening up several connections, one for each tile. But they should definitely be closed once the request is finished.

Oh the horror, no! Connections should not be closed once request is
finished, because opening and closing connections are extremely expensive operations.

The problem is, a decent connection pool library allows for setting
minimum and maximum number of pooled (kept open) connections, whilst
apparently the geotools one does not... that's the source
of problems, apparently the class does not close connections after a
certain inactivity delay as a normal connection pool library does.

Always having 2-10 connections open to fastly serve request is ok,
having 20+ is obviosly not unless you have to serve tons of
concurrent requests (usually customers in companys I've worked with
limited our apps to use no more than 6-10 connections).

Cheers
Andrea

Ah, I forgot about the connection pool.
Ignore my previous message =)

But what it sounds like is that the connections are left on indefinitely.

Brent Owens
(The Open Planning Project)

Andrea Aime wrote:

Brent Owens ha scritto:

Thanks Sasha
We will look into it some more.

I know there are some issues with Google Maps opening up several connections, one for each tile. But they should definitely be closed once the request is finished.

Oh the horror, no! Connections should not be closed once request is
finished, because opening and closing connections are extremely expensive operations.

The problem is, a decent connection pool library allows for setting
minimum and maximum number of pooled (kept open) connections, whilst
apparently the geotools one does not... that's the source
of problems, apparently the class does not close connections after a
certain inactivity delay as a normal connection pool library does.

Always having 2-10 connections open to fastly serve request is ok,
having 20+ is obviosly not unless you have to serve tons of
concurrent requests (usually customers in companys I've worked with
limited our apps to use no more than 6-10 connections).

Cheers
Andrea

Sasha ha scritto:

I've just discovered that even Mapbuilder leaves connections open, the only difference is that it opens only one connection while Google Maps opens more then one connection.

Oh well,
I took the time to look into geotools connection pooling and I don't
like what I've found out. Basically it seems geotools uses
the connection pools included sometimes in the drivers, and wraps
them with another pool... all in all it's pretty scary.

All I can tell you if you want a quick solution is to go into
the postgis codebase and replace the connection pooling code
with one based on c3p0 or DBCP... I'm going to complain and
take action on the geotools mailing list, but I think
we won't see the changes for any 1.4.0 code because geotools
2.2.x is at it's end of life and Geoserver is going to
start using 2.3.x as well...

Cheers
Andrea Aime