[Geoserver-users] KML scale problems again (GEOS-1368)

Hey guys,

Notice that GEOS-1368 has been closed but it looks like its not completely resolved. Shall i clone the issue or can it be re-opened?

This is still not working with 1.6RC1. I am hoping its something small that i can patch?

"to reproduce:

1. Grab a rather large dataset, tiger roads would suffice
2. Configure an appropriate style with a min/max scale denomination (mine is set to 1:20k)
3. Request the appropriate WMS call for the KML using the "large" layer and set bbox to -180,-90,180,90
4. Watch server resources closely. In my situation (using tomcat or jetty), my cpu flatlines at 100% for approximately 5 minutes before Geoserver finally returns no features

I have not done further debugging but clearly there is a problem in the way Geoserver calc's whether to retrieve the data or not. In the above situation i can see a lot of

"Failure tranforming feature to KML:featuretype.fid"

One would have expected no features to be accessed when outside the defined scales.

This is pretty critical for KML if others can reproduce because Google Earth & Maps always requests the full world extents when loading networklink's even if a LookAt is defined. Currently this behaviour puts Geoserver out of action every time someone goes to connect using a Google client."

--
Chris Tweedie

Yeah, that's pretty bad.

I would say this is a new bug though, since the scale denominator is fixed, since you say it successfully returns no features, right?

It's just that it's not doing the right optimization, it seems to be iterating through all the features. I believe we must have previously handled this right, and there's probably a regression. It sounds similar to another report that came in today.

Just to confirm a few things - this is with a PostGIS database? And at the scale you're requesting it would be a raster returned by geoserver, right? Like there's enough data it would be rendered by the server, not returned as KML?

I agree that this is a critical issue, we'll get it fixed.

Chris

Chris Tweedie wrote:

Hey guys,

Notice that GEOS-1368 has been closed but it looks like its not completely resolved. Shall i clone the issue or can it be re-opened?

This is still not working with 1.6RC1. I am hoping its something small that i can patch?

"to reproduce:

1. Grab a rather large dataset, tiger roads would suffice
2. Configure an appropriate style with a min/max scale denomination (mine is set to 1:20k)
3. Request the appropriate WMS call for the KML using the "large" layer and set bbox to -180,-90,180,90
4. Watch server resources closely. In my situation (using tomcat or jetty), my cpu flatlines at 100% for approximately 5 minutes before Geoserver finally returns no features

I have not done further debugging but clearly there is a problem in the way Geoserver calc's whether to retrieve the data or not. In the above situation i can see a lot of

"Failure tranforming feature to KML:featuretype.fid"

One would have expected no features to be accessed when outside the defined scales.

This is pretty critical for KML if others can reproduce because Google Earth & Maps always requests the full world extents when loading networklink's even if a LookAt is defined. Currently this behaviour puts Geoserver out of action every time someone goes to connect using a Google client."

--
Chris Tweedie

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:4005,47413db5312822143011171!

Chris, quick question. Which backend are you using? Shapefile? Postgis?
I think Andrea has tracked down the problem and it is related to
shapefile. If you are not using shapefile it must be a different problem.

-Justin

Chris Tweedie wrote:

Hey guys,

Notice that GEOS-1368 has been closed but it looks like its not completely resolved. Shall i clone the issue or can it be re-opened?

This is still not working with 1.6RC1. I am hoping its something small that i can patch?

"to reproduce:

1. Grab a rather large dataset, tiger roads would suffice
2. Configure an appropriate style with a min/max scale denomination (mine is set to 1:20k)
3. Request the appropriate WMS call for the KML using the "large" layer and set bbox to -180,-90,180,90
4. Watch server resources closely. In my situation (using tomcat or jetty), my cpu flatlines at 100% for approximately 5 minutes before Geoserver finally returns no features

I have not done further debugging but clearly there is a problem in the way Geoserver calc's whether to retrieve the data or not. In the above situation i can see a lot of

"Failure tranforming feature to KML:featuretype.fid"

One would have expected no features to be accessed when outside the defined scales.

This is pretty critical for KML if others can reproduce because Google Earth & Maps always requests the full world extents when loading networklink's even if a LookAt is defined. Currently this behaviour puts Geoserver out of action every time someone goes to connect using a Google client."

--
Chris Tweedie

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:4007,47413db0312841015089218!

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org

Justin Deoliveira ha scritto:

Chris, quick question. Which backend are you using? Shapefile? Postgis?
I think Andrea has tracked down the problem and it is related to
shapefile. If you are not using shapefile it must be a different problem.

Hem, I meant the shapefile renderer. So it's 100% sure it cannot be the same bug, thought it might be a similar logic problem appearing in two
different code bases.
Cheers
Andrea

Chris Holmes ha scritto:

Yeah, that's pretty bad.

I would say this is a new bug though, since the scale denominator is fixed, since you say it successfully returns no features, right?

It's just that it's not doing the right optimization, it seems to be iterating through all the features. I believe we must have previously handled this right, and there's probably a regression. It sounds similar to another report that came in today.

Just to confirm a few things - this is with a PostGIS database? And at the scale you're requesting it would be a raster returned by geoserver, right? Like there's enough data it would be rendered by the server, not returned as KML?

I agree that this is a critical issue, we'll get it fixed.

As I reported in GEOS-1368, this is not a regression, kml encoding
simply does not have any check on the current scale and the
used filters, it first loads all of the features in the current bbox
regardless of the scale and the actual filter expressed in the sld, and
then applies the scale and filter rules to each loaded feature.

Filtering in the GE demos we usually do work only because we use
cql_filters, that are applied during the map context construction,
and not inside the renderer (the renderer gets feature sources that
are already filtered).

Fixing this issue means injecting new code in between rc's...
I almost have a patch that fixes the issue, but I'm uncomfortable
putting it inside like this. Justin, can you review it when it's done?
Cheers
Andrea

Fixing this issue means injecting new code in between rc's...
I almost have a patch that fixes the issue, but I'm uncomfortable
putting it inside like this. Justin, can you review it when it's done?

Where can I get the patch? I just checked the issue report and nothing
attached.

I share your concern about doing this now... but at the same time it is
a pretty serious bug... i am torn. I would hate to see this cause
another regression with our kml stuff.

Also. I am concerned that this will just be more duplication from the
normal renderer. These kinds of things really need to be factored out so
that every output format that does some sort of rendering can use it:
kml, svg, raster, all do a lot of the same operations yet all implement
alot of the operations in the rendering pipeline.

I have not yet to see the patch Andrea so I could be wrong here but I
assume that is more or less a copy of something from the normal renderer?

-Justin

Cheers
Andrea

!DSPAM:4007,474436b25086491211187!

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org

Justin Deoliveira ha scritto:

Fixing this issue means injecting new code in between rc's...
I almost have a patch that fixes the issue, but I'm uncomfortable
putting it inside like this. Justin, can you review it when it's done?

Where can I get the patch? I just checked the issue report and nothing
attached.

Patch incoming, just a few minutes :slight_smile:

I share your concern about doing this now... but at the same time it is
a pretty serious bug... i am torn. I would hate to see this cause
another regression with our kml stuff.

Also. I am concerned that this will just be more duplication from the
normal renderer. These kinds of things really need to be factored out so
that every output format that does some sort of rendering can use it:
kml, svg, raster, all do a lot of the same operations yet all implement
alot of the operations in the rendering pipeline.

Indeed, yet everyone is doing that in a slightly different manner.
Anyways, today we have enough use cases around (raster, svg, pdf, kml)
that indeed it could be possible to try and do a general renderer.
That would be at odds with requests like Jason Pickering "fully templated" kml output one thought...

I have not yet to see the patch Andrea so I could be wrong here but I
assume that is more or less a copy of something from the normal renderer?

More or less, the renderer shaves attributes off too, sets up rendering
buffers as part of the main control loop, and the logic is different down the pipe... long story short, we can devise a general renderer, but
that would not be an adaptation of the streaming renderer, but more a
rewrite of it.

Cheers
Andrea

Well I am not sure we need a full blown general renderer... maybe just a
set of rendering convenience methods that are clearly thought out.

I have not yet to see the patch Andrea so I could be wrong here but I
assume that is more or less a copy of something from the normal renderer?

More or less, the renderer shaves attributes off too, sets up rendering
buffers as part of the main control loop, and the logic is different
down the pipe... long story short, we can devise a general renderer, but
that would not be an adaptation of the streaming renderer, but more a
rewrite of it.

Cheers
Andrea

!DSPAM:4007,474439779701431913854!

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org