[Geoserver-users] KMSCORE parameter is misbehavin'

One of the FeatureTypes that I would like to serve to GoogleEarth has a
fairly large number of records (2000+). I have been trying to use the
KMSCORE parameter to speed things up when too many features are in view, but
it seems to return far more features than the documentation indicates. For
KMSCORE=10, I would expect to see an image returned (when viewing more than
4 features), but instead GeoServer returns a large number of features
(100-1000?) as individual placemarks in KML.

Additional notes:
I am using the ArcSDE 9.2 plugin.
I don't encounter this issue when I serve up the sample tiger:poi data with
the same KMSCORE values. This dataset behaves has described in the
documentation.

Here a the format of the KML file of the network links:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.1&quot;&gt;
<Folder>
  <name>V_Location</name>
  <visibility>0</visibility>
  <open>1</open>
  <NetworkLink>
    <name>V_Location - KMSCORE=0</name>
    <visibility>0</visibility>
    <Url>
    
<href>http://gisdev:8080/geoserver/wms?styles=&amp;Format=kmz&amp;KMSCORE=0&amp;request=GetMap&amp;layers=mtri:SDE_NRCS.DBO.V_LOCATION&amp;width=800&amp;height=375&amp;srs=EPSG:4326&amp;&lt;/href&gt;
      <viewRefreshMode>onRequest</viewRefreshMode>
    </Url>
  </NetworkLink>
  <NetworkLink>
    <name>V_Location - KMSCORE=10</name>
    <visibility>0</visibility>
    <Url>
    
<href>http://gisdev:8080/geoserver/wms?styles=&amp;Format=kmz&amp;KMSCORE=10&amp;request=GetMap&amp;layers=mtri:SDE_NRCS.DBO.V_LOCATION&amp;width=800&amp;height=375&amp;srs=EPSG:4326&amp;&lt;/href&gt;
      <viewRefreshMode>onRequest</viewRefreshMode>
    </Url>
  </NetworkLink>
</Folder>
</kml>

KMSCORE=0
http://www.nabble.com/file/p11421303/screenshot_kmscore_0.jpg

KMSCORE=10
http://www.nabble.com/file/p11421303/screenshot_kmscore_10.jpg

The logs look pretty normal...

INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.vfny.geoserver.wms.requests.GetMapKvpReader - Set KMScore: 10
INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.vfny.geoserver.servlets.AbstractService - handling request:
INFO | jvm 1 | 2007/07/03 17:24:14 | GetMap Request
INFO | jvm 1 | 2007/07/03 17:24:14 | version: 1.1.1
INFO | jvm 1 | 2007/07/03 17:24:14 | output format: kmz
INFO | jvm 1 | 2007/07/03 17:24:14 | width height: 375,800
INFO | jvm 1 | 2007/07/03 17:24:14 | bbox: Env[-87.71528922820525 :
-81.15297453390464, 40.84272190296385 : 45.25420697827713]
INFO | jvm 1 | 2007/07/03 17:24:14 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/03 17:24:14 | styles: measurement_location
INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.geotools.arcsde.data.ArcSDEDataStore - getCount
INFO | jvm 1 | 2007/07/03 17:24:47 | 10219611 [INFO]
org.geotools.arcsde.data.ArcSDEQuery - No FID attribute was contained in
your query. Appending the discovered one to the list of columns to be
fetched.
INFO | jvm 1 | 2007/07/03 17:24:47 | 10220142 [INFO]
org.vfny.geoserver.servlets.AbstractService - Service handled

INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.vfny.geoserver.wms.requests.GetMapKvpReader - Set KMScore: 0
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.vfny.geoserver.servlets.AbstractService - handling request:
INFO | jvm 1 | 2007/07/03 17:27:58 | GetMap Request
INFO | jvm 1 | 2007/07/03 17:27:58 | version: 1.1.1
INFO | jvm 1 | 2007/07/03 17:27:58 | output format: kmz
INFO | jvm 1 | 2007/07/03 17:27:58 | width height: 375,800
INFO | jvm 1 | 2007/07/03 17:27:58 | bbox: Env[-87.71528922820525 :
-81.15297453390464, 40.84272190296385 : 45.25420697827713]
INFO | jvm 1 | 2007/07/03 17:27:58 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/03 17:27:58 | styles: measurement_location
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.geotools.arcsde.data.ArcSDEDataStore - getCount
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410761 [INFO]
org.geotools.arcsde.data.ArcSDEQuery - No FID attribute was contained in
your query. Appending the discovered one to the list of columns to be
fetched.
INFO | jvm 1 | 2007/07/03 17:27:59 | 10411417 [INFO]
org.vfny.geoserver.servlets.AbstractService - Service handled
--
View this message in context: http://www.nabble.com/KMSCORE-parameter-is-misbehavin'-tf4021151.html#a11421303
Sent from the GeoServer - User mailing list archive at Nabble.com.

Hi Tyler,

I did a little testing of my own and noticed that the calculation was
indeed off due to integer truncation in the exponent... But I would
expect this to result in the opposite case, ie. GeoServer returning a
raster more often then it should.

My suspicion would be that the datastore (arcsde) is returning an
inaccurate count of features and is throwing off the calculation. We
never managed to get our arcsde test server up and running
unfortunately, so this one is hard for me to test hands on.

Perhaps Saul (our sde expert) can help us out on this one.

-Justin

Tyler Erickson wrote:

One of the FeatureTypes that I would like to serve to GoogleEarth has a
fairly large number of records (2000+). I have been trying to use the
KMSCORE parameter to speed things up when too many features are in view, but
it seems to return far more features than the documentation indicates. For
KMSCORE=10, I would expect to see an image returned (when viewing more than
4 features), but instead GeoServer returns a large number of features
(100-1000?) as individual placemarks in KML.

Additional notes:
I am using the ArcSDE 9.2 plugin.
I don't encounter this issue when I serve up the sample tiger:poi data with
the same KMSCORE values. This dataset behaves has described in the
documentation.

Here a the format of the KML file of the network links:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.1&quot;&gt;
<Folder>
  <name>V_Location</name>
  <visibility>0</visibility>
  <open>1</open>
  <NetworkLink>
    <name>V_Location - KMSCORE=0</name>
    <visibility>0</visibility>
    <Url>
    
<href>http://gisdev:8080/geoserver/wms?styles=&amp;Format=kmz&amp;KMSCORE=0&amp;request=GetMap&amp;layers=mtri:SDE_NRCS.DBO.V_LOCATION&amp;width=800&amp;height=375&amp;srs=EPSG:4326&amp;&lt;/href&gt;
      <viewRefreshMode>onRequest</viewRefreshMode>
    </Url>
  </NetworkLink>
  <NetworkLink>
    <name>V_Location - KMSCORE=10</name>
    <visibility>0</visibility>
    <Url>
    
<href>http://gisdev:8080/geoserver/wms?styles=&amp;Format=kmz&amp;KMSCORE=10&amp;request=GetMap&amp;layers=mtri:SDE_NRCS.DBO.V_LOCATION&amp;width=800&amp;height=375&amp;srs=EPSG:4326&amp;&lt;/href&gt;
      <viewRefreshMode>onRequest</viewRefreshMode>
    </Url>
  </NetworkLink>
</Folder>
</kml>

KMSCORE=0
http://www.nabble.com/file/p11421303/screenshot_kmscore_0.jpg

KMSCORE=10
http://www.nabble.com/file/p11421303/screenshot_kmscore_10.jpg

The logs look pretty normal...

INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.vfny.geoserver.wms.requests.GetMapKvpReader - Set KMScore: 10
INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.vfny.geoserver.servlets.AbstractService - handling request:
INFO | jvm 1 | 2007/07/03 17:24:14 | GetMap Request
INFO | jvm 1 | 2007/07/03 17:24:14 | version: 1.1.1
INFO | jvm 1 | 2007/07/03 17:24:14 | output format: kmz
INFO | jvm 1 | 2007/07/03 17:24:14 | width height: 375,800
INFO | jvm 1 | 2007/07/03 17:24:14 | bbox: Env[-87.71528922820525 :
-81.15297453390464, 40.84272190296385 : 45.25420697827713]
INFO | jvm 1 | 2007/07/03 17:24:14 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/03 17:24:14 | styles: measurement_location
INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.geotools.arcsde.data.ArcSDEDataStore - getCount
INFO | jvm 1 | 2007/07/03 17:24:47 | 10219611 [INFO]
org.geotools.arcsde.data.ArcSDEQuery - No FID attribute was contained in
your query. Appending the discovered one to the list of columns to be
fetched.
INFO | jvm 1 | 2007/07/03 17:24:47 | 10220142 [INFO]
org.vfny.geoserver.servlets.AbstractService - Service handled

INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.vfny.geoserver.wms.requests.GetMapKvpReader - Set KMScore: 0
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.vfny.geoserver.servlets.AbstractService - handling request:
INFO | jvm 1 | 2007/07/03 17:27:58 | GetMap Request
INFO | jvm 1 | 2007/07/03 17:27:58 | version: 1.1.1
INFO | jvm 1 | 2007/07/03 17:27:58 | output format: kmz
INFO | jvm 1 | 2007/07/03 17:27:58 | width height: 375,800
INFO | jvm 1 | 2007/07/03 17:27:58 | bbox: Env[-87.71528922820525 :
-81.15297453390464, 40.84272190296385 : 45.25420697827713]
INFO | jvm 1 | 2007/07/03 17:27:58 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/03 17:27:58 | styles: measurement_location
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.geotools.arcsde.data.ArcSDEDataStore - getCount
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410761 [INFO]
org.geotools.arcsde.data.ArcSDEQuery - No FID attribute was contained in
your query. Appending the discovered one to the list of columns to be
fetched.
INFO | jvm 1 | 2007/07/03 17:27:59 | 10411417 [INFO]
org.vfny.geoserver.servlets.AbstractService - Service handled

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org

Tyler, Justin,

The other week I ran into a problem where the ArcSDE 'quick count' method of trying to pre-determine the number of features in a particular result was returning inaccurate results.

I fiddled with the spatial index and unit size on the arcsde layer and by tweaking them correctly I was able to get the optimized results-count method to be correct. Is it possible, Tyler, that there's somethin funny about the ArcSDE data that you're looking at? Is there a proper spatial index on the data? Is the unit-size correct?

I'll try and take a look at this issue again later, perhaps putting in a check to make sure that the estimated returned feature count and the actual returned feature count match (to alert us to the fact that something's wrong when it *does* happen). This probably won't happen till late next week at the earliest, however.

--saul

________________________________

From: Justin Deoliveira [mailto:jdeolive@anonymised.com]
Sent: Tue 7/3/2007 6:37 PM
To: Tyler Erickson
Cc: geoserver-users@lists.sourceforge.net; Farber, Saul (ENV)
Subject: Re: [Geoserver-users] KMSCORE parameter is misbehavin'

Hi Tyler,

I did a little testing of my own and noticed that the calculation was
indeed off due to integer truncation in the exponent... But I would
expect this to result in the opposite case, ie. GeoServer returning a
raster more often then it should.

My suspicion would be that the datastore (arcsde) is returning an
inaccurate count of features and is throwing off the calculation. We
never managed to get our arcsde test server up and running
unfortunately, so this one is hard for me to test hands on.

Perhaps Saul (our sde expert) can help us out on this one.

-Justin

Tyler Erickson wrote:

One of the FeatureTypes that I would like to serve to GoogleEarth has a
fairly large number of records (2000+). I have been trying to use the
KMSCORE parameter to speed things up when too many features are in view, but
it seems to return far more features than the documentation indicates. For
KMSCORE=10, I would expect to see an image returned (when viewing more than
4 features), but instead GeoServer returns a large number of features
(100-1000?) as individual placemarks in KML.

Additional notes:
I am using the ArcSDE 9.2 plugin.
I don't encounter this issue when I serve up the sample tiger:poi data with
the same KMSCORE values. This dataset behaves has described in the
documentation.

Here a the format of the KML file of the network links:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.1&quot;&gt;
<Folder>
      <name>V_Location</name>
      <visibility>0</visibility>
      <open>1</open>
      <NetworkLink>
              <name>V_Location - KMSCORE=0</name>
              <visibility>0</visibility>
              <Url>
             
<href>http://gisdev:8080/geoserver/wms?styles=&Format=kmz&KMSCORE=0&request=GetMap&layers=mtri:SDE_NRCS.DBO.V_LOCATION&width=800&height=375&srs=EPSG:4326&&lt;/href&gt;
                      <viewRefreshMode>onRequest</viewRefreshMode>
              </Url>
      </NetworkLink>
      <NetworkLink>
              <name>V_Location - KMSCORE=10</name>
              <visibility>0</visibility>
              <Url>
             
<href>http://gisdev:8080/geoserver/wms?styles=&Format=kmz&KMSCORE=10&request=GetMap&layers=mtri:SDE_NRCS.DBO.V_LOCATION&width=800&height=375&srs=EPSG:4326&&lt;/href&gt;
                      <viewRefreshMode>onRequest</viewRefreshMode>
              </Url>
      </NetworkLink>
</Folder>
</kml>

KMSCORE=0
http://www.nabble.com/file/p11421303/screenshot_kmscore_0.jpg

KMSCORE=10
http://www.nabble.com/file/p11421303/screenshot_kmscore_10.jpg

The logs look pretty normal...

INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.vfny.geoserver.wms.requests.GetMapKvpReader - Set KMScore: 10
INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.vfny.geoserver.servlets.AbstractService - handling request:
INFO | jvm 1 | 2007/07/03 17:24:14 | GetMap Request
INFO | jvm 1 | 2007/07/03 17:24:14 | version: 1.1.1
INFO | jvm 1 | 2007/07/03 17:24:14 | output format: kmz
INFO | jvm 1 | 2007/07/03 17:24:14 | width height: 375,800
INFO | jvm 1 | 2007/07/03 17:24:14 | bbox: Env[-87.71528922820525 :
-81.15297453390464, 40.84272190296385 : 45.25420697827713]
INFO | jvm 1 | 2007/07/03 17:24:14 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/03 17:24:14 | styles: measurement_location
INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.geotools.arcsde.data.ArcSDEDataStore - getCount
INFO | jvm 1 | 2007/07/03 17:24:47 | 10219611 [INFO]
org.geotools.arcsde.data.ArcSDEQuery - No FID attribute was contained in
your query. Appending the discovered one to the list of columns to be
fetched.
INFO | jvm 1 | 2007/07/03 17:24:47 | 10220142 [INFO]
org.vfny.geoserver.servlets.AbstractService - Service handled

INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.vfny.geoserver.wms.requests.GetMapKvpReader - Set KMScore: 0
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.vfny.geoserver.servlets.AbstractService - handling request:
INFO | jvm 1 | 2007/07/03 17:27:58 | GetMap Request
INFO | jvm 1 | 2007/07/03 17:27:58 | version: 1.1.1
INFO | jvm 1 | 2007/07/03 17:27:58 | output format: kmz
INFO | jvm 1 | 2007/07/03 17:27:58 | width height: 375,800
INFO | jvm 1 | 2007/07/03 17:27:58 | bbox: Env[-87.71528922820525 :
-81.15297453390464, 40.84272190296385 : 45.25420697827713]
INFO | jvm 1 | 2007/07/03 17:27:58 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/03 17:27:58 | styles: measurement_location
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.geotools.arcsde.data.ArcSDEDataStore - getCount
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410761 [INFO]
org.geotools.arcsde.data.ArcSDEQuery - No FID attribute was contained in
your query. Appending the discovered one to the list of columns to be
fetched.
INFO | jvm 1 | 2007/07/03 17:27:59 | 10411417 [INFO]
org.vfny.geoserver.servlets.AbstractService - Service handled

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org/&gt;

Saul,

I think my spatial index parameters are reasonable, but any suggestions
would be appreciated.

I'm using unprojected coordinate system in decimal degrees (GCS_WGS_1984).
The data extent is the complete world (-180<x<180 and -90<y<90).
The geometry is a point feature class.

Below are the stats for ArcSDE's spatial index. I also tried a grid size of
1.0, but had the same KMSCORE issue.

- Tyler

C:\Documents and Settings\terickson>sdelayer -o si_stats -s sql1 -i 5153 -D
sde_
nrcs -u "MTRI\taericks" -l T_Location,SHAPE

ArcSDE 9.2 for SQL Server Build 1137 Mon Mar 5 15:52:01 2007
Layer Administration Utility
-----------------------------------------------------
Layer 84 Spatial Index Statistics:
Level 1, Grid Size 0.0001
|-------------------------------------------------------------------|
| Grid Records: 3877 |
| Feature Records: 3877 |
| Grids/Feature Ratio: 1.00 |
| Avg. Features per Grid: 1.55 |
| Max. Features per Grid: 63 |
| % of Features Wholly Inside 1 Grid: 100.00 |
|-------------------------------------------------------------------|
| Spatial Index Record Count By Group |
| Grids: <=4 >4 >10 >25 >50 >100 >250 >500 |
|---------- ------ ------ ------ ------ ------ ------ ------ ------ |
| Features: 3877 0 0 0 0 0 0 0 |
| % Total: 100% 0% 0% 0% 0% 0% 0% 0%|
|-------------------------------------------------------------------|

sfarber wrote:

Tyler, Justin,

The other week I ran into a problem where the ArcSDE 'quick count' method
of trying to pre-determine the number of features in a particular result
was returning inaccurate results.

I fiddled with the spatial index and unit size on the arcsde layer and by
tweaking them correctly I was able to get the optimized results-count
method to be correct. Is it possible, Tyler, that there's somethin funny
about the ArcSDE data that you're looking at? Is there a proper spatial
index on the data? Is the unit-size correct?

I'll try and take a look at this issue again later, perhaps putting in a
check to make sure that the estimated returned feature count and the
actual returned feature count match (to alert us to the fact that
something's wrong when it *does* happen). This probably won't happen till
late next week at the earliest, however.

--saul

________________________________

From: Justin Deoliveira [mailto:jdeolive@anonymised.com]
Sent: Tue 7/3/2007 6:37 PM
To: Tyler Erickson
Cc: geoserver-users@lists.sourceforge.net; Farber, Saul (ENV)
Subject: Re: [Geoserver-users] KMSCORE parameter is misbehavin'

Hi Tyler,

I did a little testing of my own and noticed that the calculation was
indeed off due to integer truncation in the exponent... But I would
expect this to result in the opposite case, ie. GeoServer returning a
raster more often then it should.

My suspicion would be that the datastore (arcsde) is returning an
inaccurate count of features and is throwing off the calculation. We
never managed to get our arcsde test server up and running
unfortunately, so this one is hard for me to test hands on.

Perhaps Saul (our sde expert) can help us out on this one.

-Justin

Tyler Erickson wrote:

One of the FeatureTypes that I would like to serve to GoogleEarth has a
fairly large number of records (2000+). I have been trying to use the
KMSCORE parameter to speed things up when too many features are in view,
but
it seems to return far more features than the documentation indicates.
For
KMSCORE=10, I would expect to see an image returned (when viewing more
than
4 features), but instead GeoServer returns a large number of features
(100-1000?) as individual placemarks in KML.

Additional notes:
I am using the ArcSDE 9.2 plugin.
I don't encounter this issue when I serve up the sample tiger:poi data
with
the same KMSCORE values. This dataset behaves has described in the
documentation.

Here a the format of the KML file of the network links:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.1&quot;&gt;
<Folder>
      <name>V_Location</name>
      <visibility>0</visibility>
      <open>1</open>
      <NetworkLink>
              <name>V_Location - KMSCORE=0</name>
              <visibility>0</visibility>
              <Url>
             
<href>http://gisdev:8080/geoserver/wms?styles=&Format=kmz&KMSCORE=0&request=GetMap&layers=mtri:SDE_NRCS.DBO.V_LOCATION&width=800&height=375&srs=EPSG:4326&&lt;/href&gt;
                      <viewRefreshMode>onRequest</viewRefreshMode>
              </Url>
      </NetworkLink>
      <NetworkLink>
              <name>V_Location - KMSCORE=10</name>
              <visibility>0</visibility>
              <Url>
             
<href>http://gisdev:8080/geoserver/wms?styles=&Format=kmz&KMSCORE=10&request=GetMap&layers=mtri:SDE_NRCS.DBO.V_LOCATION&width=800&height=375&srs=EPSG:4326&&lt;/href&gt;
                      <viewRefreshMode>onRequest</viewRefreshMode>
              </Url>
      </NetworkLink>
</Folder>
</kml>

KMSCORE=0
http://www.nabble.com/file/p11421303/screenshot_kmscore_0.jpg

KMSCORE=10
http://www.nabble.com/file/p11421303/screenshot_kmscore_10.jpg

The logs look pretty normal...

INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.vfny.geoserver.wms.requests.GetMapKvpReader - Set KMScore: 10
INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.vfny.geoserver.servlets.AbstractService - handling request:
INFO | jvm 1 | 2007/07/03 17:24:14 | GetMap Request
INFO | jvm 1 | 2007/07/03 17:24:14 | version: 1.1.1
INFO | jvm 1 | 2007/07/03 17:24:14 | output format: kmz
INFO | jvm 1 | 2007/07/03 17:24:14 | width height: 375,800
INFO | jvm 1 | 2007/07/03 17:24:14 | bbox: Env[-87.71528922820525 :
-81.15297453390464, 40.84272190296385 : 45.25420697827713]
INFO | jvm 1 | 2007/07/03 17:24:14 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/03 17:24:14 | styles: measurement_location
INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.geotools.arcsde.data.ArcSDEDataStore - getCount
INFO | jvm 1 | 2007/07/03 17:24:47 | 10219611 [INFO]
org.geotools.arcsde.data.ArcSDEQuery - No FID attribute was contained in
your query. Appending the discovered one to the list of columns to be
fetched.
INFO | jvm 1 | 2007/07/03 17:24:47 | 10220142 [INFO]
org.vfny.geoserver.servlets.AbstractService - Service handled

INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.vfny.geoserver.wms.requests.GetMapKvpReader - Set KMScore: 0
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.vfny.geoserver.servlets.AbstractService - handling request:
INFO | jvm 1 | 2007/07/03 17:27:58 | GetMap Request
INFO | jvm 1 | 2007/07/03 17:27:58 | version: 1.1.1
INFO | jvm 1 | 2007/07/03 17:27:58 | output format: kmz
INFO | jvm 1 | 2007/07/03 17:27:58 | width height: 375,800
INFO | jvm 1 | 2007/07/03 17:27:58 | bbox: Env[-87.71528922820525 :
-81.15297453390464, 40.84272190296385 : 45.25420697827713]
INFO | jvm 1 | 2007/07/03 17:27:58 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/03 17:27:58 | styles: measurement_location
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.geotools.arcsde.data.ArcSDEDataStore - getCount
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410761 [INFO]
org.geotools.arcsde.data.ArcSDEQuery - No FID attribute was contained in
your query. Appending the discovered one to the list of columns to be
fetched.
INFO | jvm 1 | 2007/07/03 17:27:59 | 10411417 [INFO]
org.vfny.geoserver.servlets.AbstractService - Service handled

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org/&gt;

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
View this message in context: http://www.nabble.com/KMSCORE-parameter-is-misbehavin'-tf4021151.html#a11510705
Sent from the GeoServer - User mailing list archive at Nabble.com.

I think they look pretty good to me too, but I can't claim to be an expert on SDE grid-size tuning! From what I understand, though, point features aren't particularly well-served by the si_stats option to sdelayer. I mean, every point is in exactly one grid cell, right? That's the definition of a *point*.

Anyway, I'm not totally sure what's up with your situation, but I *can* say that I've seen ArcSDE point layers tell geotools via the 'getCount()' method that a particular query captures 5 points, when in actuality it captures 6. After playing with the spatial index on that point layer, I got the results of the 'getCount()' method to agree with the results of a getReader()-for-loop calculation.

However, I'm not really very clear on why ArcSDE was returning bad counts via the algorithm coded into the 'getCount()' method. I don't think I understand ArcSDE's spatial query strategy well enough to figure it out.

The short summary might be this:

* I have once encountered an ArcSDE point layer which returned bogus feature count information via the 'getCount()' method
* Justin suggested that if a query was mis-representing it's size, this might cause an issue wit the KMSCORE parameter.
* You are using an ArcSDE point layer.

The way I fixed my ArcSDE point layer was to fiddle with the ArcSDE spatial index till it worked. Most of the time ArcSDE would reject my spatial index grid sizes, but when I eventually picked one that was on the small end of the valid range of grid sizes, my problem with bogus 'getCount()' information went away.

Let me know if you get it to work by fiddling with your spatial index.

But also make sure that the middle bullet above is appliccable! If there's something *else* wrong which is causing the KMSCORE parameter to misbehave, then this is likely not the right solution!

Good luck tyler,
--saul

________________________________

From: geoserver-users-bounces@lists.sourceforge.net on behalf of Tyler Erickson
Sent: Mon 7/9/2007 5:20 PM
To: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] KMSCORE parameter is misbehavin'

Saul,

I think my spatial index parameters are reasonable, but any suggestions
would be appreciated.

I'm using unprojected coordinate system in decimal degrees (GCS_WGS_1984).
The data extent is the complete world (-180<x<180 and -90<y<90).
The geometry is a point feature class.

Below are the stats for ArcSDE's spatial index. I also tried a grid size of
1.0, but had the same KMSCORE issue.

- Tyler

C:\Documents and Settings\terickson>sdelayer -o si_stats -s sql1 -i 5153 -D
sde_
nrcs -u "MTRI\taericks" -l T_Location,SHAPE

ArcSDE 9.2 for SQL Server Build 1137 Mon Mar 5 15:52:01 2007
Layer Administration Utility
-----------------------------------------------------
Layer 84 Spatial Index Statistics:
Level 1, Grid Size 0.0001
|-------------------------------------------------------------------|
| Grid Records: 3877 |
| Feature Records: 3877 |
| Grids/Feature Ratio: 1.00 |
| Avg. Features per Grid: 1.55 |
| Max. Features per Grid: 63 |
| % of Features Wholly Inside 1 Grid: 100.00 |
|-------------------------------------------------------------------|
| Spatial Index Record Count By Group |
| Grids: <=4 >4 >10 >25 >50 >100 >250 >500 |
|---------- ------ ------ ------ ------ ------ ------ ------ ------ |
| Features: 3877 0 0 0 0 0 0 0 |
| % Total: 100% 0% 0% 0% 0% 0% 0% 0%|
|-------------------------------------------------------------------|

sfarber wrote:

Tyler, Justin,

The other week I ran into a problem where the ArcSDE 'quick count' method
of trying to pre-determine the number of features in a particular result
was returning inaccurate results.

I fiddled with the spatial index and unit size on the arcsde layer and by
tweaking them correctly I was able to get the optimized results-count
method to be correct. Is it possible, Tyler, that there's somethin funny
about the ArcSDE data that you're looking at? Is there a proper spatial
index on the data? Is the unit-size correct?

I'll try and take a look at this issue again later, perhaps putting in a
check to make sure that the estimated returned feature count and the
actual returned feature count match (to alert us to the fact that
something's wrong when it *does* happen). This probably won't happen till
late next week at the earliest, however.

--saul

________________________________

From: Justin Deoliveira [mailto:jdeolive@anonymised.com]
Sent: Tue 7/3/2007 6:37 PM
To: Tyler Erickson
Cc: geoserver-users@lists.sourceforge.net; Farber, Saul (ENV)
Subject: Re: [Geoserver-users] KMSCORE parameter is misbehavin'

Hi Tyler,

I did a little testing of my own and noticed that the calculation was
indeed off due to integer truncation in the exponent... But I would
expect this to result in the opposite case, ie. GeoServer returning a
raster more often then it should.

My suspicion would be that the datastore (arcsde) is returning an
inaccurate count of features and is throwing off the calculation. We
never managed to get our arcsde test server up and running
unfortunately, so this one is hard for me to test hands on.

Perhaps Saul (our sde expert) can help us out on this one.

-Justin

Tyler Erickson wrote:

One of the FeatureTypes that I would like to serve to GoogleEarth has a
fairly large number of records (2000+). I have been trying to use the
KMSCORE parameter to speed things up when too many features are in view,
but
it seems to return far more features than the documentation indicates.
For
KMSCORE=10, I would expect to see an image returned (when viewing more
than
4 features), but instead GeoServer returns a large number of features
(100-1000?) as individual placemarks in KML.

Additional notes:
I am using the ArcSDE 9.2 plugin.
I don't encounter this issue when I serve up the sample tiger:poi data
with
the same KMSCORE values. This dataset behaves has described in the
documentation.

Here a the format of the KML file of the network links:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.1&quot;&gt;
<Folder>
      <name>V_Location</name>
      <visibility>0</visibility>
      <open>1</open>
      <NetworkLink>
              <name>V_Location - KMSCORE=0</name>
              <visibility>0</visibility>
              <Url>
            
<href>http://gisdev:8080/geoserver/wms?styles=&Format=kmz&KMSCORE=0&request=GetMap&layers=mtri:SDE_NRCS.DBO.V_LOCATION&width=800&height=375&srs=EPSG:4326&&lt;/href&gt;
                      <viewRefreshMode>onRequest</viewRefreshMode>
              </Url>
      </NetworkLink>
      <NetworkLink>
              <name>V_Location - KMSCORE=10</name>
              <visibility>0</visibility>
              <Url>
            
<href>http://gisdev:8080/geoserver/wms?styles=&Format=kmz&KMSCORE=10&request=GetMap&layers=mtri:SDE_NRCS.DBO.V_LOCATION&width=800&height=375&srs=EPSG:4326&&lt;/href&gt;
                      <viewRefreshMode>onRequest</viewRefreshMode>
              </Url>
      </NetworkLink>
</Folder>
</kml>

KMSCORE=0
http://www.nabble.com/file/p11421303/screenshot_kmscore_0.jpg

KMSCORE=10
http://www.nabble.com/file/p11421303/screenshot_kmscore_10.jpg

The logs look pretty normal...

INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.vfny.geoserver.wms.requests.GetMapKvpReader - Set KMScore: 10
INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.vfny.geoserver.servlets.AbstractService - handling request:
INFO | jvm 1 | 2007/07/03 17:24:14 | GetMap Request
INFO | jvm 1 | 2007/07/03 17:24:14 | version: 1.1.1
INFO | jvm 1 | 2007/07/03 17:24:14 | output format: kmz
INFO | jvm 1 | 2007/07/03 17:24:14 | width height: 375,800
INFO | jvm 1 | 2007/07/03 17:24:14 | bbox: Env[-87.71528922820525 :
-81.15297453390464, 40.84272190296385 : 45.25420697827713]
INFO | jvm 1 | 2007/07/03 17:24:14 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/03 17:24:14 | styles: measurement_location
INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.geotools.arcsde.data.ArcSDEDataStore - getCount
INFO | jvm 1 | 2007/07/03 17:24:47 | 10219611 [INFO]
org.geotools.arcsde.data.ArcSDEQuery - No FID attribute was contained in
your query. Appending the discovered one to the list of columns to be
fetched.
INFO | jvm 1 | 2007/07/03 17:24:47 | 10220142 [INFO]
org.vfny.geoserver.servlets.AbstractService - Service handled

INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.vfny.geoserver.wms.requests.GetMapKvpReader - Set KMScore: 0
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.vfny.geoserver.servlets.AbstractService - handling request:
INFO | jvm 1 | 2007/07/03 17:27:58 | GetMap Request
INFO | jvm 1 | 2007/07/03 17:27:58 | version: 1.1.1
INFO | jvm 1 | 2007/07/03 17:27:58 | output format: kmz
INFO | jvm 1 | 2007/07/03 17:27:58 | width height: 375,800
INFO | jvm 1 | 2007/07/03 17:27:58 | bbox: Env[-87.71528922820525 :
-81.15297453390464, 40.84272190296385 : 45.25420697827713]
INFO | jvm 1 | 2007/07/03 17:27:58 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/03 17:27:58 | styles: measurement_location
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.geotools.arcsde.data.ArcSDEDataStore - getCount
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410761 [INFO]
org.geotools.arcsde.data.ArcSDEQuery - No FID attribute was contained in
your query. Appending the discovered one to the list of columns to be
fetched.
INFO | jvm 1 | 2007/07/03 17:27:59 | 10411417 [INFO]
org.vfny.geoserver.servlets.AbstractService - Service handled

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org/&gt; <http://topp.openplans.org/&gt;

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
View this message in context: http://www.nabble.com/KMSCORE-parameter-is-misbehavin'-tf4021151.html#a11510705
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Is there a place to look in the logs to find out what counts the getCount() and getReader()-for-loop are returning?

  • Tyler

Farber, Saul (EEA) wrote:

···


Tyler Erickson, Ph.D.
Research Scientist
Michigan Tech Research Institute (MTRI)
3600 Green Court, Suite 100
Ann Arbor, MI 48105
Phone: 734-913-6846 (NOTE: NEW as of 1/28/07)
Fax: 734-913-6880 (NOTE: NEW as of 1/28/07)
tyler.erickson@anonymised.com
http://www.mtri.org


I tried using spatial index sizes of Grid Size = 1, 0.1, 0.01, 0.001, and
0.0001. All seem to have the same issue. (i.e. returning a KML instead of a
KMZ)

Could the GeoServer/Geotools code for ArcSDE be easily modified so that it
reports the number returned by GetCount() to the log file? I think that
could give some insight into what is going on for my particular situation.

Also, in the log there is an info message about not having a Feature ID in
the query. Is this warning something to be concerned about?

Thanks,
Tyler

INFO | jvm 1 | 2007/07/13 15:04:57 | 2862652 [btpool0-6] INFO
org.vfny.geoserver.requests.readers.wms - Set KMScore: 1
INFO | jvm 1 | 2007/07/13 15:04:57 | 2862652 [btpool0-6] INFO
org.vfny.geoserver.servlets - handling request:
INFO | jvm 1 | 2007/07/13 15:04:57 | GetMap Request
INFO | jvm 1 | 2007/07/13 15:04:57 | version: 1.1.1
INFO | jvm 1 | 2007/07/13 15:04:57 | output format: kmz
INFO | jvm 1 | 2007/07/13 15:04:57 | width height: 375,800
INFO | jvm 1 | 2007/07/13 15:04:57 | bbox: Env[-84.53711255000218 :
-84.53293824271131, 43.13842050177628 : 43.14115210476584]
INFO | jvm 1 | 2007/07/13 15:04:57 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/13 15:04:57 | styles: measurement_location
INFO | jvm 1 | 2007/07/13 15:04:57 | 2862652 [btpool0-6] INFO
org.geotools.arcsde.data - getCount
INFO | jvm 1 | 2007/07/13 15:04:57 | 2862777 [btpool0-6] INFO
org.geotools.arcsde.data - No FID attribute was contained in your query.
Appending the discovered one to the list of columns to be fetched.
INFO | jvm 1 | 2007/07/13 15:04:57 | 2862886 [btpool0-6] INFO
org.vfny.geoserver.servlets - Service handled

sfarber wrote:

I think they look pretty good to me too, but I can't claim to be an expert
on SDE grid-size tuning! From what I understand, though, point features
aren't particularly well-served by the si_stats option to sdelayer. I
mean, every point is in exactly one grid cell, right? That's the
definition of a *point*.

Anyway, I'm not totally sure what's up with your situation, but I *can*
say that I've seen ArcSDE point layers tell geotools via the 'getCount()'
method that a particular query captures 5 points, when in actuality it
captures 6. After playing with the spatial index on that point layer, I
got the results of the 'getCount()' method to agree with the results of a
getReader()-for-loop calculation.

However, I'm not really very clear on why ArcSDE was returning bad counts
via the algorithm coded into the 'getCount()' method. I don't think I
understand ArcSDE's spatial query strategy well enough to figure it out.

The short summary might be this:

* I have once encountered an ArcSDE point layer which returned bogus
feature count information via the 'getCount()' method
* Justin suggested that if a query was mis-representing it's size, this
might cause an issue wit the KMSCORE parameter.
* You are using an ArcSDE point layer.

The way I fixed my ArcSDE point layer was to fiddle with the ArcSDE
spatial index till it worked. Most of the time ArcSDE would reject my
spatial index grid sizes, but when I eventually picked one that was on the
small end of the valid range of grid sizes, my problem with bogus
'getCount()' information went away.

Let me know if you get it to work by fiddling with your spatial index.

But also make sure that the middle bullet above is appliccable! If
there's something *else* wrong which is causing the KMSCORE parameter to
misbehave, then this is likely not the right solution!

Good luck tyler,
--saul

________________________________

From: geoserver-users-bounces@lists.sourceforge.net on behalf of Tyler
Erickson
Sent: Mon 7/9/2007 5:20 PM
To: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] KMSCORE parameter is misbehavin'

Saul,

I think my spatial index parameters are reasonable, but any suggestions
would be appreciated.

I'm using unprojected coordinate system in decimal degrees (GCS_WGS_1984).
The data extent is the complete world (-180<x<180 and -90<y<90).
The geometry is a point feature class.

Below are the stats for ArcSDE's spatial index. I also tried a grid size
of
1.0, but had the same KMSCORE issue.

- Tyler

C:\Documents and Settings\terickson>sdelayer -o si_stats -s sql1 -i 5153
-D
sde_
nrcs -u "MTRI\taericks" -l T_Location,SHAPE

ArcSDE 9.2 for SQL Server Build 1137 Mon Mar 5 15:52:01 2007
Layer Administration Utility
-----------------------------------------------------
Layer 84 Spatial Index Statistics:
Level 1, Grid Size 0.0001
|-------------------------------------------------------------------|
| Grid Records: 3877 |
| Feature Records: 3877 |
| Grids/Feature Ratio: 1.00 |
| Avg. Features per Grid: 1.55 |
| Max. Features per Grid: 63 |
| % of Features Wholly Inside 1 Grid: 100.00 |
|-------------------------------------------------------------------|
| Spatial Index Record Count By Group |
| Grids: <=4 >4 >10 >25 >50 >100 >250 >500 |
|---------- ------ ------ ------ ------ ------ ------ ------ ------ |
| Features: 3877 0 0 0 0 0 0 0 |
| % Total: 100% 0% 0% 0% 0% 0% 0% 0%|
|-------------------------------------------------------------------|

sfarber wrote:

Tyler, Justin,

The other week I ran into a problem where the ArcSDE 'quick count' method
of trying to pre-determine the number of features in a particular result
was returning inaccurate results.

I fiddled with the spatial index and unit size on the arcsde layer and by
tweaking them correctly I was able to get the optimized results-count
method to be correct. Is it possible, Tyler, that there's somethin funny
about the ArcSDE data that you're looking at? Is there a proper spatial
index on the data? Is the unit-size correct?

I'll try and take a look at this issue again later, perhaps putting in a
check to make sure that the estimated returned feature count and the
actual returned feature count match (to alert us to the fact that
something's wrong when it *does* happen). This probably won't happen
till
late next week at the earliest, however.

--saul

________________________________

From: Justin Deoliveira [mailto:jdeolive@anonymised.com]
Sent: Tue 7/3/2007 6:37 PM
To: Tyler Erickson
Cc: geoserver-users@lists.sourceforge.net; Farber, Saul (ENV)
Subject: Re: [Geoserver-users] KMSCORE parameter is misbehavin'

Hi Tyler,

I did a little testing of my own and noticed that the calculation was
indeed off due to integer truncation in the exponent... But I would
expect this to result in the opposite case, ie. GeoServer returning a
raster more often then it should.

My suspicion would be that the datastore (arcsde) is returning an
inaccurate count of features and is throwing off the calculation. We
never managed to get our arcsde test server up and running
unfortunately, so this one is hard for me to test hands on.

Perhaps Saul (our sde expert) can help us out on this one.

-Justin

Tyler Erickson wrote:

One of the FeatureTypes that I would like to serve to GoogleEarth has a
fairly large number of records (2000+). I have been trying to use the
KMSCORE parameter to speed things up when too many features are in view,
but
it seems to return far more features than the documentation indicates.
For
KMSCORE=10, I would expect to see an image returned (when viewing more
than
4 features), but instead GeoServer returns a large number of features
(100-1000?) as individual placemarks in KML.

Additional notes:
I am using the ArcSDE 9.2 plugin.
I don't encounter this issue when I serve up the sample tiger:poi data
with
the same KMSCORE values. This dataset behaves has described in the
documentation.

Here a the format of the KML file of the network links:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.1&quot;&gt;
<Folder>
      <name>V_Location</name>
      <visibility>0</visibility>
      <open>1</open>
      <NetworkLink>
              <name>V_Location - KMSCORE=0</name>
              <visibility>0</visibility>
              <Url>
            
<href>http://gisdev:8080/geoserver/wms?styles=&Format=kmz&KMSCORE=0&request=GetMap&layers=mtri:SDE_NRCS.DBO.V_LOCATION&width=800&height=375&srs=EPSG:4326&&lt;/href&gt;
                      <viewRefreshMode>onRequest</viewRefreshMode>
              </Url>
      </NetworkLink>
      <NetworkLink>
              <name>V_Location - KMSCORE=10</name>
              <visibility>0</visibility>
              <Url>
            
<href>http://gisdev:8080/geoserver/wms?styles=&Format=kmz&KMSCORE=10&request=GetMap&layers=mtri:SDE_NRCS.DBO.V_LOCATION&width=800&height=375&srs=EPSG:4326&&lt;/href&gt;
                      <viewRefreshMode>onRequest</viewRefreshMode>
              </Url>
      </NetworkLink>
</Folder>
</kml>

KMSCORE=0
http://www.nabble.com/file/p11421303/screenshot_kmscore_0.jpg

KMSCORE=10
http://www.nabble.com/file/p11421303/screenshot_kmscore_10.jpg

The logs look pretty normal...

INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.vfny.geoserver.wms.requests.GetMapKvpReader - Set KMScore: 10
INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.vfny.geoserver.servlets.AbstractService - handling request:
INFO | jvm 1 | 2007/07/03 17:24:14 | GetMap Request
INFO | jvm 1 | 2007/07/03 17:24:14 | version: 1.1.1
INFO | jvm 1 | 2007/07/03 17:24:14 | output format: kmz
INFO | jvm 1 | 2007/07/03 17:24:14 | width height: 375,800
INFO | jvm 1 | 2007/07/03 17:24:14 | bbox: Env[-87.71528922820525
:
-81.15297453390464, 40.84272190296385 : 45.25420697827713]
INFO | jvm 1 | 2007/07/03 17:24:14 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/03 17:24:14 | styles: measurement_location
INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.geotools.arcsde.data.ArcSDEDataStore - getCount
INFO | jvm 1 | 2007/07/03 17:24:47 | 10219611 [INFO]
org.geotools.arcsde.data.ArcSDEQuery - No FID attribute was contained in
your query. Appending the discovered one to the list of columns to be
fetched.
INFO | jvm 1 | 2007/07/03 17:24:47 | 10220142 [INFO]
org.vfny.geoserver.servlets.AbstractService - Service handled

INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.vfny.geoserver.wms.requests.GetMapKvpReader - Set KMScore: 0
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.vfny.geoserver.servlets.AbstractService - handling request:
INFO | jvm 1 | 2007/07/03 17:27:58 | GetMap Request
INFO | jvm 1 | 2007/07/03 17:27:58 | version: 1.1.1
INFO | jvm 1 | 2007/07/03 17:27:58 | output format: kmz
INFO | jvm 1 | 2007/07/03 17:27:58 | width height: 375,800
INFO | jvm 1 | 2007/07/03 17:27:58 | bbox: Env[-87.71528922820525
:
-81.15297453390464, 40.84272190296385 : 45.25420697827713]
INFO | jvm 1 | 2007/07/03 17:27:58 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/03 17:27:58 | styles: measurement_location
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.geotools.arcsde.data.ArcSDEDataStore - getCount
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410761 [INFO]
org.geotools.arcsde.data.ArcSDEQuery - No FID attribute was contained in
your query. Appending the discovered one to the list of columns to be
fetched.
INFO | jvm 1 | 2007/07/03 17:27:59 | 10411417 [INFO]
org.vfny.geoserver.servlets.AbstractService - Service handled

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org/&gt;
<http://topp.openplans.org/&gt;

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
View this message in context:
http://www.nabble.com/KMSCORE-parameter-is-misbehavin'-tf4021151.html#a11510705
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
View this message in context: http://www.nabble.com/KMSCORE-parameter-is-misbehavin'-tf4021151.html#a11588036
Sent from the GeoServer - User mailing list archive at Nabble.com.

Tyler,

I don't think ID message is anything to worry about.

Can you create a JIRA for this issue in the geotools JIRA, under the 'data arcsde' category, and assign it to me?

The issue could be called something like 'ArcSDE query.getCount() sometimes returns a different result from actually doing the query and counting the results'

I'll try and get some logging into the ArcSDE plugin and get you a fresh copy early next week.

--saul

________________________________

From: geoserver-users-bounces@lists.sourceforge.net on behalf of Tyler Erickson
Sent: Fri 7/13/2007 5:43 PM
To: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] KMSCORE parameter is misbehavin'

I tried using spatial index sizes of Grid Size = 1, 0.1, 0.01, 0.001, and
0.0001. All seem to have the same issue. (i.e. returning a KML instead of a
KMZ)

Could the GeoServer/Geotools code for ArcSDE be easily modified so that it
reports the number returned by GetCount() to the log file? I think that
could give some insight into what is going on for my particular situation.

Also, in the log there is an info message about not having a Feature ID in
the query. Is this warning something to be concerned about?

Thanks,
Tyler

INFO | jvm 1 | 2007/07/13 15:04:57 | 2862652 [btpool0-6] INFO
org.vfny.geoserver.requests.readers.wms - Set KMScore: 1
INFO | jvm 1 | 2007/07/13 15:04:57 | 2862652 [btpool0-6] INFO
org.vfny.geoserver.servlets - handling request:
INFO | jvm 1 | 2007/07/13 15:04:57 | GetMap Request
INFO | jvm 1 | 2007/07/13 15:04:57 | version: 1.1.1
INFO | jvm 1 | 2007/07/13 15:04:57 | output format: kmz
INFO | jvm 1 | 2007/07/13 15:04:57 | width height: 375,800
INFO | jvm 1 | 2007/07/13 15:04:57 | bbox: Env[-84.53711255000218 :
-84.53293824271131, 43.13842050177628 : 43.14115210476584]
INFO | jvm 1 | 2007/07/13 15:04:57 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/13 15:04:57 | styles: measurement_location
INFO | jvm 1 | 2007/07/13 15:04:57 | 2862652 [btpool0-6] INFO
org.geotools.arcsde.data - getCount
INFO | jvm 1 | 2007/07/13 15:04:57 | 2862777 [btpool0-6] INFO
org.geotools.arcsde.data - No FID attribute was contained in your query.
Appending the discovered one to the list of columns to be fetched.
INFO | jvm 1 | 2007/07/13 15:04:57 | 2862886 [btpool0-6] INFO
org.vfny.geoserver.servlets - Service handled

sfarber wrote:

I think they look pretty good to me too, but I can't claim to be an expert
on SDE grid-size tuning! From what I understand, though, point features
aren't particularly well-served by the si_stats option to sdelayer. I
mean, every point is in exactly one grid cell, right? That's the
definition of a *point*.

Anyway, I'm not totally sure what's up with your situation, but I *can*
say that I've seen ArcSDE point layers tell geotools via the 'getCount()'
method that a particular query captures 5 points, when in actuality it
captures 6. After playing with the spatial index on that point layer, I
got the results of the 'getCount()' method to agree with the results of a
getReader()-for-loop calculation.

However, I'm not really very clear on why ArcSDE was returning bad counts
via the algorithm coded into the 'getCount()' method. I don't think I
understand ArcSDE's spatial query strategy well enough to figure it out.

The short summary might be this:

* I have once encountered an ArcSDE point layer which returned bogus
feature count information via the 'getCount()' method
* Justin suggested that if a query was mis-representing it's size, this
might cause an issue wit the KMSCORE parameter.
* You are using an ArcSDE point layer.

The way I fixed my ArcSDE point layer was to fiddle with the ArcSDE
spatial index till it worked. Most of the time ArcSDE would reject my
spatial index grid sizes, but when I eventually picked one that was on the
small end of the valid range of grid sizes, my problem with bogus
'getCount()' information went away.

Let me know if you get it to work by fiddling with your spatial index.

But also make sure that the middle bullet above is appliccable! If
there's something *else* wrong which is causing the KMSCORE parameter to
misbehave, then this is likely not the right solution!

Good luck tyler,
--saul

________________________________

From: geoserver-users-bounces@lists.sourceforge.net on behalf of Tyler
Erickson
Sent: Mon 7/9/2007 5:20 PM
To: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] KMSCORE parameter is misbehavin'

Saul,

I think my spatial index parameters are reasonable, but any suggestions
would be appreciated.

I'm using unprojected coordinate system in decimal degrees (GCS_WGS_1984).
The data extent is the complete world (-180<x<180 and -90<y<90).
The geometry is a point feature class.

Below are the stats for ArcSDE's spatial index. I also tried a grid size
of
1.0, but had the same KMSCORE issue.

- Tyler

C:\Documents and Settings\terickson>sdelayer -o si_stats -s sql1 -i 5153
-D
sde_
nrcs -u "MTRI\taericks" -l T_Location,SHAPE

ArcSDE 9.2 for SQL Server Build 1137 Mon Mar 5 15:52:01 2007
Layer Administration Utility
-----------------------------------------------------
Layer 84 Spatial Index Statistics:
Level 1, Grid Size 0.0001
|-------------------------------------------------------------------|
| Grid Records: 3877 |
| Feature Records: 3877 |
| Grids/Feature Ratio: 1.00 |
| Avg. Features per Grid: 1.55 |
| Max. Features per Grid: 63 |
| % of Features Wholly Inside 1 Grid: 100.00 |
|-------------------------------------------------------------------|
| Spatial Index Record Count By Group |
| Grids: <=4 >4 >10 >25 >50 >100 >250 >500 |
|---------- ------ ------ ------ ------ ------ ------ ------ ------ |
| Features: 3877 0 0 0 0 0 0 0 |
| % Total: 100% 0% 0% 0% 0% 0% 0% 0%|
|-------------------------------------------------------------------|

sfarber wrote:

Tyler, Justin,

The other week I ran into a problem where the ArcSDE 'quick count' method
of trying to pre-determine the number of features in a particular result
was returning inaccurate results.

I fiddled with the spatial index and unit size on the arcsde layer and by
tweaking them correctly I was able to get the optimized results-count
method to be correct. Is it possible, Tyler, that there's somethin funny
about the ArcSDE data that you're looking at? Is there a proper spatial
index on the data? Is the unit-size correct?

I'll try and take a look at this issue again later, perhaps putting in a
check to make sure that the estimated returned feature count and the
actual returned feature count match (to alert us to the fact that
something's wrong when it *does* happen). This probably won't happen
till
late next week at the earliest, however.

--saul

________________________________

From: Justin Deoliveira [mailto:jdeolive@anonymised.com]
Sent: Tue 7/3/2007 6:37 PM
To: Tyler Erickson
Cc: geoserver-users@lists.sourceforge.net; Farber, Saul (ENV)
Subject: Re: [Geoserver-users] KMSCORE parameter is misbehavin'

Hi Tyler,

I did a little testing of my own and noticed that the calculation was
indeed off due to integer truncation in the exponent... But I would
expect this to result in the opposite case, ie. GeoServer returning a
raster more often then it should.

My suspicion would be that the datastore (arcsde) is returning an
inaccurate count of features and is throwing off the calculation. We
never managed to get our arcsde test server up and running
unfortunately, so this one is hard for me to test hands on.

Perhaps Saul (our sde expert) can help us out on this one.

-Justin

Tyler Erickson wrote:

One of the FeatureTypes that I would like to serve to GoogleEarth has a
fairly large number of records (2000+). I have been trying to use the
KMSCORE parameter to speed things up when too many features are in view,
but
it seems to return far more features than the documentation indicates.
For
KMSCORE=10, I would expect to see an image returned (when viewing more
than
4 features), but instead GeoServer returns a large number of features
(100-1000?) as individual placemarks in KML.

Additional notes:
I am using the ArcSDE 9.2 plugin.
I don't encounter this issue when I serve up the sample tiger:poi data
with
the same KMSCORE values. This dataset behaves has described in the
documentation.

Here a the format of the KML file of the network links:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.1&quot;&gt;
<Folder>
      <name>V_Location</name>
      <visibility>0</visibility>
      <open>1</open>
      <NetworkLink>
              <name>V_Location - KMSCORE=0</name>
              <visibility>0</visibility>
              <Url>
           
<href>http://gisdev:8080/geoserver/wms?styles=&Format=kmz&KMSCORE=0&request=GetMap&layers=mtri:SDE_NRCS.DBO.V_LOCATION&width=800&height=375&srs=EPSG:4326&&lt;/href&gt;
                      <viewRefreshMode>onRequest</viewRefreshMode>
              </Url>
      </NetworkLink>
      <NetworkLink>
              <name>V_Location - KMSCORE=10</name>
              <visibility>0</visibility>
              <Url>
           
<href>http://gisdev:8080/geoserver/wms?styles=&Format=kmz&KMSCORE=10&request=GetMap&layers=mtri:SDE_NRCS.DBO.V_LOCATION&width=800&height=375&srs=EPSG:4326&&lt;/href&gt;
                      <viewRefreshMode>onRequest</viewRefreshMode>
              </Url>
      </NetworkLink>
</Folder>
</kml>

KMSCORE=0
http://www.nabble.com/file/p11421303/screenshot_kmscore_0.jpg

KMSCORE=10
http://www.nabble.com/file/p11421303/screenshot_kmscore_10.jpg

The logs look pretty normal...

INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.vfny.geoserver.wms.requests.GetMapKvpReader - Set KMScore: 10
INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.vfny.geoserver.servlets.AbstractService - handling request:
INFO | jvm 1 | 2007/07/03 17:24:14 | GetMap Request
INFO | jvm 1 | 2007/07/03 17:24:14 | version: 1.1.1
INFO | jvm 1 | 2007/07/03 17:24:14 | output format: kmz
INFO | jvm 1 | 2007/07/03 17:24:14 | width height: 375,800
INFO | jvm 1 | 2007/07/03 17:24:14 | bbox: Env[-87.71528922820525
:
-81.15297453390464, 40.84272190296385 : 45.25420697827713]
INFO | jvm 1 | 2007/07/03 17:24:14 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/03 17:24:14 | styles: measurement_location
INFO | jvm 1 | 2007/07/03 17:24:14 | 10186784 [INFO]
org.geotools.arcsde.data.ArcSDEDataStore - getCount
INFO | jvm 1 | 2007/07/03 17:24:47 | 10219611 [INFO]
org.geotools.arcsde.data.ArcSDEQuery - No FID attribute was contained in
your query. Appending the discovered one to the list of columns to be
fetched.
INFO | jvm 1 | 2007/07/03 17:24:47 | 10220142 [INFO]
org.vfny.geoserver.servlets.AbstractService - Service handled

INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.vfny.geoserver.wms.requests.GetMapKvpReader - Set KMScore: 0
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.vfny.geoserver.servlets.AbstractService - handling request:
INFO | jvm 1 | 2007/07/03 17:27:58 | GetMap Request
INFO | jvm 1 | 2007/07/03 17:27:58 | version: 1.1.1
INFO | jvm 1 | 2007/07/03 17:27:58 | output format: kmz
INFO | jvm 1 | 2007/07/03 17:27:58 | width height: 375,800
INFO | jvm 1 | 2007/07/03 17:27:58 | bbox: Env[-87.71528922820525
:
-81.15297453390464, 40.84272190296385 : 45.25420697827713]
INFO | jvm 1 | 2007/07/03 17:27:58 | layers:
mtri:SDE_NRCS.DBO.V_LOCATION
INFO | jvm 1 | 2007/07/03 17:27:58 | styles: measurement_location
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410745 [INFO]
org.geotools.arcsde.data.ArcSDEDataStore - getCount
INFO | jvm 1 | 2007/07/03 17:27:58 | 10410761 [INFO]
org.geotools.arcsde.data.ArcSDEQuery - No FID attribute was contained in
your query. Appending the discovered one to the list of columns to be
fetched.
INFO | jvm 1 | 2007/07/03 17:27:59 | 10411417 [INFO]
org.vfny.geoserver.servlets.AbstractService - Service handled

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org/&gt; <http://topp.openplans.org/&gt;
<http://topp.openplans.org/&gt;

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
View this message in context:
http://www.nabble.com/KMSCORE-parameter-is-misbehavin'-tf4021151.html#a11510705
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
View this message in context: http://www.nabble.com/KMSCORE-parameter-is-misbehavin'-tf4021151.html#a11588036
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Saul,

Here is the jira issue:
http://jira.codehaus.org/browse/GEOT-1394

I wasn't able to figure out how to assign the issue to you... possibly I
don't have the necessary permissions to do so. (I'm only a member of the
jira-users group.)

- Tyler

sfarber wrote:

Tyler,

I don't think ID message is anything to worry about.

Can you create a JIRA for this issue in the geotools JIRA, under the 'data
arcsde' category, and assign it to me?

The issue could be called something like 'ArcSDE query.getCount()
sometimes returns a different result from actually doing the query and
counting the results'

I'll try and get some logging into the ArcSDE plugin and get you a fresh
copy early next week.

--saul

--
View this message in context: http://www.nabble.com/KMSCORE-parameter-is-misbehavin'-tf4021151.html#a11601655
Sent from the GeoServer - User mailing list archive at Nabble.com.

Tyler Erickson ha scritto:

Saul,

Here is the jira issue:
http://jira.codehaus.org/browse/GEOT-1394

I wasn't able to figure out how to assign the issue to you...

Assigned to Saul, which will have to decide for which version
to schedule the fix of this bug.

Cheers
Andrea