[Geoserver-devel] Max size for return from postgisDB ?

Niels_Svennekjaer · April 10, 2003, 12:43pm

Hi

Is there a maxsize for how much data there can be return from the postgis DB in one request ?

I am trying to fetch 2446 rows of this (table name fg)

And I am getting this response

--
XML Parsing Error: not well-formed
Location: http://131.165.211.29:8080/geoserver/GetFeature?typeName=fg
Line Number 1, Column 23464:

EPSG:23032"><gml:coordinates>511558.474524356,6242256.83409024</gml:coordinates></gml:Point> .......

--

First I was running with loglevel FINEST and did consider that maby the transmition
of data from postgis to geoserver did take to long (it did take > 20 sec.) so I did change
the loglevel to INFO, but still have the problem (same column no)

I don't attach the catalina.out, unless you want 1.2Mb in the mail

but it seems that there is 500 lines returned from the request (well I know there is
an LIMIT 500 on the retrival statement, but what with the remaning ~1900 rows ?)

I have changed Typeinfo fo iso-8859-1

--

btw, is the field 'objectid' mandatory ?

-- info.xml
<?xml version="1.0" encoding="UTF-8"?>
<featureType xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

  <Name>fg</Name>
  <Title>Forekomster af Forurenede grunde</Title>
  <Abstract>Forekomster af forurenede grunde i Viborg Amt</Abstract>
  <Keywords>Forurenede grunde, Viborg Amt</Keywords>
  <SRS>EPSG:23032</SRS>
  <Operations/>

  <Host>localhost</Host>
  <Port>5432</Port>
  <DatabaseName>vibamt</DatabaseName>
  <User>gis</User>
  <Password></Password>
</featureType>

-- schema.xml

  <xs:complexType name="fg_Type">
    <xs:complexContent>
      <xs:extension base="gml:AbstractFeatureType">
        <xs:sequence>

<xs:element name="fg.gid" nillable="true" minOccurs="0" maxOccurs="1">
</xs:element>

          <xs:element name="fg.id_nr" nillable="true" minOccurs="0" maxOccurs="1">
            <xs:simpleType>
              <xs:restriction base="xs:string">
                <xs:maxLength value="100"/>
              </xs:restriction>
            </xs:simpleType>
          </xs:element>

          <xs:element name="fg.navn" nillable="true" minOccurs="0" maxOccurs="1">
            <xs:simpleType>
              <xs:restriction base="xs:string">
                <xs:maxLength value="100"/>
              </xs:restriction>
            </xs:simpleType>
          </xs:element>

          <xs:element name="fg.address" nillable="true" minOccurs="0" maxOccurs="1">
            <xs:simpleType>
              <xs:restriction base="xs:string">
                <xs:maxLength value="100"/>
              </xs:restriction>
            </xs:simpleType>
          </xs:element>

<xs:element name="fg.zip" nillable="true" minOccurs="0" maxOccurs="1">
</xs:element>

          <xs:element name="fg.name" nillable="true" minOccurs="0" maxOccurs="1">
            <xs:simpleType>
              <xs:restriction base="xs:string">
                <xs:maxLength value="100"/>
              </xs:restriction>
            </xs:simpleType>
          </xs:element>

<xs:element name="fg.x" nillable="true" minOccurs="0" maxOccurs="1">
</xs:element>

<xs:element name="fg.y" nillable="true" minOccurs="0" maxOccurs="1">
</xs:element>

          <xs:element name="fg.status" nillable="true" minOccurs="0" maxOccurs="1">
            <xs:simpleType>
              <xs:restriction base="xs:string">
                <xs:maxLength value="100"/>
              </xs:restriction>
            </xs:simpleType>
          </xs:element>

<xs:element name=".the_geom" type="gml:PointPropertyType" nillable="false" minOccurs="1" maxOccurs="1"/>

        </xs:sequence>
      </xs:extension>
    </xs:complexContent>
  </xs:complexType>

Chris_Holmes1 · April 10, 2003, 6:37pm

Is there a maxsize for how much data there can be return from the
postgis DB in one request ?

Yes, it is currently set at 500, from way back when. I just changed it to
2000, and added a nice new tag in the global configuration called
MaxFeatures, so you can set the number of features to return in your
configuration.xml file. Unfortunately we at geoserver are still working
hard on spec compliance, and haven't focused too much on speed and memory
efficiency, so you'll have to play with with maxFeatures to see how many
you can handle. I've been able to handle about 15,000, but beyond that my
machine would choke from memory problems, as we've got a lot of strings
and features flying around. After the 1.0 release we'll get some time to
go back and really beef things up. And if you don't have the latest cvs
version (I put the new maxFeature code in about 10 minutes ago), than you
can also just go to org.vfny.geoserver.request.FeatureRequest and change
line 28 SOFT_MAX_FEATURES to what you want the max to be (and you should
also probably change line 32 HARD_MAX_FEATURES to be greater than the soft
max).

And I am getting this response

--
XML Parsing Error: not well-formed
Location: http://131.165.211.29:8080/geoserver/GetFeature?typeName=fg
Line Number 1, Column 23464:

EPSG:23032"><gml:coordinates>511558.474524356,6242256.83409024</gml:coordinates></gml:Point>
.......

I'm not sure what the problem is here. I'd need to see the full response.
You can email large files directly to me...my mail server should be able
to handle them. Just send them after you send to the list, and I'll
attach the relevant parts in my reply.

First I was running with loglevel FINEST and did consider that maby the
transmition
of data from postgis to geoserver did take to long (it did take > 20
sec.) so I did change
the loglevel to INFO, but still have the problem (same column no)

(I also just added a configuration tag called LogLevel, so you can just
set it there, instead of going into the source code).

btw, is the field 'objectid' mandatory ?

No, I don't think so. Where did you get that impression? I'm updating
the documentation soon, and I want to be sure it's right. On the backend
we do make use of postgresql's invisible oid column to serve as the
featureId. But we're actually trying to move away from doing that, as
postgresql is moving away from the oid column. So for now if you declare
a primary key we use that as the fid for your feature. But that's not
recommended right now if you are going to do transactions, as it's not
updated to perform inserts correctly (we have to generate new fids, so we
probably need a sequence or something).

Best,

Chris

-- info.xml
<?xml version="1.0" encoding="UTF-8"?>
<featureType xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

  <Name>fg</Name>
  <Title>Forekomster af Forurenede grunde</Title>
  <Abstract>Forekomster af forurenede grunde i Viborg Amt</Abstract>
  <Keywords>Forurenede grunde, Viborg Amt</Keywords>
  <SRS>EPSG:23032</SRS>
  <Operations/>

  <LatLonBoundingBox minx="-174.561" miny="-8.20422e+12" maxx="154.107"
maxy="8.20422e+12" />

   <MetadataURL>
    <url>none</url>
    <format>XML</format>
    <type>TC211</type>
  </MetadataURL>

  <Host>localhost</Host>
  <Port>5432</Port>
  <DatabaseName>vibamt</DatabaseName>
  <User>gis</User>
  <Password></Password>
</featureType>

-- schema.xml

  <xs:complexType name="fg_Type">
    <xs:complexContent>
      <xs:extension base="gml:AbstractFeatureType">
        <xs:sequence>

          <xs:element name="fg.gid" nillable="true" minOccurs="0"
maxOccurs="1">
          </xs:element>

          <xs:element name="fg.id_nr" nillable="true" minOccurs="0"
maxOccurs="1">
            <xs:simpleType>
              <xs:restriction base="xs:string">
                <xs:maxLength value="100"/>
              </xs:restriction>
            </xs:simpleType>
          </xs:element>

          <xs:element name="fg.navn" nillable="true" minOccurs="0"
maxOccurs="1">
            <xs:simpleType>
              <xs:restriction base="xs:string">
                <xs:maxLength value="100"/>
              </xs:restriction>
            </xs:simpleType>
          </xs:element>

          <xs:element name="fg.address" nillable="true" minOccurs="0"
maxOccurs="1">
            <xs:simpleType>
              <xs:restriction base="xs:string">
                <xs:maxLength value="100"/>
              </xs:restriction>
            </xs:simpleType>
          </xs:element>

          <xs:element name="fg.zip" nillable="true" minOccurs="0"
maxOccurs="1">
          </xs:element>

          <xs:element name="fg.name" nillable="true" minOccurs="0"
maxOccurs="1">
            <xs:simpleType>
              <xs:restriction base="xs:string">
                <xs:maxLength value="100"/>
              </xs:restriction>
            </xs:simpleType>
          </xs:element>

          <xs:element name="fg.x" nillable="true" minOccurs="0"
maxOccurs="1">
          </xs:element>

          <xs:element name="fg.y" nillable="true" minOccurs="0"
maxOccurs="1">
          </xs:element>

          <xs:element name="fg.status" nillable="true" minOccurs="0"
maxOccurs="1">
            <xs:simpleType>
              <xs:restriction base="xs:string">
                <xs:maxLength value="100"/>
              </xs:restriction>
            </xs:simpleType>
          </xs:element>

          <xs:element name=".the_geom" type="gml:PointPropertyType"
nillable="false" minOccurs="1" maxOccurs="1"/>

        </xs:sequence>
      </xs:extension>
    </xs:complexContent>
  </xs:complexType>

-------------------------------------------------------
This SF.net email is sponsored by: Etnus, makers of TotalView, The debugger
for complex code. Debugging C/C++ programs can leave you feeling lost and
disoriented. TotalView can help you find your way. Available on major UNIX
and Linux platforms. Try it free. www.etnus.com
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel