[Geoserver-users] WFS: download files (zip shapefile) truncated

GeoServer GeoServer User
1

Hello all:

I’m having some problems with downloading zipped shapefiles from layers preview page.

Layers look ok over Open Layers viewer and also the KML in GE.

The problem is with the WFS download file types. None of them seems to give all the records/features. the file types CSV, shapefiles, begin OK but then are truncated.

How can I correct this problem?

Any help will be much appreciated.

Iván Santiago

GIS Specialist
Information Technologies
Office of Management and Budget
787.977.9200 x 1230
Calle Cruz 254
PO Box 9023228
San Juan, PR 00902-3228
-66°04’00.33”,18°26’55.78”(NAD83)

2

On Fri, Nov 5, 2010 at 10:24 PM, Ivan Santiago
<isantiago@anonymised.com> wrote:

Hello all:

I’m having some problems with downloading zipped shapefiles from layers
preview page.

Layers look ok over Open Layers viewer and also the KML in GE.

The problem is with the WFS download file types. None of them seems to give
all the records/features. the file types CSV, shapefiles, begin OK but then
are truncated.

How can I correct this problem?

Judging by the lack fo responses I guess the answer is "we have no idea".
Never experienced that specific problem. I guess it might be specific
to your enviroment.
Things to try out:
- upgrade the java version
- change the web container (if you used your own instead of the one built into
  the installers)
- try out a more recent version of GeoServer (grab a nightly build from here:

http://gridlock.opengeo.org/

Cheers
Andrea

-----------------------------------------------------
Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-----------------------------------------------------

3

OK. We reinstalled geoserver to version 2.1 beta with SDE Extension.

The “truncated” shapefile problem still exists…

Here is an extract that I found interesting just from the “tiger_roads” example built in geoserver:

To gt2: Query:

feature type: tiger_roads

filter: Filter.INCLUDE

[properties: ALL ]

2010-11-16 16:32:50,359 INFO [geoserver.wfs] -

Request: getFeature

handle = null

service = WFS

version = 1.0.0

baseUrl = http://172.16.110.232:8080/geoserver/

providedVersion = null

extendedProperties = {}

query = [net.opengis.wfs.impl.QueryTypeImpl@anonymised.com (group: null, propertyName: null, function: null, filter: null, sortBy: null, featureVersion: null, handle: null, srsName: null, typeName: [{http://www.census.gov}tiger_roads])]

maxFeatures = 50

outputFormat = SHAPE-ZIP

resultType = results

traverseXlinkDepth = null

traverseXlinkExpiry = null

formatOptions = {}

metadata = null

2010-11-16 16:32:50,359 DEBUG [data.shapefile] - Write lock: file:/C:/WINDOWS/TEMP/shpziptemp0.8081999345029207/tiger_roads.shp by ShpFiles for deleting all files

Why maxfeatures = 50?

I checked the shapefile in arcgis… yes… the number of features are 50!.

We repeated the test for other shapefile. again… the number of features are 50!.

Where can I change this get feature limitation?

attached is the complete log file.

Thanks in advance.

Iván Santiago

GIS Specialist

Information Technologies

Office of Management and Budget

787.977.9200 x 1230

Calle Cruz 254

PO Box 9023228

San Juan, PR 00902-3228

-66°04’00.33”,18°26’55.78”(NAD83)

et post malam segetem segendum est.

«Incluso después de una mala cosecha, es preciso sembrar», Séneca.

-----Original Message-----
From: andrea.aime@anonymised.com [mailto:andrea.aime@anonymised.com] On Behalf Of Andrea Aime
Sent: Saturday, November 06, 2010 9:58 AM
To: Ivan Santiago
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] WFS: download files (zip shapefile) truncated

On Fri, Nov 5, 2010 at 10:24 PM, Ivan Santiago

isantiago@anonymised.com wrote:

Hello all:

I’m having some problems with downloading zipped shapefiles from layers

preview page.

Layers look ok over Open Layers viewer and also the KML in GE.

The problem is with the WFS download file types. None of them seems to give

all the records/features. the file types CSV, shapefiles, begin OK but then

are truncated.

How can I correct this problem?

Judging by the lack fo responses I guess the answer is “we have no idea”.

Never experienced that specific problem. I guess it might be specific

to your enviroment.

Things to try out:

  • upgrade the java version

  • change the web container (if you used your own instead of the one built into

the installers)

  • try out a more recent version of GeoServer (grab a nightly build from here:

http://gridlock.opengeo.org/

Cheers

Andrea

Ing. Andrea Aime

Senior Software Engineer

GeoSolutions S.A.S.

Via Poggio alle Viti 1187

55054 Massarosa (LU)

Italy

phone: +39 0584962313

fax: +39 0584962313

http://www.geo-solutions.it

http://geo-solutions.blogspot.com/

http://www.linkedin.com/in/andreaaime

http://twitter.com/geowolf

geoserver.log (1.09 MB)

4

The preview links are restricted to 50 features to avoid troubles for
people that are playing
with larger data sets (50... millions features).

The preview is just a demo tool, if you want to actually download data
build your own
wfs request and don't set &maxFeature=50 into it

Cheers
Andrea

On Tue, Nov 16, 2010 at 9:41 PM, Ivan Santiago
<isantiago@anonymised.com> wrote:

OK. We reinstalled geoserver to version 2.1 beta with SDE Extension.

The "truncated" shapefile problem still exists...

Here is an extract that I found interesting just from the "tiger_roads"
example built in geoserver:

To gt2: Query:

feature type: tiger_roads

filter: Filter.INCLUDE

[properties: ALL ]

2010-11-16 16:32:50,359 INFO [geoserver.wfs] -

Request: getFeature

  handle = null

  service = WFS

  version = 1\.0\.0

  baseUrl = http://172.16.110.232:8080/geoserver/

  providedVersion = null

  extendedProperties = \{\}

  query = \[net\.opengis\.wfs\.impl\.QueryTypeImpl@anonymised\.com \(group: null,

propertyName: null, function: null, filter: null, sortBy: null,
featureVersion: null, handle: null, srsName: null, typeName:
[{http://www.census.gov}tiger_roads])]

  maxFeatures = 50

  outputFormat = SHAPE\-ZIP

  resultType = results

  traverseXlinkDepth = null

  traverseXlinkExpiry = null

  formatOptions = \{\}

  metadata = null

2010-11-16 16:32:50,359 DEBUG [data.shapefile] - Write lock:
file:/C:/WINDOWS/TEMP/shpziptemp0.8081999345029207/tiger_roads.shp by
ShpFiles for deleting all files

Why maxfeatures = 50?

I checked the shapefile in arcgis... yes.. the number of features are 50!.

We repeated the test for other shapefile. again... the number of features
are 50!.

Where can I change this get feature limitation?

attached is the complete log file.

Thanks in advance.

---------------------------

Iván Santiago

GIS Specialist

Information Technologies

Office of Management and Budget

787.977.9200 x 1230

Calle Cruz 254

PO Box 9023228

San Juan, PR 00902-3228

-66°04’00.33”,18°26’55.78”(NAD83)

et post malam segetem segendum est.

«Incluso después de una mala cosecha, es preciso sembrar», Séneca.

-----Original Message-----
From: andrea.aime@anonymised.com [mailto:andrea.aime@anonymised.com] On Behalf Of
Andrea Aime
Sent: Saturday, November 06, 2010 9:58 AM
To: Ivan Santiago
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] WFS: download files (zip shapefile) truncated

On Fri, Nov 5, 2010 at 10:24 PM, Ivan Santiago

<isantiago@anonymised.com> wrote:

Hello all:

I’m having some problems with downloading zipped shapefiles from layers

preview page.

Layers look ok over Open Layers viewer and also the KML in GE.

The problem is with the WFS download file types. None of them seems to
give

all the records/features. the file types CSV, shapefiles, begin OK but
then

are truncated.

How can I correct this problem?

Judging by the lack fo responses I guess the answer is "we have no idea".

Never experienced that specific problem. I guess it might be specific

to your enviroment.

Things to try out:

- upgrade the java version

- change the web container (if you used your own instead of the one built
into

the installers)

- try out a more recent version of GeoServer (grab a nightly build from
here:

http://gridlock.opengeo.org/

Cheers

Andrea

-----------------------------------------------------

Ing. Andrea Aime

Senior Software Engineer

GeoSolutions S.A.S.

Via Poggio alle Viti 1187

55054 Massarosa (LU)

Italy

phone: +39 0584962313

fax: +39 0584962313

http://www.geo-solutions.it

http://geo-solutions.blogspot.com/

http://www.linkedin.com/in/andreaaime

http://twitter.com/geowolf

-----------------------------------------------------

--
-----------------------------------------------------
Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-----------------------------------------------------

5

Thanks and sorry to bother you.

The preview links are restricted to 50 features to avoid troubles for
people that are playing
with larger data sets (50... millions features).

Preview links are?

The preview is just a demo tool, if you want to actually download data
build your own
wfs request and don't set &maxFeature=50 into it

Where I can find how to build that? is that an XML file? Where should I put it?

Thanks in advance.
---------------------------
Iván Santiago
GIS Specialist
Information Technologies
Office of Management and Budget
787.977.9200 x 1230
Calle Cruz 254
PO Box 9023228
San Juan, PR 00902-3228
-66°04'00.33",18°26'55.78"(NAD83)
et post malam segetem segendum est.

-----Original Message-----
From: andrea.aime@anonymised.com [mailto:andrea.aime@anonymised.com] On Behalf Of Andrea Aime
Sent: Tuesday, November 16, 2010 7:09 PM
To: Ivan Santiago
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] WFS: download files (zip shapefile) truncated

The preview links are restricted to 50 features to avoid troubles for
people that are playing
with larger data sets (50... millions features).

The preview is just a demo tool, if you want to actually download data
build your own
wfs request and don't set &maxFeature=50 into it

Cheers
Andrea

On Tue, Nov 16, 2010 at 9:41 PM, Ivan Santiago
<isantiago@anonymised.com> wrote:

OK. We reinstalled geoserver to version 2.1 beta with SDE Extension.

The "truncated" shapefile problem still exists...

Here is an extract that I found interesting just from the "tiger_roads"
example built in geoserver:

To gt2: Query:

feature type: tiger_roads

filter: Filter.INCLUDE

[properties: ALL ]

2010-11-16 16:32:50,359 INFO [geoserver.wfs] -

Request: getFeature

  handle = null

  service = WFS

  version = 1\.0\.0

  baseUrl = http://172.16.110.232:8080/geoserver/

  providedVersion = null

  extendedProperties = \{\}

  query = \[net\.opengis\.wfs\.impl\.QueryTypeImpl@anonymised\.com \(group: null,

propertyName: null, function: null, filter: null, sortBy: null,
featureVersion: null, handle: null, srsName: null, typeName:
[{http://www.census.gov}tiger_roads])]

  maxFeatures = 50

  outputFormat = SHAPE\-ZIP

  resultType = results

  traverseXlinkDepth = null

  traverseXlinkExpiry = null

  formatOptions = \{\}

  metadata = null

2010-11-16 16:32:50,359 DEBUG [data.shapefile] - Write lock:
file:/C:/WINDOWS/TEMP/shpziptemp0.8081999345029207/tiger_roads.shp by
ShpFiles for deleting all files

Why maxfeatures = 50?

I checked the shapefile in arcgis... yes.. the number of features are 50!.

We repeated the test for other shapefile. again... the number of features
are 50!.

Where can I change this get feature limitation?

attached is the complete log file.

Thanks in advance.

---------------------------

Iván Santiago

GIS Specialist

Information Technologies

Office of Management and Budget

787.977.9200 x 1230

Calle Cruz 254

PO Box 9023228

San Juan, PR 00902-3228

-66°04'00.33",18°26'55.78"(NAD83)

et post malam segetem segendum est.

«Incluso después de una mala cosecha, es preciso sembrar», Séneca.

-----Original Message-----
From: andrea.aime@anonymised.com [mailto:andrea.aime@anonymised.com] On Behalf Of
Andrea Aime
Sent: Saturday, November 06, 2010 9:58 AM
To: Ivan Santiago
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] WFS: download files (zip shapefile) truncated

On Fri, Nov 5, 2010 at 10:24 PM, Ivan Santiago

<isantiago@anonymised.com> wrote:

Hello all:

I'm having some problems with downloading zipped shapefiles from layers

preview page.

Layers look ok over Open Layers viewer and also the KML in GE.

The problem is with the WFS download file types. None of them seems to
give

all the records/features. the file types CSV, shapefiles, begin OK but
then

are truncated.

How can I correct this problem?

Judging by the lack fo responses I guess the answer is "we have no idea".

Never experienced that specific problem. I guess it might be specific

to your enviroment.

Things to try out:

- upgrade the java version

- change the web container (if you used your own instead of the one built
into

the installers)

- try out a more recent version of GeoServer (grab a nightly build from
here:

http://gridlock.opengeo.org/

Cheers

Andrea

-----------------------------------------------------

Ing. Andrea Aime

Senior Software Engineer

GeoSolutions S.A.S.

Via Poggio alle Viti 1187

55054 Massarosa (LU)

Italy

phone: +39 0584962313

fax: +39 0584962313

http://www.geo-solutions.it

http://geo-solutions.blogspot.com/

http://www.linkedin.com/in/andreaaime

http://twitter.com/geowolf

-----------------------------------------------------

--
-----------------------------------------------------
Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-----------------------------------------------------