Mosaic index generation fails over a large tile set
---------------------------------------------------
Key: GEOS-3548
URL: http://jira.codehaus.org/browse/GEOS-3548
Project: GeoServer
Issue Type: Bug
Reporter: Andrea Aime
Assignee: Simone Giannecchini
Fix For: 2.0-RC2
Was trying to import a 512 tiles mosaic into GeoServer by pointing at the directory containing them. The store was generated with no error messages but the mosaic contained only part of the tiles (see attached screenshot of the visualized index, it should be the full rectangular board).
Stopped GS, cleaned up the index and property files, restarted it with VERBOSE logging. This time it failed (no idea why this time, but not the previous one) with the following errors in the log:
{code}
2009-09-27 03:35:28,859 DEBUG [gce.gtopo30] - Unrecognized file (file extension doesn't match)
java.io.IOException: Unrecognized file (file extension doesn't match)
at org.geotools.gce.gtopo30.GTopo30Reader.<init>(GTopo30Reader.java:268)
at org.geotools.gce.gtopo30.GTopo30Reader.<init>(GTopo30Reader.java:173)
at org.geotools.gce.gtopo30.GTopo30Format.accepts(GTopo30Format.java:173)
at org.geotools.coverage.grid.io.GridFormatFinder.findFormats(GridFormatFinder.java:174)
at org.geotools.coverage.grid.io.GridFormatFinder.findFormat(GridFormatFinder.java:203)
at org.geotools.gce.imagemosaic.IndexBuilder$MosaicDirectoryWalker.handleFile(IndexBuilder.java:561)
at org.apache.commons.io.DirectoryWalker.walk(DirectoryWalker.java:367)
...
2009-09-27 03:35:28,860 DEBUG [coverageio.gdal] - /opt/data/bluemarble_512/world-topo-bathy-70200-29700.tif (Too many op
en files)
java.io.FileNotFoundException: /opt/data/bluemarble_512/world-topo-bathy-70200-29700.tif (Too many open files)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
at it.geosolutions.imageio.stream.eraf.EnhancedRandomAccessFile.<init>(EnhancedRandomAccessFile.java:191)
at it.geosolutions.imageio.stream.eraf.EnhancedRandomAccessFile.<init>(EnhancedRandomAccessFile.java:174)
at it.geosolutions.imageio.stream.input.FileImageInputStreamExtImpl.<init>(FileImageInputStreamExtImpl.java:221)
at it.geosolutions.imageio.stream.input.FileImageInputStreamExtImpl.<init>(FileImageInputStreamExtImpl.java:172)
at it.geosolutions.imageio.gdalframework.GDALImageReaderSpi.canDecodeInput(GDALImageReaderSpi.java:157)
at it.geosolutions.imageio.plugins.mrsid.MrSIDImageReaderSpi.canDecodeInput(MrSIDImageReaderSpi.java:110)
...
2009-09-27 03:35:28,861 ERROR [imagemosaic.ImageMosaicUtils] - Unable to build mosaic
java.lang.IllegalArgumentException: No input stream for the provided source
at org.geotools.gce.geotiff.GeoTiffReader.<init>(GeoTiffReader.java:230)
at org.geotools.gce.geotiff.GeoTiffFormat.getReader(GeoTiffFormat.java:261)
at org.geotools.gce.geotiff.GeoTiffFormat.getReader(GeoTiffFormat.java:222)
at org.geotools.gce.imagemosaic.IndexBuilder$MosaicDirectoryWalker.handleFile(IndexBuilder.java:566)
at org.apache.commons.io.DirectoryWalker.walk(DirectoryWalker.java:367)
{code}
Note the " (Too many open files)" issue, meaning I should raise the ulimit.
The problem is difficult to diagnose thought, the first time I got no errors, the second time the only ERROR level message did not contain an indication about the too many open files (so if I did not raise the logging level to VERBOSE I woudl have not noticed it).
Also, the mosaic generation does not really require the grids to be kept open, imho it should keep on working despite the high number of files and eventually fail when trying to access them at runtime (possibly with an error giving some hint about the excessive number of files open).
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira