[GRASS-user] GRASS7 crashes on Win10 with large VRT-file.

Hi Markus,

Thanks for confirming the VRT option should work.
I’ve started a new thread for this issue.

I’m currently running GRASS 7 for Windows on Win10-64Bit.
The ‘real’ process will run on an Ubuntu server.

g.version -bge returns

version=7.0.4
date=2016
revision=68343
build_date=2016-05-01
build_platform=x86_64-w64-mingw32

./configure --host=x86_64-w64-mingw32 '–with-libs=/c/msys64/usr/src/grass704/mswindows/osgeo4w/lib ’ --with-includes=/c/OSGeo4W64/include --libexecdir=/c/OSGeo4W64/bin --prefix=/c/OSGeo4W64/apps/grass --bindir=/c/OSGeo4W64/bin --includedir=/c/OSGeo4W64/include --without-x --with-cxx --enable-shared --enable-largefile --with-fftw --with-freetype --with-proj-share=/c/OSGeo4W64/share/proj --with-gdal=/c/msys64/usr/src/grass704/mswindows/osgeo4w/gdal-config --with-geos=/c/msys64/usr/src/grass704/mswindows/osgeo4w/geos-config --with-liblas=/c/msys64/usr/src/grass704/mswindows/osgeo4w/liblas-config --with-sqlite --with-regex --with-nls --with-freetype-includes=/c/OSGeo4W64/include/freetype2 --with-odbc --with-cairo --with-postgres --with-opengl=windows
proj4=4.9.2
gdal=2.0.2
geos=3.5.0
sqlite=3.7.17

I also have QGis and MapWindow5 GIS on my machine. Both also use the proj4, gdal, geos and sqlite libraries. And I run PostgreSQL with PostGIS which is using GDAL as well.

My tiff-files and my VRT file are on an external USB harddrive.
I created the VRT file using GDAL 1.11.3, released 2015/09/16 because that version is in my path.
gdalinfo of my VRT is:

Driver: VRT/Virtual Raster
Files: ahn2.vrt
F:\AHN2\r01cz1.tif

[…]

F:\AHN2\r72hz2.tif
Size is 532000, 600000
Coordinate System is:
PROJCS[“Amersfoort / RD New”,
GEOGCS[“Amersfoort”,
DATUM[“Amersfoort”,
SPHEROID[“Bessel 1841”,6377397.155,299.1528128000009,
AUTHORITY[“EPSG”,“7004”]],
TOWGS84[565.417,50.3319,465.552,-0.398957,0.343988,-1.8774,4.0725],
AUTHORITY[“EPSG”,“6289”]],
PRIMEM[“Greenwich”,0],
UNIT[“degree”,0.0174532925199433],
AUTHORITY[“EPSG”,“4289”]],
PROJECTION[“Oblique_Stereographic”],
PARAMETER[“latitude_of_origin”,52.15616055555555],
PARAMETER[“central_meridian”,5.38763888888889],
PARAMETER[“scale_factor”,0.9999079],
PARAMETER[“false_easting”,155000],
PARAMETER[“false_northing”,463000],
UNIT[“metre”,1,
AUTHORITY[“EPSG”,“9001”]],
AUTHORITY[“EPSG”,“28992”]]
Origin = (13000.000000000000000,606250.000000000000000)
Pixel Size = (0.500000000000000,-0.500000000000000)
Corner Coordinates:
Upper Left ( 13000.000, 606250.000) ( 3d15’ 4.06"E, 53d25’28.87"N)
Lower Left ( 13000.000, 306250.000) ( 3d22’34.17"E, 50d43’46.25"N)
Upper Right ( 279000.000, 606250.000) ( 7d15’12.50"E, 53d25’45.07"N)
Lower Right ( 279000.000, 306250.000) ( 7d 8’39.35"E, 50d44’ 1.23"N)
Center ( 146000.000, 456250.000) ( 5d15’22.63"E, 52d 5’43.48"N)
Band 1 Block=128x128 Type=Float32, ColorInterp=Gray
NoData Value=-3.4028234663852901e+038

To be more precise. I have 1351 tiff-files with a total size of 579 GB.

If needed I could try on another pc with Win10, without QGIS, MW5 and PostGIS to test if I can open the VRT file using GRASS7.
But eventually I need to combine my PostGIS data because it holds my vector files.

···

Thanks,

Paul

2016-06-28 23:04 GMT+02:00 Markus Neteler <neteler@osgeo.org>:

Hi Paul,

On Tue, Jun 28, 2016 at 8:32 AM, Paul Meems <bontepaarden@gmail.com> wrote:

Thanks Markus for your suggestion.

I first tried Stefan’s suggestion of using a VRT-file.
I have 1300 tiff-files with a combined file size of 400GB. Creating the VRT-file was no problem, but when I tried to import the VRT-file in GRASS7 using r.in.gdal GRASS crashes and dies.

Huh - should not happen … But which operating system are you using,
which GRASS version etc?
You may post the result of

g.version -bge

Loading the same VRT-file in QGis did take some time to load but it worked.

Please post also the output of

gdalinfo yourfile.vrt

So I think I need to set-up a GeoServer after all, just to get a small portion of my height data to process in GRASS without worrying about polygons that are partially on multiple tiff-files.

Of course I’m open to any other suggestion. As said I’m new to GRASS.

Let’s try harder, data in the your file size range should not be a problem.

Markus

On Thu, Jun 30, 2016 at 11:15 AM, Paul Meems <bontepaarden@gmail.com> wrote:

Hi Markus,

Thanks for confirming the VRT option should work.
I've started a new thread for this issue.

I'm currently running GRASS 7 for Windows on Win10-64Bit.
The 'real' process will run on an Ubuntu server.

g.version -bge returns

version=7.0.4
date=2016
revision=68343
build_date=2016-05-01
build_platform=x86_64-w64-mingw32

...

My tiff-files and my VRT file are on an external USB harddrive.
I created the VRT file using GDAL 1.11.3, released 2015/09/16 because that
version is in my path.
gdalinfo of my VRT is:
Driver: VRT/Virtual Raster
Files: ahn2.vrt
       F:\AHN2\r01cz1.tif
       [...]
       F:\AHN2\r72hz2.tif
Size is 532000, 600000
Coordinate System is:
PROJCS["Amersfoort / RD New",
    GEOGCS["Amersfoort",
        DATUM["Amersfoort",
            SPHEROID["Bessel 1841",6377397.155,299.1528128000009,
                AUTHORITY["EPSG","7004"]],

TOWGS84[565.417,50.3319,465.552,-0.398957,0.343988,-1.8774,4.0725],
            AUTHORITY["EPSG","6289"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433],
        AUTHORITY["EPSG","4289"]],
    PROJECTION["Oblique_Stereographic"],
    PARAMETER["latitude_of_origin",52.15616055555555],
    PARAMETER["central_meridian",5.38763888888889],
    PARAMETER["scale_factor",0.9999079],
    PARAMETER["false_easting",155000],
    PARAMETER["false_northing",463000],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AUTHORITY["EPSG","28992"]]
Origin = (13000.000000000000000,606250.000000000000000)
Pixel Size = (0.500000000000000,-0.500000000000000)
Corner Coordinates:
Upper Left ( 13000.000, 606250.000) ( 3d15' 4.06"E, 53d25'28.87"N)
Lower Left ( 13000.000, 306250.000) ( 3d22'34.17"E, 50d43'46.25"N)
Upper Right ( 279000.000, 606250.000) ( 7d15'12.50"E, 53d25'45.07"N)
Lower Right ( 279000.000, 306250.000) ( 7d 8'39.35"E, 50d44' 1.23"N)
Center ( 146000.000, 456250.000) ( 5d15'22.63"E, 52d 5'43.48"N)
Band 1 Block=128x128 Type=Float32, ColorInterp=Gray
  NoData Value=-3.4028234663852901e+038

To be more precise. I have 1351 tiff-files with a total size of 579 GB.

Suggestion: try r.external as well since it avoids the true import.
Would that work?

Markus