I am experiencing significant performance issues with virtual rasters build with r.buildvrt over GDAL-linked (r.external) raster maps (source is in GeoTiff format) on NFS.
The GeoTiff data are located on a relatively fast NFS network storage, while the GRASS DB is located on a somewhat slower NFS storage system.
In order to drill down to the issue I ran r.univar on three GDAL linked raster maps that cover my computational region. The process which consists of three r.univar runs takes ~40 seconds.
Using the same computational region, one r.univar on a virtual raster of the same three raster map has not finished after 30 minutes. After converting the GDAL linked maps to native GRASS format (on the slower NFS) I built a VRT over the GRASS GIS format raster maps. For that VRT, performance is almost identical to to r.univar onr the individual raster maps (= no VRT). So it seems the issue is reading GDAL linked raster maps through GRASS VRTs.
Is that a limitation of virtual rasters in GRASS GIS or is it a bug in how VRTs are read? Is there a point to open an issue for this?
Currently, VRT with GDAL linked data seems practically unusable… Any suggestions on what can be done is very welcome. Would be a pity if I had to abandon VRTs…
Now, I investigated a bit more. It seems to be an issue with how VRTs are read.
In debug=2 mode I see waaaaay more calls to:
G_file_name()
G_find_raster2()
file open: read (mode = r)
G__read_Cell_head
G__read_Cell_head_array
when running r.univar on a VRT compared to reading the same maps not going through VRT.
Gesendet: Mittwoch, 18. September 2024 um 15:56 Uhr Von: “Stefan Blumentrath via grass-user” grass-user@lists.osgeo.org An:grass-user@lists.osgeo.org Betreff: [GRASS-user] r.buildvrt: performace issue with virtual rasters of GDAL linked raster maps on NFS
Hi,
I am experiencing significant performance issues with virtual rasters build with r.buildvrt over GDAL-linked (r.external) raster maps (source is in GeoTiff format) on NFS.
The GeoTiff data are located on a relatively fast NFS network storage, while the GRASS DB is located on a somewhat slower NFS storage system.
In order to drill down to the issue I ran r.univar on three GDAL linked raster maps that cover my computational region. The process which consists of three r.univar runs takes ~40 seconds.
Using the same computational region, one r.univar on a virtual raster of the same three raster map has not finished after 30 minutes. After converting the GDAL linked maps to native GRASS format (on the slower NFS) I built a VRT over the GRASS GIS format raster maps. For that VRT, performance is almost identical to to r.univar onr the individual raster maps (= no VRT). So it seems the issue is reading GDAL linked raster maps through GRASS VRTs.
Is that a limitation of virtual rasters in GRASS GIS or is it a bug in how VRTs are read? Is there a point to open an issue for this?
Currently, VRT with GDAL linked data seems practically unusable… Any suggestions on what can be done is very welcome. Would be a pity if I had to abandon VRTs…