[GRASS-user] r.buildvrt: performace issue with virtual rasters of GDAL linked raster maps on NFS

Hi,

I am experiencing significant performance issues with virtual rasters build with r.buildvrt over GDAL-linked (r.external) raster maps (source is in GeoTiff format) on NFS.

The GeoTiff data are located on a relatively fast NFS network storage, while the GRASS DB is located on a somewhat slower NFS storage system.

In order to drill down to the issue I ran r.univar on three GDAL linked raster maps that cover my computational region. The process which consists of three r.univar runs takes ~40 seconds.

Using the same computational region, one r.univar on a virtual raster of the same three raster map has not finished after 30 minutes. After converting the GDAL linked maps to native GRASS format (on the slower NFS) I built a VRT over the GRASS GIS format raster maps. For that VRT, performance is almost identical to to r.univar onr the individual raster maps (= no VRT). So it seems the issue is reading GDAL linked raster maps through GRASS VRTs.

Is that a limitation of virtual rasters in GRASS GIS or is it a bug in how VRTs are read? Is there a point to open an issue for this?
Currently, VRT with GDAL linked data seems practically unusable… Any suggestions on what can be done is very welcome. Would be a pity if I had to abandon VRTs…

Cheers,
Stefan

Hi again,

Now, I investigated a bit more. It seems to be an issue with how VRTs are read.
In debug=2 mode I see waaaaay more calls to:
G_file_name()
G_find_raster2()
file open: read (mode = r)
G__read_Cell_head
G__read_Cell_head_array

when running r.univar on a VRT compared to reading the same maps not going through VRT.

So I would say this is a bug with VRTs. Opened a ticket: https://github.com/OSGeo/grass/issues/4345

Cheers,
Stefan

Gesendet: Mittwoch, 18. September 2024 um 15:56 Uhr
Von: “Stefan Blumentrath via grass-user” grass-user@lists.osgeo.org
An: grass-user@lists.osgeo.org
Betreff: [GRASS-user] r.buildvrt: performace issue with virtual rasters of GDAL linked raster maps on NFS

Hi,

I am experiencing significant performance issues with virtual rasters build with r.buildvrt over GDAL-linked (r.external) raster maps (source is in GeoTiff format) on NFS.

The GeoTiff data are located on a relatively fast NFS network storage, while the GRASS DB is located on a somewhat slower NFS storage system.

In order to drill down to the issue I ran r.univar on three GDAL linked raster maps that cover my computational region. The process which consists of three r.univar runs takes ~40 seconds.

Using the same computational region, one r.univar on a virtual raster of the same three raster map has not finished after 30 minutes. After converting the GDAL linked maps to native GRASS format (on the slower NFS) I built a VRT over the GRASS GIS format raster maps. For that VRT, performance is almost identical to to r.univar onr the individual raster maps (= no VRT). So it seems the issue is reading GDAL linked raster maps through GRASS VRTs.

Is that a limitation of virtual rasters in GRASS GIS or is it a bug in how VRTs are read? Is there a point to open an issue for this?
Currently, VRT with GDAL linked data seems practically unusable… Any suggestions on what can be done is very welcome. Would be a pity if I had to abandon VRTs…

Cheers,
Stefan

_______________________________________________ grass-user mailing list grass-user@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-user