[GRASS-dev] [GRASS GIS] #2045: r.to.vect: use less memory

#2045: r.to.vect: use less memory
--------------------------+-------------------------
  Reporter: mlennert | Owner: grass-dev@…
      Type: enhancement | Status: new
  Priority: normal | Milestone: 7.0.5
Component: Raster | Version: svn-trunk
Resolution: | Keywords: r.to.vect
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------

Comment (by mmetz):

Replying to [comment:21 mlennert]:
> Replying to [comment:20 mmetz]:
> > Replying to [comment:19 mlennert]:
> > >
> > > Data and info provided via private mail.
> >
> > I have received the data, thanks!
> >
> > >
> > > Previous posters have also reported that setting GRASS_VECTOR_LOWMEM
didn't make much of a difference, so maybe the problem is there ?
> >
> > I don't think so. Without GRASS_VECTOR_LOWMEM, v.build needs about 26
GB of RAM. With GRASS_VECTOR_LOWMEM, v.build needs about 11 GB of RAM but
is about 4.5 times slower, that is expected because some temporary data
are kept on disk and not in memory.
>
> Why does it still need 11GB of RAM when working with disk-based data ?
Is this really unavoidable ?

Technically, this is not unavoidable. Main topology and the category index
could also be managed on disk, but this is not easy to implement. You
could ask for enhancements for GRASS 8.
>
> GRASS has always had a tradition of allowing to work on limited
resources.

With the exception of vector topology which requires a lot of resources.
RAM consumption for vector topology has been substantially reduced in
GRASS 7 compared to GRASS 6, independent of the GRASS_VECTOR_LOWMEM
switch.

> Even though RAM is cheap, I would still consider > 11GB as rather high-
end in many offices. I wouldn't mind if it makes the command much slower,
but it would be nice if it could get the job done with less RAM usage...

Something for GRASS 8+. Also, a vector with >17 million areas is rather
unusual. For example, this vector can not be exported as a shapefile
because "The Shapefile format explicitly uses 32bit offsets" and I
estimate the *.shp file to be >10GB.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2045#comment:22&gt;
GRASS GIS <https://grass.osgeo.org>