[GRASS-user] i.segment.stats memory usage error

GRASS GRASS user
21

On 03/11/16 10:58, Moritz Lennert wrote:

On 02/11/16 17:15, Markus Metz wrote:

On Wed, Nov 2, 2016 at 3:47 PM, Moritz Lennert
<mlennert@club.worldonline.be> wrote:

On 02/11/16 11:05, James Duffy wrote:

I can't register with osgeo to submit a bug as no-one has replied with a
'mantra' for me to do so... what a convoluted bug reporting system!

And in your opinion Moritz, does it look like my workflow will not be
possible unless I find a 64bit machine with a decent amount of RAM?

I'm not sure about the necessity of 64bit, but more RAM probably.

IIUC, all the stats of all the zones are put into memory during the run of
the module. Maybe a file-based store of the information might be possible
which would avoid this memory usage, but I'm not sure.

@MarkusM: could you have a look at this ? See
https://lists.osgeo.org/pipermail/grass-user/2016-October/075493.html for
the beginning of the thread. In short: James seems to be hitting memory
limits of r.univar and I'm wondering out loud whether there is something
that could be done about this...

As Anna wrote, the -e flag increases memory consumption of r.univar
considerably. A file based approach for extended stats of r.univar is
difficult because the number of values per segment is initially
unknown to r.univar, thus the amount of disk space needed for extended
stats for each segment is unknown.

I would rather use r.univar for standard stats and r.stats.quantile
for extended stats. The options for raster_statistics of
i.segment.stats would then reduce to "min, max, range, mean,
mean_of_abs, stddev, variance, coeff_var, sum, sum_abs", and a new
option, e.g. raster_percentiles could be added, accepting a list of
percentiles which can be directly fed by i.segment.stats to
r.stats.quantile percentiles=<list of percentiles>.

Ok, I'll look into this. The only disadvantage is that r.stats.quantile
creates a new raster map for each percentile, which then have to be
treated to extract the statistics, instead of allowing to print the
results directly to a file. But I guess the overhead of that is worth
the fact that it reduces memory usage.

Forget that, I just saw the -p option. Need new glasses, I guess :wink:

Moritz

22

Moritz, have you seen the reply on the bug rep? https://trac.osgeo.org/grass/ticket/3197

Do you think the alternative r.univar2 could be used in your function?

···

On 3 November 2016 at 10:01, Moritz Lennert <mlennert@club.worldonline.be> wrote:

On 03/11/16 10:58, Moritz Lennert wrote:

On 02/11/16 17:15, Markus Metz wrote:

On Wed, Nov 2, 2016 at 3:47 PM, Moritz Lennert
<mlennert@club.worldonline.be> wrote:

On 02/11/16 11:05, James Duffy wrote:

I can’t register with osgeo to submit a bug as no-one has replied with a
‘mantra’ for me to do so… what a convoluted bug reporting system!

And in your opinion Moritz, does it look like my workflow will not be
possible unless I find a 64bit machine with a decent amount of RAM?

I’m not sure about the necessity of 64bit, but more RAM probably.

IIUC, all the stats of all the zones are put into memory during the run of
the module. Maybe a file-based store of the information might be possible
which would avoid this memory usage, but I’m not sure.

@MarkusM: could you have a look at this ? See
https://lists.osgeo.org/pipermail/grass-user/2016-October/075493.html for
the beginning of the thread. In short: James seems to be hitting memory
limits of r.univar and I’m wondering out loud whether there is something
that could be done about this…

As Anna wrote, the -e flag increases memory consumption of r.univar
considerably. A file based approach for extended stats of r.univar is
difficult because the number of values per segment is initially
unknown to r.univar, thus the amount of disk space needed for extended
stats for each segment is unknown.

I would rather use r.univar for standard stats and r.stats.quantile
for extended stats. The options for raster_statistics of
i.segment.stats would then reduce to “min, max, range, mean,
mean_of_abs, stddev, variance, coeff_var, sum, sum_abs”, and a new
option, e.g. raster_percentiles could be added, accepting a list of
percentiles which can be directly fed by i.segment.stats to
r.stats.quantile percentiles=.

Ok, I’ll look into this. The only disadvantage is that r.stats.quantile
creates a new raster map for each percentile, which then have to be
treated to extract the statistics, instead of allowing to print the
results directly to a file. But I guess the overhead of that is worth
the fact that it reduces memory usage.

Forget that, I just saw the -p option. Need new glasses, I guess :wink:

Moritz

–

James Duffy
PhD Researcher
Environment and Sustainability Institute
Penryn Campus
University of Exeter
Penryn
Cornwall
TR10 9FE

23

On Wed, Nov 2, 2016 at 5:15 PM, Markus Metz
<markus.metz.giswork@gmail.com> wrote:

On Wed, Nov 2, 2016 at 3:47 PM, Moritz Lennert
<mlennert@club.worldonline.be> wrote:

On 02/11/16 11:05, James Duffy wrote:

I can't register with osgeo to submit a bug as no-one has replied with a
'mantra' for me to do so... what a convoluted bug reporting system!

And in your opinion Moritz, does it look like my workflow will not be
possible unless I find a 64bit machine with a decent amount of RAM?

I'm not sure about the necessity of 64bit, but more RAM probably.

IIUC, all the stats of all the zones are put into memory during the run of
the module. Maybe a file-based store of the information might be possible
which would avoid this memory usage, but I'm not sure.

@MarkusM: could you have a look at this ? See
https://lists.osgeo.org/pipermail/grass-user/2016-October/075493.html for
the beginning of the thread. In short: James seems to be hitting memory
limits of r.univar and I'm wondering out loud whether there is something
that could be done about this...

As Anna wrote, the -e flag increases memory consumption of r.univar
considerably. A file based approach for extended stats of r.univar is
difficult because the number of values per segment is initially
unknown to r.univar, thus the amount of disk space needed for extended
stats for each segment is unknown.

I would rather use r.univar for standard stats and r.stats.quantile
for extended stats.

There were some bugs in r.stats.quantile, fixed in r69770-2 in all G7 branches.

About the limit to MAX_CATS categories in the base map (ticket #3198),
there is not really a reason for it. I guess it is meant to protect
against out-of-memory errors.

Markus M

The options for raster_statistics of
i.segment.stats would then reduce to "min, max, range, mean,
mean_of_abs, stddev, variance, coeff_var, sum, sum_abs", and a new
option, e.g. raster_percentiles could be added, accepting a list of
percentiles which can be directly fed by i.segment.stats to
r.stats.quantile percentiles=<list of percentiles>.

Markus M

24

On 04/11/16 12:24, James Duffy wrote:

Moritz, have you seen the reply on the bug rep?
https://trac.osgeo.org/grass/ticket/3197
<https://trac.osgeo.org/grass/ticket/3197&gt;

Do you think the alternative r.univar2 could be used in your function?

Yes, I saw the answer. As Pietro said, r.univar2 was not ported to grass7 and I don't have time for that now. I also don't really know why the module is faster, but ideally the changes should be merged into r.univar instead of creating a second, similar module.

Two years ago, we had an exchange about this [1], but there was no follow-up.

@Pietro: could you maybe explain how you changed the logic to see how difficult it would be to integrate this into r.univar ?

Moritz

[1] https://lists.osgeo.org/pipermail/grass-dev/2014-January/066823.html

25

On 04/11/16 14:29, Markus Metz wrote:

On Wed, Nov 2, 2016 at 5:15 PM, Markus Metz
<markus.metz.giswork@gmail.com> wrote:

On Wed, Nov 2, 2016 at 3:47 PM, Moritz Lennert
<mlennert@club.worldonline.be> wrote:

On 02/11/16 11:05, James Duffy wrote:

I can't register with osgeo to submit a bug as no-one has replied with a
'mantra' for me to do so... what a convoluted bug reporting system!

And in your opinion Moritz, does it look like my workflow will not be
possible unless I find a 64bit machine with a decent amount of RAM?

I'm not sure about the necessity of 64bit, but more RAM probably.

IIUC, all the stats of all the zones are put into memory during the run of
the module. Maybe a file-based store of the information might be possible
which would avoid this memory usage, but I'm not sure.

@MarkusM: could you have a look at this ? See
https://lists.osgeo.org/pipermail/grass-user/2016-October/075493.html for
the beginning of the thread. In short: James seems to be hitting memory
limits of r.univar and I'm wondering out loud whether there is something
that could be done about this...

As Anna wrote, the -e flag increases memory consumption of r.univar
considerably. A file based approach for extended stats of r.univar is
difficult because the number of values per segment is initially
unknown to r.univar, thus the amount of disk space needed for extended
stats for each segment is unknown.

I would rather use r.univar for standard stats and r.stats.quantile
for extended stats.

There were some bugs in r.stats.quantile, fixed in r69770-2 in all G7 branches.

Great, thanks !

About the limit to MAX_CATS categories in the base map (ticket #3198),
there is not really a reason for it. I guess it is meant to protect
against out-of-memory errors.

Does this mean we can just take out the lines referencing it ?

Moritz

26

On Fri, Nov 4, 2016 at 3:51 PM, Moritz Lennert
<mlennert@club.worldonline.be> wrote:

On 04/11/16 14:29, Markus Metz wrote:

On Wed, Nov 2, 2016 at 5:15 PM, Markus Metz
<markus.metz.giswork@gmail.com> wrote:

On Wed, Nov 2, 2016 at 3:47 PM, Moritz Lennert
<mlennert@club.worldonline.be> wrote:

On 02/11/16 11:05, James Duffy wrote:

I can't register with osgeo to submit a bug as no-one has replied with
a
'mantra' for me to do so... what a convoluted bug reporting system!

And in your opinion Moritz, does it look like my workflow will not be
possible unless I find a 64bit machine with a decent amount of RAM?

I'm not sure about the necessity of 64bit, but more RAM probably.

IIUC, all the stats of all the zones are put into memory during the run
of
the module. Maybe a file-based store of the information might be
possible
which would avoid this memory usage, but I'm not sure.

@MarkusM: could you have a look at this ? See
https://lists.osgeo.org/pipermail/grass-user/2016-October/075493.html
for
the beginning of the thread. In short: James seems to be hitting memory
limits of r.univar and I'm wondering out loud whether there is something
that could be done about this...

As Anna wrote, the -e flag increases memory consumption of r.univar
considerably. A file based approach for extended stats of r.univar is
difficult because the number of values per segment is initially
unknown to r.univar, thus the amount of disk space needed for extended
stats for each segment is unknown.

I would rather use r.univar for standard stats and r.stats.quantile
for extended stats.

There were some bugs in r.stats.quantile, fixed in r69770-2 in all G7
branches.

Great, thanks !

About the limit to MAX_CATS categories in the base map (ticket #3198),
there is not really a reason for it. I guess it is meant to protect
against out-of-memory errors.

Does this mean we can just take out the lines referencing it ?

Maybe replace the error with a warning. I tested with a base map with
1.4 million categories (output of i.segment) and it works, but I
needed to reduce the number of bins from 1000 to 100 to avoid OOM.
With a low number of bins, the memory consumption of r.stats.quantile
is much less than what r.univar requires, but a low number of bins can
in some cases (histogram of the cell values with a peak) lead to
higher memory consumption.

r.univar does memory reallocation on demand which means that for a
short time, double the amount of memory is needed. r.univar2 makes a
guess about how much memory is required and then allocates all at
once. Still, the binning approach of r.(stats.)quantile usually needs
less memory.

Markus M

27

Sorry to post again on this long thread…

I have just managed to get my hands on a 64bit osgeolive install with 12GB RAM available and the afformentioned function worked. Just to reiterate this was i.segment.stats on gp_seg_optimum_clump (which was the output of i.segment, run through r.clump). I haven’t created the vectormap, but it seems that the csv was created correctly.

I don’t know if it was the 64bit or the bit of extra RAM that made the difference here.

James

···

On 4 November 2016 at 15:34, Markus Metz <markus.metz.giswork@gmail.com> wrote:

On Fri, Nov 4, 2016 at 3:51 PM, Moritz Lennert

<mlennert@club.worldonline.be> wrote:

On 04/11/16 14:29, Markus Metz wrote:

On Wed, Nov 2, 2016 at 5:15 PM, Markus Metz
<markus.metz.giswork@gmail.com> wrote:

On Wed, Nov 2, 2016 at 3:47 PM, Moritz Lennert
<mlennert@club.worldonline.be> wrote:

On 02/11/16 11:05, James Duffy wrote:

I can’t register with osgeo to submit a bug as no-one has replied with
a
‘mantra’ for me to do so… what a convoluted bug reporting system!

And in your opinion Moritz, does it look like my workflow will not be
possible unless I find a 64bit machine with a decent amount of RAM?

I’m not sure about the necessity of 64bit, but more RAM probably.

IIUC, all the stats of all the zones are put into memory during the run
of
the module. Maybe a file-based store of the information might be
possible
which would avoid this memory usage, but I’m not sure.

@MarkusM: could you have a look at this ? See
https://lists.osgeo.org/pipermail/grass-user/2016-October/075493.html
for
the beginning of the thread. In short: James seems to be hitting memory
limits of r.univar and I’m wondering out loud whether there is something
that could be done about this…

As Anna wrote, the -e flag increases memory consumption of r.univar
considerably. A file based approach for extended stats of r.univar is
difficult because the number of values per segment is initially
unknown to r.univar, thus the amount of disk space needed for extended
stats for each segment is unknown.

I would rather use r.univar for standard stats and r.stats.quantile
for extended stats.

There were some bugs in r.stats.quantile, fixed in r69770-2 in all G7
branches.

Great, thanks !

About the limit to MAX_CATS categories in the base map (ticket #3198),
there is not really a reason for it. I guess it is meant to protect
against out-of-memory errors.

Does this mean we can just take out the lines referencing it ?

Maybe replace the error with a warning. I tested with a base map with
1.4 million categories (output of i.segment) and it works, but I
needed to reduce the number of bins from 1000 to 100 to avoid OOM.
With a low number of bins, the memory consumption of r.stats.quantile
is much less than what r.univar requires, but a low number of bins can
in some cases (histogram of the cell values with a peak) lead to
higher memory consumption.

r.univar does memory reallocation on demand which means that for a
short time, double the amount of memory is needed. r.univar2 makes a
guess about how much memory is required and then allocates all at
once. Still, the binning approach of r.(stats.)quantile usually needs
less memory.

Markus M

28

On Fri, Nov 4, 2016 at 3:51 PM, Moritz Lennert <mlennert@club.worldonline.be

wrote:

On 04/11/16 12:24, James Duffy wrote:

Moritz, have you seen the reply on the bug rep?
https://trac.osgeo.org/grass/ticket/3197
<https://trac.osgeo.org/grass/ticket/3197&gt;

Do you think the alternative r.univar2 could be used in your function?

Yes, I saw the answer. As Pietro said, r.univar2 was not ported to grass7
and I don't have time for that now. I also don't really know why the module
is faster, but ideally the changes should be merged into r.univar instead
of creating a second, similar module.

r.univar2 was developed for grass7. Anyway it seems that now there are
better tools for this task.

@Pietro: could you maybe explain how you changed the logic to see how
difficult it would be to integrate this into r.univar ?

It starts looking how many zones exists and how many cells per zone are
present in the region. In this way it allocates only what is needed, and
using the extended statistics free the memory after each zone is done.
Probably this free rule should be moved also in the case user is not
computing the extending statistics. In this way I was able to characterize
the zones of the i.segment module.

All the best

Pietro

29

On 07/11/16 08:16, Pietro wrote:

On Fri, Nov 4, 2016 at 3:51 PM, Moritz Lennert
<mlennert@club.worldonline.be <mailto:mlennert@club.worldonline.be>> wrote:

    On 04/11/16 12:24, James Duffy wrote:

        Moritz, have you seen the reply on the bug rep?
        https://trac.osgeo.org/grass/ticket/3197
        <https://trac.osgeo.org/grass/ticket/3197&gt;
        <https://trac.osgeo.org/grass/ticket/3197
        <https://trac.osgeo.org/grass/ticket/3197&gt;&gt;

        Do you think the alternative r.univar2 could be used in your
        function?

    Yes, I saw the answer. As Pietro said, r.univar2 was not ported to
    grass7 and I don't have time for that now. I also don't really know
    why the module is faster, but ideally the changes should be merged
    into r.univar instead of creating a second, similar module.

r.univar2 was developed for grass7. Anyway it seems that now there are
better tools for this task.

Sorry, I read too quickly over your phrase "It was not backported to trunk" in the ticket. So, it was developed for grass7.0, but not for trunk ?

    @Pietro: could you maybe explain how you changed the logic to see
    how difficult it would be to integrate this into r.univar ?

It starts looking how many zones exists and how many cells per zone are
present in the region. In this way it allocates only what is needed, and
using the extended statistics free the memory after each zone is done.
Probably this free rule should be moved also in the case user is not
computing the extending statistics. In this way I was able to
characterize the zones of the i.segment module.

Thanks for the explanation !

Moritz

30

Hi James,

On 04/11/16 22:08, James Duffy wrote:

Sorry to post again on this long thread...

No worries :slight_smile:

I have just managed to get my hands on a 64bit osgeolive install with
12GB RAM available and the afformentioned function worked. Just to
reiterate this was i.segment.stats on gp_seg_optimum_clump (which was
the output of i.segment, run through r.clump). I haven't created the
vectormap, but it seems that the csv was created correctly.

I don't know if it was the 64bit or the bit of extra RAM that made the
difference here.

I would guess the extra RAM, but am no expert on these questions.

I'm working on a revised version of i.segment.stats to use r.stats.quantile. It would be great if you could test that once its done.

Moritz

James

On 4 November 2016 at 15:34, Markus Metz <markus.metz.giswork@gmail.com
<mailto:markus.metz.giswork@gmail.com>> wrote:

    On Fri, Nov 4, 2016 at 3:51 PM, Moritz Lennert
    <mlennert@club.worldonline.be <mailto:mlennert@club.worldonline.be>>
    wrote:
    > On 04/11/16 14:29, Markus Metz wrote:
    >>
    >> On Wed, Nov 2, 2016 at 5:15 PM, Markus Metz
    >> <markus.metz.giswork@gmail.com
    <mailto:markus.metz.giswork@gmail.com>> wrote:
    >>>
    >>> On Wed, Nov 2, 2016 at 3:47 PM, Moritz Lennert
    >>> <mlennert@club.worldonline.be
    <mailto:mlennert@club.worldonline.be>> wrote:
    >>>>
    >>>> On 02/11/16 11:05, James Duffy wrote:
    >>>>>
    >>>>> I can't register with osgeo to submit a bug as no-one has
    replied with
    >>>>> a
    >>>>> 'mantra' for me to do so... what a convoluted bug reporting
    system!
    >>>>>
    >>>>> And in your opinion Moritz, does it look like my workflow will
    not be
    >>>>> possible unless I find a 64bit machine with a decent amount of
    RAM?
    >>>>
    >>>> I'm not sure about the necessity of 64bit, but more RAM probably.
    >>>>
    >>>> IIUC, all the stats of all the zones are put into memory during
    the run
    >>>> of
    >>>> the module. Maybe a file-based store of the information might be
    >>>> possible
    >>>> which would avoid this memory usage, but I'm not sure.
    >>>>
    >>>> @MarkusM: could you have a look at this ? See
    >>>>
    https://lists.osgeo.org/pipermail/grass-user/2016-October/075493.html
    >>>> for
    >>>> the beginning of the thread. In short: James seems to be
    hitting memory
    >>>> limits of r.univar and I'm wondering out loud whether there is
    something
    >>>> that could be done about this...
    >>>
    >>> As Anna wrote, the -e flag increases memory consumption of r.univar
    >>> considerably. A file based approach for extended stats of
    r.univar is
    >>> difficult because the number of values per segment is initially
    >>> unknown to r.univar, thus the amount of disk space needed for
    extended
    >>> stats for each segment is unknown.
    >>>
    >>> I would rather use r.univar for standard stats and r.stats.quantile
    >>> for extended stats.
    >>
    >> There were some bugs in r.stats.quantile, fixed in r69770-2 in all G7
    >> branches.
    >
    > Great, thanks !
    >
    >>
    >> About the limit to MAX_CATS categories in the base map (ticket
    #3198),
    >> there is not really a reason for it. I guess it is meant to protect
    >> against out-of-memory errors.
    >
    > Does this mean we can just take out the lines referencing it ?

    Maybe replace the error with a warning. I tested with a base map with
    1.4 million categories (output of i.segment) and it works, but I
    needed to reduce the number of bins from 1000 to 100 to avoid OOM.
    With a low number of bins, the memory consumption of r.stats.quantile
    is much less than what r.univar requires, but a low number of bins can
    in some cases (histogram of the cell values with a peak) lead to
    higher memory consumption.

    r.univar does memory reallocation on demand which means that for a
    short time, double the amount of memory is needed. r.univar2 makes a
    guess about how much memory is required and then allocates all at
    once. Still, the binning approach of r.(stats.)quantile usually needs
    less memory.

    Markus M

31

On 7 November 2016 at 14:13, Moritz Lennert <mlennert@club.worldonline.be>
wrote:

Hi James,

On 04/11/16 22:08, James Duffy wrote:

Sorry to post again on this long thread...

No worries :slight_smile:

I have just managed to get my hands on a 64bit osgeolive install with
12GB RAM available and the afformentioned function worked. Just to
reiterate this was i.segment.stats on gp_seg_optimum_clump (which was
the output of i.segment, run through r.clump). I haven't created the
vectormap, but it seems that the csv was created correctly.

I don't know if it was the 64bit or the bit of extra RAM that made the
difference here.

I would guess the extra RAM, but am no expert on these questions.

I'm working on a revised version of i.segment.stats to use
r.stats.quantile. It would be great if you could test that once its done.

Yes, happy to test it out for you. Please let me know when to reinstall the
extension.

Moritz

James

On 4 November 2016 at 15:34, Markus Metz <markus.metz.giswork@gmail.com
<mailto:markus.metz.giswork@gmail.com>> wrote:

    On Fri, Nov 4, 2016 at 3:51 PM, Moritz Lennert
    <mlennert@club.worldonline.be <mailto:mlennert@club.worldonline.be>>
    wrote:
    > On 04/11/16 14:29, Markus Metz wrote:
    >>
    >> On Wed, Nov 2, 2016 at 5:15 PM, Markus Metz
    >> <markus.metz.giswork@gmail.com
    <mailto:markus.metz.giswork@gmail.com>> wrote:
    >>>
    >>> On Wed, Nov 2, 2016 at 3:47 PM, Moritz Lennert
    >>> <mlennert@club.worldonline.be
    <mailto:mlennert@club.worldonline.be>> wrote:
    >>>>
    >>>> On 02/11/16 11:05, James Duffy wrote:
    >>>>>
    >>>>>
    >>>>> I can't register with osgeo to submit a bug as no-one has
    replied with
    >>>>> a
    >>>>> 'mantra' for me to do so... what a convoluted bug reporting
    system!
    >>>>>
    >>>>> And in your opinion Moritz, does it look like my workflow will
    not be
    >>>>> possible unless I find a 64bit machine with a decent amount of
    RAM?
    >>>>
    >>>>
    >>>>
    >>>> I'm not sure about the necessity of 64bit, but more RAM probably.
    >>>>
    >>>> IIUC, all the stats of all the zones are put into memory during
    the run
    >>>> of
    >>>> the module. Maybe a file-based store of the information might be
    >>>> possible
    >>>> which would avoid this memory usage, but I'm not sure.
    >>>>
    >>>> @MarkusM: could you have a look at this ? See
    >>>>
    https://lists.osgeo.org/pipermail/grass-user/2016-October/075493.html
<https://lists.osgeo.org/pipermail/grass-user/2016-October/075493.html&gt;

    >>>> for
    >>>> the beginning of the thread. In short: James seems to be
    hitting memory
    >>>> limits of r.univar and I'm wondering out loud whether there is
    something
    >>>> that could be done about this...
    >>>
    >>>
    >>> As Anna wrote, the -e flag increases memory consumption of
r.univar
    >>> considerably. A file based approach for extended stats of
    r.univar is
    >>> difficult because the number of values per segment is initially
    >>> unknown to r.univar, thus the amount of disk space needed for
    extended
    >>> stats for each segment is unknown.
    >>>
    >>> I would rather use r.univar for standard stats and
r.stats.quantile
    >>> for extended stats.
    >>
    >>
    >> There were some bugs in r.stats.quantile, fixed in r69770-2 in all
G7
    >> branches.
    >
    >
    > Great, thanks !
    >
    >>
    >> About the limit to MAX_CATS categories in the base map (ticket
    #3198),
    >> there is not really a reason for it. I guess it is meant to protect
    >> against out-of-memory errors.
    >
    >
    > Does this mean we can just take out the lines referencing it ?

    Maybe replace the error with a warning. I tested with a base map with
    1.4 million categories (output of i.segment) and it works, but I
    needed to reduce the number of bins from 1000 to 100 to avoid OOM.
    With a low number of bins, the memory consumption of r.stats.quantile
    is much less than what r.univar requires, but a low number of bins can
    in some cases (histogram of the cell values with a peak) lead to
    higher memory consumption.

    r.univar does memory reallocation on demand which means that for a
    short time, double the amount of memory is needed. r.univar2 makes a
    guess about how much memory is required and then allocates all at
    once. Still, the binning approach of r.(stats.)quantile usually needs
    less memory.

    Markus M

--
*James Duffy*
PhD Researcher
Environment and Sustainability Institute
Penryn Campus
University of Exeter
Penryn
Cornwall
TR10 9FE