[GRASS-user] Calculate average of 3 highest values for a set of raster maps

I have a set of 7 raster maps.
I am able to calculate several stats with these mapas using r.series.
Now I need to calculate the mean of the tree highest values of my 7 maps.
Any idea of how can I do that?

cheers
--
Miltinho - mcr@rc.unesp.br
Laboratório de Ecologia Espacial e Conservação - LEEC
Depto de Ecologia - UNESP - Rio Claro
Av. 24A, 1515- Bela Vista
13506-900 Rio Claro, SP, Brasil

Fone: +55 19 3526-9647 (office) 19 3526-9680 (lab)
Cel: 19 9853-3220 / 19 9853-5430

Depto Ecologia http://www.rc.unesp.br/ib/ecologia/

PG ECO & BIODIV http://www.rc.unesp.br/ib/ecologia/posbiodiversidade/index.php

CV http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K4792988H6&mostrarNroCitacoesISI=true&mostrarNroCitacoesScopus=true

Google citations http://scholar.google.com/citations?user=OWX_2eAAAAAJ

Hi Milton,

A possible approach could be, for example, to do the following with each map:

r.stats -1 input=map1 output=cellvaluesmap1

It will create a file containing the cell values for each map. Then you can read this file into R and average the three highest values. Something like:

values.map1<-read.table(“cellvaluesmap1”)
threehighest<-values.map1$V1[order(values.map1,decreasing=FALSE)][1:3]
mean3highest<-mean(threehighest)

Hope it helps. Success

Miguel

2013/3/15 Milton Cezar Ribeiro <miltinho.astronauta@gmail.com>

I have a set of 7 raster maps.
I am able to calculate several stats with these mapas using r.series.
Now I need to calculate the mean of the tree highest values of my 7 maps.
Any idea of how can I do that?

cheers

Miltinho - mcr@rc.unesp.br
Laboratório de Ecologia Espacial e Conservação - LEEC
Depto de Ecologia - UNESP - Rio Claro
Av. 24A, 1515- Bela Vista
13506-900 Rio Claro, SP, Brasil

Fone: +55 19 3526-9647 (office) 19 3526-9680 (lab)
Cel: 19 9853-3220 / 19 9853-5430

Depto Ecologia http://www.rc.unesp.br/ib/ecologia/

PG ECO & BIODIV http://www.rc.unesp.br/ib/ecologia/posbiodiversidade/index.php

CV http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K4792988H6&mostrarNroCitacoesISI=true&mostrarNroCitacoesScopus=true

Google citations http://scholar.google.com/citations?user=OWX_2eAAAAAJ


grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[apologies for taking the liberty to re-sort previous posts]

Milton Cezar Ribeiro:

> I have a set of 7 raster maps.
> I am able to calculate several stats with these mapas using r.series.
> Now I need to calculate the mean of the tree highest values of my 7 maps.
> Any idea of how can I do that?

José Miguel Barrios wrote:

A possible approach could be, for example, to do the following with each
map:

r.stats -1 input=map1 output=cellvaluesmap1

It will create a file containing the cell values for each map. Then you
can read this file into R and average the three highest values. Something
like:

values.map1<-read.table("cellvaluesmap1")
threehighest<-values.map1$V1[order(values.map1,decreasing=FALSE)][1:3]
mean3highest<-mean(threehighest)

If in Linux, there are several ways to do this. One (very) nice approach as
shown in <http://stackoverflow.com/a/3096575/1172302&gt;, could be:

# get the numbers
r.stats -1 input=map1 output=Values1

# sort and keep first 3
sort -rn Values1 | head -3 > Values1.sorted

# sum and divide by 3
echo "( $( paste -sd+ Values1.sorted ) ) / 3" | bc

# or a one-liner!
sort -rn numbers | head -3 | echo "( $( paste -sd+ ) ) / 3" | bc

Other options would be "awk" or "python". Also, note that r.stats has also
the "-x" and "-g" flags. You could grab them as well and, if doing the math
"externally", know where to place the results each time?

However, it seems like a very "resources" hungry approach, if your maps are
many and large. There must be a "smart"(er) way to do this directly using
r.mapcalc...

Best, Nikos

Hi Miltinho

I assume you want the mean of the three highest values for each raster cell? If so, you can do this using another step with r.series

1) If your 7 maps are a1 .. a7:

mapcalc "a1 = 1"; r.mapcalc "a2 = 2"; r.mapcalc "a3 = 3"; r.mapcalc "a4 = 4"; r.mapcalc "a5 = 5"; r.mapcalc "a6 = 6"; r.mapcalc "a7 = 7"

2) calculate the 4/7 quantile. Actually, it should be a little bit smaller to make sure you include your 3rd largest layer. In this example below, 0.57 as approximation is fine

r.series --overwrite input=a1,a2,a3,a4,a5,a6,a7 quantile=0.57 output=res1 method=quantile

2) Next, for all 7 layers, set all values < 4/7 quantile to 0

r.mapcalc "b1 = if(a1>res1,a1,null())"
r.mapcalc "b2 = if(a2>res1,a2,null())"
r.mapcalc "b3 = if(a3>res1,a3,null())"
r.mapcalc "b4 = if(a4>res1,a4,null())"
r.mapcalc "b5 = if(a5>res1,a5,null())"
r.mapcalc "b6 = if(a6>res1,a6,null())"
r.mapcalc "b7 = if(a7>res1,a7,null())"

3) Now, calculate the sum of the 7 new layers

r.series input=b1,b2,b3,b4,b5,b6,b7 method=average output=res2 --overwrite

Your output res2 should have a value 6 for all raster cells.

Cheers,

Paulo

On Fri 15 Mar 2013 03:57:35 AM CET, Milton Cezar Ribeiro wrote:

I have a set of 7 raster maps.
I am able to calculate several stats with these mapas using r.series.
Now I need to calculate the mean of the tree highest values of my 7 maps.
Any idea of how can I do that?

cheers

Sorry, some typo's (but the given code should be ok): set all values < 4/7 quantile to null(), not 0, and you want to calculate the average, not the sum

On Fri 15 Mar 2013 10:05:51 AM CET, Paulo van Breugel wrote:

Hi Miltinho

I assume you want the mean of the three highest values for each raster
cell? If so, you can do this using another step with r.series

1) If your 7 maps are a1 .. a7:

mapcalc "a1 = 1"; r.mapcalc "a2 = 2"; r.mapcalc "a3 = 3"; r.mapcalc
"a4 = 4"; r.mapcalc "a5 = 5"; r.mapcalc "a6 = 6"; r.mapcalc "a7 = 7"

2) calculate the 4/7 quantile. Actually, it should be a little bit
smaller to make sure you include your 3rd largest layer. In this
example below, 0.57 as approximation is fine

r.series --overwrite input=a1,a2,a3,a4,a5,a6,a7 quantile=0.57
output=res1 method=quantile

2) Next, for all 7 layers, set all values < 4/7 quantile to 0

r.mapcalc "b1 = if(a1>res1,a1,null())"
r.mapcalc "b2 = if(a2>res1,a2,null())"
r.mapcalc "b3 = if(a3>res1,a3,null())"
r.mapcalc "b4 = if(a4>res1,a4,null())"
r.mapcalc "b5 = if(a5>res1,a5,null())"
r.mapcalc "b6 = if(a6>res1,a6,null())"
r.mapcalc "b7 = if(a7>res1,a7,null())"

3) Now, calculate the sum of the 7 new layers

r.series input=b1,b2,b3,b4,b5,b6,b7 method=average output=res2
--overwrite

Your output res2 should have a value 6 for all raster cells.

Cheers,

Paulo

On Fri 15 Mar 2013 03:57:35 AM CET, Milton Cezar Ribeiro wrote:

I have a set of 7 raster maps.
I am able to calculate several stats with these mapas using r.series.
Now I need to calculate the mean of the tree highest values of my 7
maps.
Any idea of how can I do that?

cheers

Dear all,

Thanks for all replies. I will check it out thoughout the weekend.
Just clarifying something, I need to do the estimations focusing on
each pixels, and not for the full map (I was not that clear before).

Also as my raster maps are very large (~50,000x60,000 pixels) maybe R
(at least on w7) maybe will not handle the seven rasters.

Best wishes

miltinho

2013/3/15, Paulo van Breugel <p.vanbreugel@gmail.com>:

Sorry, some typo's (but the given code should be ok): set all values <
4/7 quantile to null(), not 0, and you want to calculate the average,
not the sum

On Fri 15 Mar 2013 10:05:51 AM CET, Paulo van Breugel wrote:

Hi Miltinho

I assume you want the mean of the three highest values for each raster
cell? If so, you can do this using another step with r.series

1) If your 7 maps are a1 .. a7:

mapcalc "a1 = 1"; r.mapcalc "a2 = 2"; r.mapcalc "a3 = 3"; r.mapcalc
"a4 = 4"; r.mapcalc "a5 = 5"; r.mapcalc "a6 = 6"; r.mapcalc "a7 = 7"

2) calculate the 4/7 quantile. Actually, it should be a little bit
smaller to make sure you include your 3rd largest layer. In this
example below, 0.57 as approximation is fine

r.series --overwrite input=a1,a2,a3,a4,a5,a6,a7 quantile=0.57
output=res1 method=quantile

2) Next, for all 7 layers, set all values < 4/7 quantile to 0

r.mapcalc "b1 = if(a1>res1,a1,null())"
r.mapcalc "b2 = if(a2>res1,a2,null())"
r.mapcalc "b3 = if(a3>res1,a3,null())"
r.mapcalc "b4 = if(a4>res1,a4,null())"
r.mapcalc "b5 = if(a5>res1,a5,null())"
r.mapcalc "b6 = if(a6>res1,a6,null())"
r.mapcalc "b7 = if(a7>res1,a7,null())"

3) Now, calculate the sum of the 7 new layers

r.series input=b1,b2,b3,b4,b5,b6,b7 method=average output=res2
--overwrite

Your output res2 should have a value 6 for all raster cells.

Cheers,

Paulo

On Fri 15 Mar 2013 03:57:35 AM CET, Milton Cezar Ribeiro wrote:

I have a set of 7 raster maps.
I am able to calculate several stats with these mapas using r.series.
Now I need to calculate the mean of the tree highest values of my 7
maps.
Any idea of how can I do that?

cheers

--
Miltinho - mcr@rc.unesp.br
Laboratório de Ecologia Espacial e Conservação - LEEC
Depto de Ecologia - UNESP - Rio Claro
Av. 24A, 1515- Bela Vista
13506-900 Rio Claro, SP, Brasil

Fone: +55 19 3526-9647 (office) 19 3526-9680 (lab)
Cel: 19 9853-3220 / 19 9853-5430

Depto Ecologia http://www.rc.unesp.br/ib/ecologia/

PG ECO & BIODIV http://www.rc.unesp.br/ib/ecologia/posbiodiversidade/index.php

CV http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K4792988H6&mostrarNroCitacoesISI=true&mostrarNroCitacoesScopus=true

Google citations http://scholar.google.com/citations?user=OWX_2eAAAAAJ

My suggestion does just that, per pixel calculations. And as it is just simple GRASS functions, you should have no problem running it (perhaps a bit of patients depending on your computer).

···

On Fri, Mar 15, 2013 at 12:13 PM, Milton Cezar Ribeiro <miltinho.astronauta@gmail.com> wrote:

Dear all,

Thanks for all replies. I will check it out thoughout the weekend.
Just clarifying something, I need to do the estimations focusing on
each pixels, and not for the full map (I was not that clear before).

Also as my raster maps are very large (~50,000x60,000 pixels) maybe R
(at least on w7) maybe will not handle the seven rasters.

Best wishes

miltinho

2013/3/15, Paulo van Breugel <p.vanbreugel@gmail.com>:

Sorry, some typo’s (but the given code should be ok): set all values <
4/7 quantile to null(), not 0, and you want to calculate the average,
not the sum

On Fri 15 Mar 2013 10:05:51 AM CET, Paulo van Breugel wrote:

Hi Miltinho

I assume you want the mean of the three highest values for each raster
cell? If so, you can do this using another step with r.series

  1. If your 7 maps are a1 … a7:

mapcalc “a1 = 1”; r.mapcalc “a2 = 2”; r.mapcalc “a3 = 3”; r.mapcalc
“a4 = 4”; r.mapcalc “a5 = 5”; r.mapcalc “a6 = 6”; r.mapcalc “a7 = 7”

  1. calculate the 4/7 quantile. Actually, it should be a little bit
    smaller to make sure you include your 3rd largest layer. In this
    example below, 0.57 as approximation is fine

r.series --overwrite input=a1,a2,a3,a4,a5,a6,a7 quantile=0.57
output=res1 method=quantile

  1. Next, for all 7 layers, set all values < 4/7 quantile to 0

r.mapcalc “b1 = if(a1>res1,a1,null())”
r.mapcalc “b2 = if(a2>res1,a2,null())”
r.mapcalc “b3 = if(a3>res1,a3,null())”
r.mapcalc “b4 = if(a4>res1,a4,null())”
r.mapcalc “b5 = if(a5>res1,a5,null())”
r.mapcalc “b6 = if(a6>res1,a6,null())”
r.mapcalc “b7 = if(a7>res1,a7,null())”

  1. Now, calculate the sum of the 7 new layers

r.series input=b1,b2,b3,b4,b5,b6,b7 method=average output=res2
–overwrite

Your output res2 should have a value 6 for all raster cells.

Cheers,

Paulo

On Fri 15 Mar 2013 03:57:35 AM CET, Milton Cezar Ribeiro wrote:

I have a set of 7 raster maps.
I am able to calculate several stats with these mapas using r.series.
Now I need to calculate the mean of the tree highest values of my 7
maps.
Any idea of how can I do that?

cheers


Miltinho - mcr@rc.unesp.br
Laboratório de Ecologia Espacial e Conservação - LEEC
Depto de Ecologia - UNESP - Rio Claro
Av. 24A, 1515- Bela Vista
13506-900 Rio Claro, SP, Brasil

Fone: +55 19 3526-9647 (office) 19 3526-9680 (lab)
Cel: 19 9853-3220 / 19 9853-5430

Depto Ecologia http://www.rc.unesp.br/ib/ecologia/

PG ECO & BIODIV http://www.rc.unesp.br/ib/ecologia/posbiodiversidade/index.php

CV http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K4792988H6&mostrarNroCitacoesISI=true&mostrarNroCitacoesScopus=true

Google citations http://scholar.google.com/citations?user=OWX_2eAAAAAJ

Hi Milton,

On Friday 15 Mar 2013 08:13:16 Milton Cezar Ribeiro wrote:

Dear all,

Thanks for all replies. I will check it out thoughout the weekend.
Just clarifying something, I need to do the estimations focusing on
each pixels, and not for the full map (I was not that clear before).

In GRASS7 you can use the RasterRow or RasterRowIO class of pygrass, something
like:

{{{
#!python
import numpy as np

from grass.pygrass.raster import RasterRow
from grass.pygrass.gis.region import Region
from grass.pygrass.functions import coor2pixel

# define the list of points that you want
POINTS = [(633042.213884, 226283.771107),
          (641197.93621, 225675.891182),
          (644034.709193, 220711.538462),
          (630028.142589, 224764.071295),
          (630002.814259, 215037.992495), ]

NEIGHS = np.array([(-1, 1),
                   (0, 1),
                   (1, 1),
                   (1, 0),
                   (1, -1),
                   (0, -1),
                   (-1, -1),
                   (-1, 0),
                   (-1, 1)])

def get_neighs(point, region, neighs=NEIGHS):
    res = point - neighs
    valid = ((0 <= res.T[0]) * (res.T[0] < region.rows) *
             (0 <= res.T[1]) * (res.T[1] < region.cols))
    return res[valid]

def do_something(rastname, points, mapset=''):
    # get the current region
    region = Region()
    raster1 = RasterRow(rastname, mapset)
    raster1.open('r')
    # instantiate more raster object here if you need
    average =
    for pnt in points:
        coor = coor2pixel(pnt, region)
        neighs = get_neighs(coor, region)
        sum_ = 0
        for nx, ny in neighs:
            # do something with the neighbour value
            sum_ += raster1[int(nx)][int(ny)]
        average.append(sum_ / len(neighs))
    raster1.close()
    # close the other rasters
    return average

print do_something('elevation', POINTS, mapset='user1')
}}}

it's working on North Carolina mapset... maybe I should add get_neighs into
the pygrass.functions module...
whit the same logic you can open more raster at the same time

Also as my raster maps are very large (~50,000x60,000 pixels) maybe R
(at least on w7) maybe will not handle the seven rasters.

I've worked with a 10**5x10**5 without problems [0].

Pietro

[0] http://www.mdpi.com/2220-9964/2/1/201

Thanks Pietro and Paulo
I will try all your suggestions.

cheers

milton

2013/3/15, Pietro <peter.zamb@gmail.com>:

Hi Milton,

On Friday 15 Mar 2013 08:13:16 Milton Cezar Ribeiro wrote:

Dear all,

Thanks for all replies. I will check it out thoughout the weekend.
Just clarifying something, I need to do the estimations focusing on
each pixels, and not for the full map (I was not that clear before).

In GRASS7 you can use the RasterRow or RasterRowIO class of pygrass,
something
like:

{{{
#!python
import numpy as np

from grass.pygrass.raster import RasterRow
from grass.pygrass.gis.region import Region
from grass.pygrass.functions import coor2pixel

# define the list of points that you want
POINTS = [(633042.213884, 226283.771107),
          (641197.93621, 225675.891182),
          (644034.709193, 220711.538462),
          (630028.142589, 224764.071295),
          (630002.814259, 215037.992495), ]

NEIGHS = np.array([(-1, 1),
                   (0, 1),
                   (1, 1),
                   (1, 0),
                   (1, -1),
                   (0, -1),
                   (-1, -1),
                   (-1, 0),
                   (-1, 1)])

def get_neighs(point, region, neighs=NEIGHS):
    res = point - neighs
    valid = ((0 <= res.T[0]) * (res.T[0] < region.rows) *
             (0 <= res.T[1]) * (res.T[1] < region.cols))
    return res[valid]

def do_something(rastname, points, mapset=''):
    # get the current region
    region = Region()
    raster1 = RasterRow(rastname, mapset)
    raster1.open('r')
    # instantiate more raster object here if you need
    average =
    for pnt in points:
        coor = coor2pixel(pnt, region)
        neighs = get_neighs(coor, region)
        sum_ = 0
        for nx, ny in neighs:
            # do something with the neighbour value
            sum_ += raster1[int(nx)][int(ny)]
        average.append(sum_ / len(neighs))
    raster1.close()
    # close the other rasters
    return average

print do_something('elevation', POINTS, mapset='user1')
}}}

it's working on North Carolina mapset... maybe I should add get_neighs into
the pygrass.functions module...
whit the same logic you can open more raster at the same time

Also as my raster maps are very large (~50,000x60,000 pixels) maybe R
(at least on w7) maybe will not handle the seven rasters.

I've worked with a 10**5x10**5 without problems [0].

Pietro

[0] http://www.mdpi.com/2220-9964/2/1/201

--
Miltinho - mcr@rc.unesp.br
Laboratório de Ecologia Espacial e Conservação - LEEC
Depto de Ecologia - UNESP - Rio Claro
Av. 24A, 1515- Bela Vista
13506-900 Rio Claro, SP, Brasil

Fone: +55 19 3526-9647 (office) 19 3526-9680 (lab)
Cel: 19 9853-3220 / 19 9853-5430

Depto Ecologia http://www.rc.unesp.br/ib/ecologia/

PG ECO & BIODIV http://www.rc.unesp.br/ib/ecologia/posbiodiversidade/index.php

CV http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K4792988H6&mostrarNroCitacoesISI=true&mostrarNroCitacoesScopus=true

Google citations http://scholar.google.com/citations?user=OWX_2eAAAAAJ