[GRASS-dev] [GRASS GIS] #2735: t.rast.mapcalc input problem

GDAL · September 9, 2015, 7:12pm

#2735: t.rast.mapcalc input problem
------------------------+-------------------------
Reporter: leohardtke | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.1.0
Component: Default | Version: svn-trunk
Keywords: | CPU: Unspecified
Platform: Linux |
------------------------+-------------------------
I'm facing a problem with an expression in the t.rast.mapcalc module which
I think is a Bug...

t.rast.mapcalc inputs=QA_mask,ndvi,ndvi_smooth_spacetime
expression="if(QA_mask==0,ndvi,ndvi_smooth_spacetime)" output=ndvi_fixed
basename=ndvi_fixed
It throws the following error:
Starting temporal sampling...
Starting mapcalc computation...
Invalid map
<MOD13Q1.A2009017.h12v12.005.2009035110230_01_4326_smooth_spacetime_0>
Parse error
ERROR: parse error
ERROR: Error while mapcalc computation

The QA_mask maps are:
QA_mask_01|PERMANENT|2009-01-01 00:00:00|2009-01-17 00:00:00
QA_mask_02|PERMANENT|2009-01-17 00:00:00|2009-02-02 00:00:00
......

The ndvi_smooth_spacetime maps are:
ndvi_smooth_spacetime_0|PERMANENT|2009-01-17 00:00:00|2009-02-02 00:00:00
ndvi_smooth_spacetime_1|PERMANENT|2009-02-02 00:00:00|2009-02-18 00:00:00
...........

The ndvi maps are
MOD13Q1.A2009001.h12v12.005.2009019145228_01_4326|PERMANENT|2009-01-01
00:00:00|2009-01-17 00:00:00
MOD13Q1.A2009017.h12v12.005.2009035110230_01_4326|PERMANENT|2009-01-17
00:00:00|2009-02-02 00:00:00
M

As you see, there is no such a map like..
MOD13Q1.A2009017.h12v12.005.2009035110230_01_4326_smooth_spacetime_0

I guess the module is mixing ndvi (a temporal dataset) with
_smooth_spacetime... the second part of the name....

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2735>
GRASS GIS <https://grass.osgeo.org>

GDAL · September 9, 2015, 7:22pm

#2735: t.rast.mapcalc input problem
--------------------------+-------------------------
  Reporter: leohardtke | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Default | Version: svn-trunk
Resolution: | Keywords:
       CPU: Unspecified | Platform: Linux
--------------------------+-------------------------
Changes (by leohardtke):

* milestone: 7.1.0 =>

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2735#comment:1>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 6, 2015, 7:26pm

#2735: t.rast.mapcalc input problem
--------------------------+----------------------------
  Reporter: leohardtke | Owner: grass-dev@…
      Type: defect | Status: closed
  Priority: normal | Milestone: 7.0.1
Component: Temporal | Version: svn-trunk
Resolution: fixed | Keywords: t.rast.mapcalc
       CPU: Unspecified | Platform: Linux
--------------------------+----------------------------
Changes (by neteler):

* status: new => closed
* component: Default => Temporal
* resolution: => fixed
* keywords: => t.rast.mapcalc
* milestone: => 7.0.1

Comment:

Fixed in r66308, tested and backported in r66313. Closing.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2735#comment:2>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 7, 2015, 8:33am

#2735: t.rast.mapcalc input problem
--------------------------+----------------------------
  Reporter: leohardtke | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: normal | Milestone: 7.0.1
Component: Temporal | Version: svn-trunk
Resolution: | Keywords: t.rast.mapcalc
       CPU: Unspecified | Platform: Linux
--------------------------+----------------------------
Changes (by huhabla):

* status: closed => reopened
* resolution: fixed =>

Comment:

This bug was not fixed in t.rast.mapcalc. A similar bug was fixed in
t.rast.extract.

However, fixing the bug in t.rast.mapcalc is more difficult since it uses
a simple search and replace approach to substitute the input STRDS with
map names and is not aware of what a space time dataset is. If the names
of input and output space time datasets are include each other, then the
module does not work properly. Choosing a specific order of the input
STRDS in the input parameter (i.e: input=ndvi_smooth,QA_mask,ndvi) may
reduce the risk of wrong substitution.

However, please use t.rast.algebra instead. This module has other issues
but is able to correctly identify space time datasets.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2735#comment:3>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 15, 2015, 7:08pm

#2735: t.rast.mapcalc input problem
--------------------------+----------------------------
  Reporter: leohardtke | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: normal | Milestone: 7.0.1
Component: Temporal | Version: svn-trunk
Resolution: | Keywords: t.rast.mapcalc
       CPU: Unspecified | Platform: Linux
--------------------------+----------------------------

Comment (by dylan):

Replying to [comment:3 huhabla]:
> This bug was not fixed in t.rast.mapcalc. A similar bug was fixed in
t.rast.extract.
>
> However, fixing the bug in t.rast.mapcalc is more difficult since it
uses a simple search and replace approach to substitute the input STRDS
with map names and is not aware of what a space time dataset is. If the
names of input and output space time datasets are include each other, then
the module does not work properly. Choosing a specific order of the input
STRDS in the input parameter (i.e: input=ndvi_smooth,QA_mask,ndvi) may
reduce the risk of wrong substitution.
>
> However, please use t.rast.algebra instead. This module has other issues
but is able to correctly identify space time datasets.

Are there any other reasons to use `t.rast.algebra` over `t.rast.mapcalc`?

I don't know if [http://osgeo-org.1560.x6.nabble.com/Error-reading-raster-
data-for-row-xxx-only-when-using-r-series-and-t-rast-series-td5229569.html
this long-winded GRASS-dev post] is related, but I am finding what appear
to be randomly corrupted output maps from `t.rast.mapcalc` when the output
is FCELL or DCELL. No problems observed when output is CELL.

I will give `t.rast.algebra` a try and report back.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2735#comment:4>
GRASS GIS <https://grass.osgeo.org>

GDAL · December 19, 2015, 8:56pm

#2735: t.rast.mapcalc input problem
--------------------------+----------------------------
  Reporter: leohardtke | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: normal | Milestone: 7.0.3
Component: Temporal | Version: svn-trunk
Resolution: | Keywords: t.rast.mapcalc
       CPU: Unspecified | Platform: Linux
--------------------------+----------------------------
Changes (by neteler):

* milestone: 7.0.1 => 7.0.3

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2735#comment:5>
GRASS GIS <https://grass.osgeo.org>

GDAL · December 19, 2015, 9:38pm

#2735: t.rast.mapcalc input problem
--------------------------+----------------------------
  Reporter: leohardtke | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: normal | Milestone: 7.0.3
Component: Temporal | Version: svn-trunk
Resolution: | Keywords: t.rast.mapcalc
       CPU: Unspecified | Platform: Linux
--------------------------+----------------------------

Comment (by neteler):

Potential duplicate of #2659

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2735#comment:6>
GRASS GIS <https://grass.osgeo.org>

GDAL · January 28, 2016, 8:40pm

#2735: t.rast.mapcalc input problem
--------------------------+----------------------------
  Reporter: leohardtke | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: normal | Milestone: 7.0.4
Component: Temporal | Version: svn-trunk
Resolution: | Keywords: t.rast.mapcalc
       CPU: Unspecified | Platform: Linux
--------------------------+----------------------------

Comment (by neteler):

Replying to [comment:4 dylan]:
> I don't know if [http://osgeo-org.1560.x6.nabble.com/Error-reading-
raster-data-for-row-xxx-only-when-using-r-series-and-t-rast-series-
td5229569.html this long-winded GRASS-dev post] is related, but I am
finding what appear to be randomly corrupted output maps from
`t.rast.mapcalc` when the output is FCELL or DCELL. No problems observed
when output is CELL.

May that be gone with r66731 (lib/gis/worker.c) which set the default
workers to zero (threads only used by explicit request)?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2735#comment:9>
GRASS GIS <https://grass.osgeo.org>

GDAL · February 21, 2018, 4:25pm

#2735: t.rast.mapcalc input problem
--------------------------+----------------------------
  Reporter: leohardtke | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: normal | Milestone: 7.0.6
Component: Temporal | Version: svn-trunk
Resolution: | Keywords: t.rast.mapcalc
       CPU: Unspecified | Platform: Linux
--------------------------+----------------------------
Description changed by martinl:

Old description:

I'm facing a problem with an expression in the t.rast.mapcalc module
which I think is a Bug...

t.rast.mapcalc inputs=QA_mask,ndvi,ndvi_smooth_spacetime
expression="if(QA_mask==0,ndvi,ndvi_smooth_spacetime)" output=ndvi_fixed
basename=ndvi_fixed
It throws the following error:
Starting temporal sampling...
Starting mapcalc computation...
Invalid map
<MOD13Q1.A2009017.h12v12.005.2009035110230_01_4326_smooth_spacetime_0>
Parse error
ERROR: parse error
ERROR: Error while mapcalc computation

The QA_mask maps are:
QA_mask_01|PERMANENT|2009-01-01 00:00:00|2009-01-17 00:00:00
QA_mask_02|PERMANENT|2009-01-17 00:00:00|2009-02-02 00:00:00
......

The ndvi_smooth_spacetime maps are:
ndvi_smooth_spacetime_0|PERMANENT|2009-01-17 00:00:00|2009-02-02 00:00:00
ndvi_smooth_spacetime_1|PERMANENT|2009-02-02 00:00:00|2009-02-18 00:00:00
...........

The ndvi maps are
MOD13Q1.A2009001.h12v12.005.2009019145228_01_4326|PERMANENT|2009-01-01
00:00:00|2009-01-17 00:00:00
MOD13Q1.A2009017.h12v12.005.2009035110230_01_4326|PERMANENT|2009-01-17
00:00:00|2009-02-02 00:00:00
M

As you see, there is no such a map like..
MOD13Q1.A2009017.h12v12.005.2009035110230_01_4326_smooth_spacetime_0

I guess the module is mixing ndvi (a temporal dataset) with
_smooth_spacetime... the second part of the name....

New description:

I'm facing a problem with an expression in the t.rast.mapcalc module which
I think is a Bug...

{{{
t.rast.mapcalc inputs=QA_mask,ndvi,ndvi_smooth_spacetime
expression="if(QA_mask==0,ndvi,ndvi_smooth_spacetime)" output=ndvi_fixed
basename=ndvi_fixed
It throws the following error:
Starting temporal sampling...
Starting mapcalc computation...
Invalid map
<MOD13Q1.A2009017.h12v12.005.2009035110230_01_4326_smooth_spacetime_0>
Parse error
ERROR: parse error
ERROR: Error while mapcalc computation
}}}

The QA_mask maps are:

{{{
QA_mask_01|PERMANENT|2009-01-01 00:00:00|2009-01-17 00:00:00
QA_mask_02|PERMANENT|2009-01-17 00:00:00|2009-02-02 00:00:00
......
}}}

The ndvi_smooth_spacetime maps are:

{{{
ndvi_smooth_spacetime_0|PERMANENT|2009-01-17 00:00:00|2009-02-02 00:00:00
ndvi_smooth_spacetime_1|PERMANENT|2009-02-02 00:00:00|2009-02-18 00:00:00
...........
}}}

The ndvi maps are

{{{
MOD13Q1.A2009001.h12v12.005.2009019145228_01_4326|PERMANENT|2009-01-01
00:00:00|2009-01-17 00:00:00
MOD13Q1.A2009017.h12v12.005.2009035110230_01_4326|PERMANENT|2009-01-17
00:00:00|2009-02-02 00:00:00
}}}

M

As you see, there is no such a map like..

{{{
MOD13Q1.A2009017.h12v12.005.2009035110230_01_4326_smooth_spacetime_0
}}}

I guess the module is mixing ndvi (a temporal dataset) with
_smooth_spacetime... the second part of the name....

--

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2735#comment:12>
GRASS GIS <https://grass.osgeo.org>

GDAL · March 20, 2018, 9:57pm

#2735: t.rast.mapcalc input problem
--------------------------+----------------------------
  Reporter: leohardtke | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: normal | Milestone: 7.0.7
Component: Temporal | Version: svn-trunk
Resolution: | Keywords: t.rast.mapcalc
       CPU: Unspecified | Platform: Linux
--------------------------+----------------------------

Comment (by veroandreo):

A similar (maybe the same) issue was reported in the mailing list in Feb
2018: https://lists.osgeo.org/pipermail/grass-
user/2018-February/077903.html

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2735#comment:14>
GRASS GIS <https://grass.osgeo.org>

GDAL · April 28, 2019, 2:50pm

#2735: t.rast.mapcalc input problem
--------------------------+----------------------------
  Reporter: leohardtke | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: normal | Milestone: 7.0.7
Component: Temporal | Version: svn-trunk
Resolution: | Keywords: t.rast.mapcalc
       CPU: Unspecified | Platform: Linux
--------------------------+----------------------------

Comment (by martinl):

Still relevant?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2735#comment:15>
GRASS GIS <https://grass.osgeo.org>

GDAL · April 29, 2019, 4:58pm

#2735: t.rast.mapcalc input problem
--------------------------+----------------------------
  Reporter: leohardtke | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: normal | Milestone: 7.0.7
Component: Temporal | Version: svn-trunk
Resolution: | Keywords: t.rast.mapcalc
       CPU: Unspecified | Platform: Linux
--------------------------+----------------------------

Comment (by veroandreo):

This is still an issue in 7.6 relbranch. Here's how to reproduce:

{{{
# create strds test
for i in `seq -w 1 10` ; do r.mapcalc expression="map_$i = 1" ; done
t.create output=test type=strds temporaltype=absolute title="test"
description="test"
t.register input=test maps=`g.list rast pat=map_* sep=,`
start='2000-01-01' increment="1 days" -i

# create strds test2
for i in `seq -w 1 10` ; do r.mapcalc expression="mapa_$i = 1" ; done
t.create output=test2 type=strds temporaltype=absolute title="test"
description="test"
t.register input=test2 maps=`g.list rast pat=mapa_* sep=,`
start='2000-01-01' increment="1 days" -i

# t.rast.mapcalc
t.rast.mapcalc inputs=test,test2 expression="test+test2" basename=test
output=test3

Starting temporal sampling...
Starting mapcalc computation...
Invalid map <map_012>
Parse error
ERROR: parse error
ERROR: Error while mapcalc computation
}}}

maybe, this is a wontfix though... AFAIU, it is recommended to use
G7:t.rast.algebra in these cases in which strds and maps within them are
nested or so similar

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2735#comment:16>
GRASS GIS <https://grass.osgeo.org>

GDAL · April 29, 2019, 8:25pm

#2735: t.rast.mapcalc input problem
--------------------------+----------------------------
  Reporter: leohardtke | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: normal | Milestone: 7.0.7
Component: Temporal | Version: svn-trunk
Resolution: | Keywords: t.rast.mapcalc
       CPU: Unspecified | Platform: Linux
--------------------------+----------------------------

Comment (by neteler):

Replying to [comment:3 huhabla]:
> However, please use t.rast.algebra instead. This module has other issues
but is able to correctly identify space time datasets.

Would it be possible to reimplement t.rast.mapcalc as a Python wrapper
around G7:t.rast.algebra?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2735#comment:17>
GRASS GIS <https://grass.osgeo.org>