[GRASS-dev] seemingly random errors in r.mapcalc

Hi,

In the last few revisions of GRASS GIS 7.0 (on Ubuntu 13.04) I am experiencing this seemingly random errors with r.mapcalc. For example:

Running the following r.mapcalc computation gave me an ERROR:

GRASS 7.0.svn (AEA):~ > r.mapcalc --overwrite “glc_influence = if(IIASA_crops==100,8.0, if((glc_change5 + glc_change6)==100, 4.0IIASArev + 8.0IIASA_crops/100.0, (glc_change1IIASArev + 2.0glc_change2IIASArev + 3.0glc_change3IIASArev + 4.0glc_change4IIASArev + glc_div + 8.0IIASA_crops) / 100.0))” --overwrite
ERROR: Error reading raster data
[Raster MASK present]

Rerunning the same, and it goes through fine:

GRASS 7.0.svn (AEA):~ > r.mapcalc --overwrite “glc_influence = if(IIASA_crops==100,8.0, if((glc_change5 + glc_change6)==100, 4.0IIASArev + 8.0IIASA_crops/100.0, (glc_change1IIASArev + 2.0glc_change2IIASArev + 3.0glc_change3IIASArev + 4.0glc_change4IIASArev + glc_div + 8.0IIASA_crops) / 100.0))” --overwrite
100%
[Raster MASK present]

This is not always the second time, sometimes it runs fine the first time, but doesn’t the second time, or it only runs fine the third time.

Initially I though it such errors are more likely to happen when the equation includes double precision / floating rasters and integer rasters. But in the example above all layers were converted to double precision first.

I am really puzzled as to what is going on here. Any idea what I am doing wrong here or what could be the cause?

Rgds

Paulo

Paulo van Breugel wrote:

I am really puzzled as to what is going on here. Any idea what I am doing
wrong here or what could be the cause?

My first suspicion would be hardware issues, e.g. a failing disk
drive.

While there are certain types of programming error which can cause
non-deterministic behaviour (e.g. reading uninitialised memory),
neither r.mapcalc nor lib/raster have had any significant changes
recently.

--
Glynn Clements <glynn@gclements.plus.com>

On Fri, Aug 2, 2013 at 4:32 AM, Glynn Clements <glynn@gclements.plus.com> wrote:

Paulo van Breugel wrote:

I am really puzzled as to what is going on here. Any idea what I am doing
wrong here or what could be the cause?

My first suspicion would be hardware issues, e.g. a failing disk
drive.

Please post your compiler flags as well (if you compiled yourself).

Markus

OK, that would be inconvenient, to use an understatement. I hadn’t though about it because I do not seem to have trouble with other programs / data on that disk, but I’ll check if the same problems occur when running the calculations with data on another HD. These are the configure / make / make install steps I follow: ./configure --prefix=/usr/local/grass7 --enable-64bit --with-libs=/lib64 --with-sqlite --with-odbc --with-cairo --with-geos --with-cxx=yes --with-gdal=/usr/local/gdal1.9/bin/gdal-config --with-python=yes --with-wxwidgets=/usr/bin/wx-config --with-tcltk-includes=/usr/include/tcl8.5 --with-readline --with-freetype --with-freetype-includes=/usr/include/freetype2 --enable-largefile --with-motif --with-motif-includes=/usr/include --with-proj-share=/usr/share/proj --with-pthread --with-postgres --with-postgres-libs=/usr/include/postgresql/libpq --with-postgres-includes=/usr/include/postgresql --with-lapack --with-blas --with-opencl make -j4 sudo make install sudo ln -s /usr/local/grass7/bin/grass70 /usr/bin/

···

On 08/02/2013 09:16 AM, Markus Neteler wrote:

On Fri, Aug 2, 2013 at 4:32 AM, Glynn Clements [<glynn@gclements.plus.com>](mailto:glynn@gclements.plus.com) wrote:

Paulo van Breugel wrote:

I am really puzzled as to what is going on here. Any idea what I am doing
wrong here or what could be the cause?

My first suspicion would be hardware issues, e.g. a failing disk
drive.

Please post your compiler flags as well (if you compiled yourself).

Markus

One reason I did not think about problem to be related to hard drive issues is that I would expect other GRASS functions to fail, at least occasionally, as well as other programs.

···

On Fri, Aug 2, 2013 at 4:32 AM, Glynn Clements <glynn@gclements.plus.com> wrote:

Paulo van Breugel wrote:

I am really puzzled as to what is going on here. Any idea what I am doing
wrong here or what could be the cause?

My first suspicion would be hardware issues, e.g. a failing disk
drive.

While there are certain types of programming error which can cause
non-deterministic behaviour (e.g. reading uninitialised memory),
neither r.mapcalc nor lib/raster have had any significant changes
recently.


Glynn Clements <glynn@gclements.plus.com>

Sorry, pushed the short cut sending the email off too early… What I wanted to ask is, is there any way I can find out what kind of error I am getting? Now, the only message is “Error”.

···

On Fri, Aug 2, 2013 at 12:38 PM, Paulo van Breugel <p.vanbreugel@gmail.com> wrote:

One reason I did not think about problem to be related to hard drive issues is that I would expect other GRASS functions to fail, at least occasionally, as well as other programs.

On Fri, Aug 2, 2013 at 4:32 AM, Glynn Clements <glynn@gclements.plus.com> wrote:

Paulo van Breugel wrote:

I am really puzzled as to what is going on here. Any idea what I am doing
wrong here or what could be the cause?

My first suspicion would be hardware issues, e.g. a failing disk
drive.

While there are certain types of programming error which can cause
non-deterministic behaviour (e.g. reading uninitialised memory),
neither r.mapcalc nor lib/raster have had any significant changes
recently.


Glynn Clements <glynn@gclements.plus.com>

Hi,
you can set the environment variable GRASS_SIGSEGV_ON_ERROR to raise a
segmentation fault signal that can be catched by gdb. Then you can
make backtrace and send it to the list.

I hope we will be able to help you then ... but i am not sure.

I assume you use bash or equivalent:
{{{
export GRASS_SIGSEGV_ON_ERROR=1

gdb r.mapcalc

(gdb) r expr="test = 1 + 1"
...
(gdb) bt
... stack
}}}

Best regards
Soeren

2013/8/2 Paulo van Breugel <p.vanbreugel@gmail.com>:

Sorry, pushed the short cut sending the email off too early.. What I wanted
to ask is, is there any way I can find out what kind of error I am getting?
Now, the only message is "Error".

On Fri, Aug 2, 2013 at 12:38 PM, Paulo van Breugel <p.vanbreugel@gmail.com>
wrote:

One reason I did not think about problem to be related to hard drive
issues is that I would expect other GRASS functions to fail, at least
occasionally, as well as other programs.

On Fri, Aug 2, 2013 at 4:32 AM, Glynn Clements <glynn@gclements.plus.com>
wrote:

Paulo van Breugel wrote:

> I am really puzzled as to what is going on here. Any idea what I am
> doing
> wrong here or what could be the cause?

My first suspicion would be hardware issues, e.g. a failing disk
drive.

While there are certain types of programming error which can cause
non-deterministic behaviour (e.g. reading uninitialised memory),
neither r.mapcalc nor lib/raster have had any significant changes
recently.

--
Glynn Clements <glynn@gclements.plus.com>

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Paulo van Breugel wrote:

Sorry, pushed the short cut sending the email off too early.. What I wanted
to ask is, is there any way I can find out what kind of error I am getting?
Now, the only message is "Error".

There are only a few possible causes of "Error reading raster data",
namely an error from lseek(), read(), G_zlib_read() or GDALRasterIO(),
or a short count returned from read().

Any of these can be caused by corrupted data, but that would normally
be repeatable.

G_zlib_read() is only applicable to floating-point maps, while
GDALRasterIO() is only applicable to GDAL-linked maps (r.external).

If it's a hardware error, it will be logged by the kernel. The log
files are usually in /var/log, but which errors go in which files
depends upon the distribution.

If you're compiling from source, be sure to run "make clean" between
"svn update" and "make". Incremental compilation isn't reliable in the
general case. If "svn update" updates the configure script, be sure to
re-run configure before doing anything else.

--
Glynn Clements <glynn@gclements.plus.com>

Thanks all for trying to help out with this and all information and suggestions, much appreciated!

It is probably not related to incremental compilation issues. I had those in the past, even though I always use make clean and re-run the configuration file. So since then I regularly do a completely clean compilation, i.e., removing the complete grass folder in /usr/local/bin and downloading the grass source files again rather then updating. I did so for the latest compilation too.I

I’ll try if I can find anything in the log files and try the suggestions of Soeren. I will also try to run the same calculations using a new mapset and new data, to rule out corrupt data.

···

On Sat, Aug 3, 2013 at 4:02 AM, Glynn Clements <glynn@gclements.plus.com> wrote:

Paulo van Breugel wrote:

Sorry, pushed the short cut sending the email off too early… What I wanted
to ask is, is there any way I can find out what kind of error I am getting?
Now, the only message is “Error”.

There are only a few possible causes of “Error reading raster data”,
namely an error from lseek(), read(), G_zlib_read() or GDALRasterIO(),
or a short count returned from read().

Any of these can be caused by corrupted data, but that would normally
be repeatable.

G_zlib_read() is only applicable to floating-point maps, while
GDALRasterIO() is only applicable to GDAL-linked maps (r.external).

If it’s a hardware error, it will be logged by the kernel. The log
files are usually in /var/log, but which errors go in which files
depends upon the distribution.

If you’re compiling from source, be sure to run “make clean” between
“svn update” and “make”. Incremental compilation isn’t reliable in the
general case. If “svn update” updates the configure script, be sure to
re-run configure before doing anything else.


Glynn Clements <glynn@gclements.plus.com>