[GRASS-dev] r.mapcalc in trunk with pthreads broken

GRASS GIS GRASS GIS developer
1

I am observing corrupted output from r.mapcalc in trunk if compiled
--with-pthread. The result is not identical to the one obtained
--without-pthread. The mapcalc expression included the functions
eval(), min(), and max(), in case that helps. Open a ticket?

Markus M

2

Markus Metz wrote:

I am observing corrupted output from r.mapcalc in trunk if compiled
--with-pthread. The result is not identical to the one obtained
--without-pthread. The mapcalc expression included the functions
eval(), min(), and max(), in case that helps. Open a ticket?

Yes.

Also, can you describe whether (and how) the behaviour is affected by:

  export WORKERS=1

The WORKERS environment variable controls the maximum number of
threads used; the default value is 8.

--
Glynn Clements <glynn@gclements.plus.com>

3

On Thu, May 17, 2012 at 7:33 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Metz wrote:

I am observing corrupted output from r.mapcalc in trunk if compiled
--with-pthread. The result is not identical to the one obtained
--without-pthread. The mapcalc expression included the functions
eval(), min(), and max(), in case that helps. Open a ticket?

Yes.

Also, can you describe whether (and how) the behaviour is affected by:

   export WORKERS=1

The WORKERS environment variable controls the maximum number of
threads used; the default value is 8.

With WORKERS=1, the r.mapcalc result is identical to pthreads
disabled, i.e. correct. Do you need more info?

Markus M

4

Markus Metz wrote:

Glynn Clements wrote:

Markus Metz wrote:

I am observing corrupted output from r.mapcalc in trunk if compiled
--with-pthread. The result is not identical to the one obtained
--without-pthread. The mapcalc expression included the functions
eval(), min(), and max(), in case that helps. Open a ticket?

Yes.

Also, can you describe whether (and how) the behaviour is affected by:

   export WORKERS=1

The WORKERS environment variable controls the maximum number of
threads used; the default value is 8.

With WORKERS=1, the r.mapcalc result is identical to pthreads
disabled, i.e. correct. Do you need more info?

More info:

I think the bug is not caused by the combination of pthreads with
eval(), min(), or max(), but rather when the same raster map appears
twice in a r.mapcalc expression. A simple example would be

r.mapcalc "result = if(isnull(map1), map2, map2 + map1)"

Maybe in r.mapcalc/map.c, get_map_row()

    pthread_mutex_lock(&m->mutex);

has no effect and the same file is accessed twice at the same time,
causing garbage?

Markus M

5

Markus Metz wrote:

> With WORKERS=1, the r.mapcalc result is identical to pthreads
> disabled, i.e. correct. Do you need more info?

More info:

I think the bug is not caused by the combination of pthreads with
eval(), min(), or max(), but rather when the same raster map appears
twice in a r.mapcalc expression. A simple example would be

r.mapcalc "result = if(isnull(map1), map2, map2 + map1)"

Maybe in r.mapcalc/map.c, get_map_row()

    pthread_mutex_lock(&m->mutex);

has no effect and the same file is accessed twice at the same time,
causing garbage?

I can't reproduce this.

Can you confirm that you did "make clean" before or after running
configure? Using a non-threaded r.mapcalc with a threaded libgis could
cause this problem. Running:

  nm -D `which r.mapcalc` | fgrep pthread

will confirm that the actual r.mapcalc binary being used was built
with pthread support, in case you have multiple versions of r.mapcalc
and/or libgis on your system.

--
Glynn Clements <glynn@gclements.plus.com>

6

Glynn Clements wrote:

Markus Metz wrote:

> With WORKERS=1, the r.mapcalc result is identical to pthreads
> disabled, i.e. correct. Do you need more info?

More info:

I think the bug is not caused by the combination of pthreads with
eval(), min(), or max(), but rather when the same raster map appears
twice in a r.mapcalc expression. A simple example would be

r.mapcalc "result = if(isnull(map1), map2, map2 + map1)"

Maybe in r.mapcalc/map.c, get_map_row()

pthread\_mutex\_lock\(&amp;m\-&gt;mutex\);

has no effect and the same file is accessed twice at the same time,
causing garbage?

I can't reproduce this.

I can't reproduce this either with the sample datasets. Maybe this
happens only with larger raster maps? The mismatch occurs with raster
maps with 400 million cells.

Can you confirm that you did "make clean" before or after running
configure? Using a non-threaded r.mapcalc with a threaded libgis could
cause this problem.

Compiling trunk takes about 2 minutes on my test system, I don't know
pthreads, so I decided to stay safe and did make distclean before
reconfiguring and recompiling with/without pthreads. And my local copy
has no modifications in r.mapcalc or libgis.

Markus M