[GRASS5] ERROR: cell_values_double: xdr_double failed for index 387.

I'm getting this error when I try to access a DCELL map after setting
the region to cross 180 degrees lon:

ERROR: cell_values_double: xdr_double failed for index 387.
(lib/gis/get_row.c line 414)

G57:~> g.region -p
projection: 3 (Latitude-Longitude)
zone: 0
datum: wgs84
ellipsoid: wgs84
north: 25:10S
south: 56S
west: 153:05E
east: 174:40W
nsres: 0:05
ewres: 0:05
rows: 370
cols: 387

g.region -d && d.zoom -p
works ok until I hit 180E, then the above error.

Works ok with a CELL map; fails on any FCELL or DCELL map.
fails using d.rast, r.mapcalc, etc. with GRASS 5.0.3 -> GRASS 5.7-cvs.

ideas?

Hamish

Hamish wrote:

I'm getting this error when I try to access a DCELL map after setting
the region to cross 180 degrees lon:

ERROR: cell_values_double: xdr_double failed for index 387.
(lib/gis/get_row.c line 414)

G57:~> g.region -p
projection: 3 (Latitude-Longitude)
zone: 0
datum: wgs84
ellipsoid: wgs84
north: 25:10S
south: 56S
west: 153:05E
east: 174:40W
nsres: 0:05
ewres: 0:05
rows: 370
cols: 387

g.region -d && d.zoom -p
works ok until I hit 180E, then the above error.

Works ok with a CELL map; fails on any FCELL or DCELL map.
fails using d.rast, r.mapcalc, etc. with GRASS 5.0.3 -> GRASS 5.7-cvs.

I can't reproduce this (using the "global" sample data set).

I would guess that the column mapping is being miscalculated, e.g. due
to rounding error, causing cell_values_{float,double} to attempt to
read beyond the end of the row. window_map.c is my prime suspect.

This won't necessarily show up in the integer case; cell_values_int
will just silently read beyond the end of the input buffer (you'll get
bogus data, but no error). OTOH, the FP versions use an XDR stream,
and xdr_{float,double} will indicate an error (by returning zero) if
you overrun the stream.

BTW, the index in the error message is incorrect.

For recent 5.3/5.7 (after my raster I/O changes), it always reports
the total number of columns, not the column number at which the error
occurred. Older versions report the number of columns left to read
(i.e. the first column is reported as columns-1 and the last column is
reported as zero).

This isn't related to why the error occurred, though.

--
Glynn Clements <glynn.clements@virgin.net>

> I'm getting this error when I try to access a DCELL map after setting
> the region to cross 180 degrees lon:
>
> ERROR: cell_values_double: xdr_double failed for index 387.
> (lib/gis/get_row.c line 414)
>
> G57:~> g.region -p
> projection: 3 (Latitude-Longitude)
> zone: 0
> datum: wgs84
> ellipsoid: wgs84
> north: 25:10S
> south: 56S
> west: 153:05E
> east: 174:40W
> nsres: 0:05
> ewres: 0:05
> rows: 370
> cols: 387
>
>
> g.region -d && d.zoom -p
> works ok until I hit 180E, then the above error.
>
> Works ok with a CELL map; fails on any FCELL or DCELL map.
> fails using d.rast, r.mapcalc, etc. with GRASS 5.0.3 -> GRASS 5.7-cvs.

I can't reproduce this (using the "global" sample data set).

Ok, global data set, try it with:
G57:~> g.region -d
G57:~> r.mapcalc nationsF='float(nations)'
G57:~> d.rast nationsF
G57:~> d.zoom # (japan)
G57:~> d.zoom -p # to the east
[...]
north: 13:50S south: 27:20S east: 178W west: 167:05E
East limit of default region reached.
ERROR: cell_values_float: xdr_float failed for index 179.

after it breaks,
G57:~> g.region -d
G57:~> d.rast nationsF
ERROR: cell_values_float: xdr_float failed for index 179.
G57:~> d.rast nationsF
[works]

Hamish

Hamish wrote:

> > I'm getting this error when I try to access a DCELL map after setting
> > the region to cross 180 degrees lon:
> >
> > ERROR: cell_values_double: xdr_double failed for index 387.
> > (lib/gis/get_row.c line 414)
> >
> > G57:~> g.region -p
> > projection: 3 (Latitude-Longitude)
> > zone: 0
> > datum: wgs84
> > ellipsoid: wgs84
> > north: 25:10S
> > south: 56S
> > west: 153:05E
> > east: 174:40W
> > nsres: 0:05
> > ewres: 0:05
> > rows: 370
> > cols: 387
> >
> >
> > g.region -d && d.zoom -p
> > works ok until I hit 180E, then the above error.
> >
> > Works ok with a CELL map; fails on any FCELL or DCELL map.
> > fails using d.rast, r.mapcalc, etc. with GRASS 5.0.3 -> GRASS 5.7-cvs.
>
> I can't reproduce this (using the "global" sample data set).

Ok, global data set, try it with:
G57:~> g.region -d
G57:~> r.mapcalc nationsF='float(nations)'
G57:~> d.rast nationsF
G57:~> d.zoom # (japan)
G57:~> d.zoom -p # to the east
[...]
north: 13:50S south: 27:20S east: 178W west: 167:05E
East limit of default region reached.
ERROR: cell_values_float: xdr_float failed for index 179.

OK, I can reproduce with:

  g.region -d
  r.mapcalc nationsF='float(nations)'
  g.region n=13:50S s=27:20S e=178W w=167:05E
  d.rast nationsF

[Now I can't understand how I failed to reproduce it before; AFAICT,
this will always occur if you read an FP raster when the region
crosses the 180-degree line.]

The problem is in how the FP versions of the resampling code access
the input data. Essentially, they assume that the column mapping will
always be monotonically increasing, and read the input data
sequentially (from an XDR stream):

    xdr_setpos(xdrs, 0);

    for (i = 0; i < n; i++)
    {
  if (!cmap[i])
  {
      c[i] = 0;
      continue;
  }

  if (cmap[i] == cmapold)
  {
      c[i] = c[i-1];
      continue;
  }

  while (cmapold++ != cmap[i]) /* skip */
      if (!xdr_float(xdrs, &c[i]))
      {
    G_fatal_error("cell_values_float: xdr_float failed for index %d.", i);
    return;
      }

  cmapold--;
    }

The loop iterates i over destination columns, with cmap[i] being the
corresponding source column; cmapold contains the previous source
column. If cmap[i] is 0, the source column is beyond the borders of
the map. If cmap[i] == cmapold, the source column is the same as for
the last pass, so the previous value can be reused. Otherwise, it
keeps reading values from the XDR stream until it reaches the desired
source column.

Except, if the source column decreases, i.e. when the data crosses the
180-degree line, it won't reach the desired source column; it will
reach the end of the stream.

It needs to check for this, and rewind the stream to the beginning if
it occurs. Try the attached patch.

FWIW, this has never worked. The only change to
cell_values_{float,double} between the Feb_26_2001 snapshot (the
oldest version of GRASS 5.x which I have) and my re-write a few weeks
back was internationalising an error message:

- sprintf(rsbbuf, "cell_values_d: xdr_double failed for index %d.", n);
+ sprintf(rsbbuf, _("cell_values_d: xdr_double failed for index %d."), n);

BTW, it doesn't apply to cell_values_int, which just does pointer
arithmetic to locate the source value.

--
Glynn Clements <glynn.clements@virgin.net>

(attachments)

get_row.c.diff (507 Bytes)

> > > I'm getting this error when I try to access a DCELL map after setting
> > > the region to cross 180 degrees lon:

[...]

> ERROR: cell_values_float: xdr_float failed for index 179.

OK, I can reproduce with:

  g.region -d
  r.mapcalc nationsF='float(nations)'
  g.region n=13:50S s=27:20S e=178W w=167:05E
  d.rast nationsF

[Now I can't understand how I failed to reproduce it before; AFAICT,
this will always occur if you read an FP raster when the region
crosses the 180-degree line.]

The problem is in how the FP versions of the resampling code access
the input data. Essentially, they assume that the column mapping will
always be monotonically increasing, and read the input data
sequentially (from an XDR stream):

[...]

It needs to check for this, and rewind the stream to the beginning if
it occurs. Try the attached patch.

seems to work fine after patching.

thanks,
Hamish