[GRASS-dev] Re: r.external

GRASS GRASS developer
,
1

On Thu, Aug 21, 2008 at 1:54 PM, Markus Neteler <neteler@osgeo.org> wrote:

On Thu, Aug 21, 2008 at 11:00 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Neteler wrote:

>> Null support is missing (r.external records any null value reported by
>> GDAL, but G_get_raster_row() etc ignore it; I suspect that it will
>> treat zero as null due to the absence of a null file).
>
> Null support has been added.

I have now tested (in my local 6.4 backport) a shaded terrain map (GeoTIFF).
The 0 pixels are registered as NULL pixels, leading to white areas instead
of black in case of cast terrain shadow.

So I confirm that zero is treated as NULL now.
Could the GDAL Null value be used instead?

Null support was added in r32924, and seems to work.

Ha - I missed to backport this one.

Excellent, it now works fine in my local backport, too.

I found a problem with FP maps:

gdalinfo /tmp/geodata2_originals_raid5/pat_DTM_5m_derived_LIDAR_UTM_WGS84/pat_DTM_2008_derived_5m_UTM_WGS84.tif
..
Band 1 Block=23522x1 Type=Float32, ColorInterp=Gray
...

File size (if that matters): 2458610122

GRASS 7.0.svn (patUTM):~/grass70 > r.external
in=/tmp/geodata2_originals_raid5/pat_DTM_5m_derived_LIDAR_UTM_WGS84/pat_DTM_2008_derived_5m_UTM_WGS84.tif
out=pat_dtm_5m
Projection of input dataset and current location appear to match
<pat_dtm_5m> created
r.external complete.

GRASS 7.0.svn (patUTM):~/grass70 > d.rast pat_dtm_5m
PNG: GRASS_TRUECOLOR status: FALSE
PNG: collecting to file: map.png,
GRASS_WIDTH=640, GRASS_HEIGHT=480
WARNING: Unable to find
         '/home/neteler/grassdata/patUTM/neteler/cell_misc/pat_dtm_5m/f_format'
WARNING: Unable to open raster map <pat_dtm_5m@neteler>
ERROR: Unable to open raster map <pat_dtm_5m>

The same happens obviously in my local 6.4.svn backport.

Markus

2

Markus Neteler wrote:

I found a problem with FP maps:

gdalinfo /tmp/geodata2_originals_raid5/pat_DTM_5m_derived_LIDAR_UTM_WGS84/pat_DTM_2008_derived_5m_UTM_WGS84.tif
..
Band 1 Block=23522x1 Type=Float32, ColorInterp=Gray
...

File size (if that matters): 2458610122

GRASS 7.0.svn (patUTM):~/grass70 > r.external
in=/tmp/geodata2_originals_raid5/pat_DTM_5m_derived_LIDAR_UTM_WGS84/pat_DTM_2008_derived_5m_UTM_WGS84.tif
out=pat_dtm_5m
Projection of input dataset and current location appear to match
<pat_dtm_5m> created
r.external complete.

GRASS 7.0.svn (patUTM):~/grass70 > d.rast pat_dtm_5m
PNG: GRASS_TRUECOLOR status: FALSE
PNG: collecting to file: map.png,
GRASS_WIDTH=640, GRASS_HEIGHT=480
WARNING: Unable to find
         '/home/neteler/grassdata/patUTM/neteler/cell_misc/pat_dtm_5m/f_format'

Try r33119.

I didn't set the lzw_compression_bits field; AFAICT, nothing uses it
(even for normal maps).

Is the f_quant file also necessary?

--
Glynn Clements <glynn@gclements.plus.com>

3

On Wed, Aug 27, 2008 at 3:20 PM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Neteler wrote:

I found a problem with FP maps:

gdalinfo /tmp/geodata2_originals_raid5/pat_DTM_5m_derived_LIDAR_UTM_WGS84/pat_DTM_2008_derived_5m_UTM_WGS84.tif
..
Band 1 Block=23522x1 Type=Float32, ColorInterp=Gray
...

File size (if that matters): 2458610122

GRASS 7.0.svn (patUTM):~/grass70 > r.external
in=/tmp/geodata2_originals_raid5/pat_DTM_5m_derived_LIDAR_UTM_WGS84/pat_DTM_2008_derived_5m_UTM_WGS84.tif
out=pat_dtm_5m
Projection of input dataset and current location appear to match
<pat_dtm_5m> created
r.external complete.

GRASS 7.0.svn (patUTM):~/grass70 > d.rast pat_dtm_5m
PNG: GRASS_TRUECOLOR status: FALSE
PNG: collecting to file: map.png,
GRASS_WIDTH=640, GRASS_HEIGHT=480
WARNING: Unable to find
         '/home/neteler/grassdata/patUTM/neteler/cell_misc/pat_dtm_5m/f_format'

Try r33119.

I didn't set the lzw_compression_bits field; AFAICT, nothing uses it
(even for normal maps).

This helps.

Is the f_quant file also necessary?

Good question:

GRASS_VERSION (patUTM):~ > d.rast pat_dtm_5m
PNG: GRASS_TRUECOLOR status: FALSE
PNG: collecting to file: map.png,
GRASS_WIDTH=640, GRASS_HEIGHT=480

WARNING: quantization file [pat_dtm_5m] in mapset [neteler] missing
100%
display map.png

.... looks good.

BTW: could GRASS_TRUECOLOR status be TRUE per default?

thanks,
Markus

4

Markus Neteler wrote:

> Is the f_quant file also necessary?

Good question:

GRASS_VERSION (patUTM):~ > d.rast pat_dtm_5m
PNG: GRASS_TRUECOLOR status: FALSE
PNG: collecting to file: map.png,
GRASS_WIDTH=640, GRASS_HEIGHT=480

WARNING: quantization file [pat_dtm_5m] in mapset [neteler] missing

Probably harmless, but it's easy enough to create it; done in r33121.

BTW: could GRASS_TRUECOLOR status be TRUE per default?

Changed in r33122 (code) and r33123 (documentation).

I think it's fair to assume that the original motivation for
defaulting to FALSE (compatibility with the GD-based version) is no
longer relevant.

The cairo driver (which will replace the PNG driver at some point)
doesn't even have the option of generating paletted output.

--
Glynn Clements <glynn@gclements.plus.com>

5

I have some troubles with the assignment of colors.
The map is a FP DTM map, and the d.rast output is
all white (while r.univar reports 815886 non-null cells).

I assume that this is to to the absence of (some) file
statistics (f_quant file?).

Markus

6

Markus Neteler wrote:

I have some troubles with the assignment of colors.
The map is a FP DTM map, and the d.rast output is
all white (while r.univar reports 815886 non-null cells).

I assume that this is to to the absence of (some) file
statistics (f_quant file?).

The current version writes the f_quant file (although not correctly;
it forgot to call G_quant_init() first; fixed in r33337). It doesn't
write a histogram; you can use "r.support -s ..." to create that.

Does the map have the correct bounds? Does it have a colour table (try
d.legend)? Can you provide any information at all about the imported
map?

--
Glynn Clements <glynn@gclements.plus.com>

7

On Mon, Sep 8, 2008 at 11:42 PM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Neteler wrote:

I have some troubles with the assignment of colors.
The map is a FP DTM map, and the d.rast output is
all white (while r.univar reports 815886 non-null cells).

I assume that this is to to the absence of (some) file
statistics (f_quant file?).

The current version writes the f_quant file (although not correctly;
it forgot to call G_quant_init() first; fixed in r33337).

Backported.

Now, after r.external of a
Band 1 Block=23522x1 Type=Float32, ColorInterp=Gray
map, the color table is gray (so, ok).

Then
r.colors pat_dtm_5m col=elevation
d.rast pat_dtm_5m

leads to a white map.

It doesn't write a histogram; you can use "r.support -s ..." to create that.

I suppose that this is in GRASS 6:
r.support.stats pat_dtm_5m
Statistics for <pat_dtm_5m> updated
r.info -r pat_dtm_5m
WARNING: category support for [pat_dtm_5m] in mapset [PERMANENT] missing
min=nan
max=nan

r.colors pat_dtm_5m col=elevation
Color table for <pat_dtm_5m> set to elevation

d.rast pat_dtm_5m
100%

... still white.

Does the map have the correct bounds?

You mean spatially? Yes (the original gray table works).

Does it have a colour table (try d.legend)?

d.legend pat_dtm_5m
WARNING: category support for [pat_dtm_5m] in mapset [PERMANENT] missing
WARNING: Category file for <pat_dtm_5m> not available

Apparently not: all "nan".

Can you provide any information at all about the imported map?

gdalinfo pat_DTM_2008_derived_5m_UTM_WGS84.tif -mm
...
Computed Min/Max=0.000,3758.871
  Overviews: 11761x9755, 5881x4878, 2941x2439, 1471x1220, 736x610,
368x305, 184x153

Could the GDAL histogram be used anyhow (r.support takes a while on
this 2GB file)?
I am on a 64bit Linux box.

thanks
Markus

8

On Fri, Sep 12, 2008 at 2:44 PM, Markus Neteler <neteler@osgeo.org> wrote:
...

r.info -r pat_dtm_5m
WARNING: category support for [pat_dtm_5m] in mapset [PERMANENT] missing
min=nan
max=nan

I observe that 'f_range' remains empty:

GRASS 6.4.svn (patUTM):~ > l
/home/neteler/grassdata/patUTM/PERMANENT/cell_misc/pat_dtm_5m
total 52
-rw-rw-r-- 1 neteler neteler 132 2008-09-12 14:07 gdal
-rw-rw-r-- 1 neteler neteler 0 2008-09-12 14:07 f_range
-rw-rw-r-- 1 neteler neteler 5 2008-09-12 14:07 f_quant
-rw-rw-r-- 1 neteler neteler 28 2008-09-12 14:07 f_format
-rw-rw-r-- 1 neteler neteler 39928 2008-09-12 22:16 histogram

Could the file be populated using GDAL functions?
http://www.gdal.org/gdal_tutorial.html

        GDALRasterBandH hBand;
        int bGotMin, bGotMax;
        double adfMinMax[2];

        adfMinMax[0] = GDALGetRasterMinimum( hBand, &bGotMin );
        adfMinMax[1] = GDALGetRasterMaximum( hBand, &bGotMax );
        if( ! (bGotMin && bGotMax) )
            GDALComputeRasterMinMax( hBand, TRUE, adfMinMax );
        printf( "Min=%.3fd, Max=%.3f\n", adfMinMax[0], adfMinMax[1] );

?
Markus

9

Markus Neteler wrote:

> It doesn't write a histogram; you can use "r.support -s ..." to create that.

I suppose that this is in GRASS 6:
r.support.stats pat_dtm_5m
Statistics for <pat_dtm_5m> updated
r.info -r pat_dtm_5m
WARNING: category support for [pat_dtm_5m] in mapset [PERMANENT] missing
min=nan
max=nan

Well, that's the problem. Does "r.external -r" help? Does r.in.gdal
import the map correctly?

--
Glynn Clements <glynn@gclements.plus.com>

10

Markus Neteler wrote:

> r.info -r pat_dtm_5m
> WARNING: category support for [pat_dtm_5m] in mapset [PERMANENT] missing
> min=nan
> max=nan

I observe that 'f_range' remains empty:

GRASS 6.4.svn (patUTM):~ > l
/home/neteler/grassdata/patUTM/PERMANENT/cell_misc/pat_dtm_5m
total 52
-rw-rw-r-- 1 neteler neteler 132 2008-09-12 14:07 gdal
-rw-rw-r-- 1 neteler neteler 0 2008-09-12 14:07 f_range
-rw-rw-r-- 1 neteler neteler 5 2008-09-12 14:07 f_quant
-rw-rw-r-- 1 neteler neteler 28 2008-09-12 14:07 f_format
-rw-rw-r-- 1 neteler neteler 39928 2008-09-12 22:16 histogram

Odd.

Could the file be populated using GDAL functions?

It is:

304 info->range[0] = GDALGetRasterMinimum(hBand, &bGotMin);
305 info->range[1] = GDALGetRasterMaximum(hBand, &bGotMax);
306 if(!(bGotMin && bGotMax))
307 GDALComputeRasterMinMax(hBand, !exact_range, info->range);

460 struct FPRange fprange;
461 fprange.min = info->range[0];
462 fprange.max = info->range[1];
463 G_write_fp_range(output, &fprange);
464 write_fp_format(output, info);
465 write_fp_quant(output);

The fact that the f_quant and f_format files are being written implies
that G_write_fp_range() is being called, but is writing an empty file.

Hmm; the -r flag won't help if GDAL is reporting a stored range. But
r.support.stats should fix that.

Is it possible that something is causing GDAL to "see" NaNs?

Maybe the presence of NaNs interferes with the range calculation. As
all inequalities return false for NaN, code such as:

  #define MIN(a,b) ((a) < (b) ? (a) : (b))
  #define MAX(a,b) ((a) > (b) ? (a) : (b))

  for (i = 0; i < n; i++) {
      min = MIN(min, x[i]);
      max = MAX(max, x[i]);
  }

will set both min and max to NaN if even a single x[i] is NaN.

--
Glynn Clements <glynn@gclements.plus.com>

11

On Sat, Sep 13, 2008 at 2:59 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Neteler wrote:

> r.info -r pat_dtm_5m
> WARNING: category support for [pat_dtm_5m] in mapset [PERMANENT] missing
> min=nan
> max=nan

I observe that 'f_range' remains empty:

GRASS 6.4.svn (patUTM):~ > l
/home/neteler/grassdata/patUTM/PERMANENT/cell_misc/pat_dtm_5m
total 52
-rw-rw-r-- 1 neteler neteler 132 2008-09-12 14:07 gdal
-rw-rw-r-- 1 neteler neteler 0 2008-09-12 14:07 f_range
-rw-rw-r-- 1 neteler neteler 5 2008-09-12 14:07 f_quant
-rw-rw-r-- 1 neteler neteler 28 2008-09-12 14:07 f_format
-rw-rw-r-- 1 neteler neteler 39928 2008-09-12 22:16 histogram

Odd.

Could the file be populated using GDAL functions?

It is:

(sorry, I was looking only in the library gdal.c)
...

The fact that the f_quant and f_format files are being written implies
that G_write_fp_range() is being called, but is writing an empty file.

Hmm; the -r flag won't help if GDAL is reporting a stored range.

I have added debug output:

GRASS 6.4.svn (patUTM):~/grass64/raster/r.external > svn diff
Index: main.c

--- main.c (revision 33396)
+++ main.c (working copy)
@@ -303,8 +303,12 @@

     info->range[0] = GDALGetRasterMinimum(hBand, &bGotMin);
     info->range[1] = GDALGetRasterMaximum(hBand, &bGotMax);
+ G_debug(0, "info->range[0]: %f", info->range[0]);
+ G_debug(0, "info->range[1]: %f", info->range[1]);
     if(!(bGotMin && bGotMax))
        GDALComputeRasterMinMax(hBand, !exact_range, info->range);
+ G_debug(0, "exact info->range[0]: %f", info->range[0]);
+ G_debug(0, "exact info->range[1]: %f", info->range[1]);

     G_init_colors(&info->colors);

and it does:

GRASS 6.4.svn (patUTM):~/grass64/raster/r.external > r.external
in=/geodata2/.../pat_DTM_2008_derived_5m_UTM_WGS84.tif out=pat_dtm_5m
--o
Projection of input dataset and current location appear to match
D0/0: info->range[0]: -4294967295.000000
D0/0: info->range[1]: 4294967295.000000
D0/0: exact info->range[0]: 0.000000
D0/0: exact info->range[1]: 3563.149902
<pat_dtm_5m> created

It doesn't match well the GDAL Computed Min/Max (see below) but at
least it is roughly defined:

gdalinfo pat_DTM_2008_derived_5m_UTM_WGS84.tif -mm
...
Computed Min/Max=0.000,3758.871

The file "f_range" continues to be empty.

But r.support.stats should fix that.

Is it possible that something is causing GDAL to "see" NaNs?

Maybe the presence of NaNs interferes with the range calculation. As
all inequalities return false for NaN, code such as:

       #define MIN(a,b) ((a) < (b) ? (a) : (b))
       #define MAX(a,b) ((a) > (b) ? (a) : (b))

       for (i = 0; i < n; i++) {
           min = MIN(min, x[i]);
           max = MAX(max, x[i]);
       }

will set both min and max to NaN if even a single x[i] is NaN.

Adding in create_map():

@@ -454,12 +458,16 @@
        struct Range range;
        range.min = (CELL)info->range[0];
        range.max = (CELL)info->range[1];
+ G_debug(0, "CELL range.min: %d", range.min);
+ G_debug(0, "CELL range.max: %d", range.max);
        G_write_range(output, &range);
     }
     else {
        struct FPRange fprange;
        fprange.min = info->range[0];
        fprange.max = info->range[1];
+ G_debug(0, "FCELL range.min: %f", fprange.min);
+ G_debug(0, "FCELL range.max: %f", fprange.max);
        G_write_fp_range(output, &fprange);
        write_fp_format(output, info);
        write_fp_quant(output);

shows
GRASS 6.4.svn (patUTM):~/grass64/raster/r.external > r.external
in=/geodata2/.../pat_DTM_2008_derived_5m_UTM_WGS84.tif out=pat_dtm_5m
--o
...
D0/0: FCELL range.min: 0.000000
D0/0: FCELL range.max: 3563.149902

So the range (slightly incorrect) survives, but
G_write_fp_range() isn't writing it out.

Markus

12

Markus Neteler wrote:

So the range (slightly incorrect) survives, but G_write_fp_range()
isn't writing it out.

Okay; I think that r33435 should fix it:

--- raster/r.external/main.c (revision 33434)
+++ raster/r.external/main.c (working copy)
@@ -454,12 +454,14 @@
   struct Range range;
   range.min = (CELL)info->range[0];
   range.max = (CELL)info->range[1];
+ range.first_time = 0;
   G_write_range(output, &range);
     }
     else {
   struct FPRange fprange;
   fprange.min = info->range[0];
   fprange.max = info->range[1];
+ fprange.first_time = 0;
   G_write_fp_range(output, &fprange);
   write_fp_format(output, info);
   write_fp_quant(output);

I'm assuming that the inaccuracy in the range is due to the lack of
the -r flag. Without that flag, GDAL calculates an approximate range
from a sampling of cells, rather than processing the entire map.

--
Glynn Clements <glynn@gclements.plus.com>

13

On Sun, Sep 14, 2008 at 1:06 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Neteler wrote:

So the range (slightly incorrect) survives, but G_write_fp_range()
isn't writing it out.

Okay; I think that r33435 should fix it:

--- raster/r.external/main.c (revision 33434)
+++ raster/r.external/main.c (working copy)
@@ -454,12 +454,14 @@
       struct Range range;
       range.min = (CELL)info->range[0];
       range.max = (CELL)info->range[1];
+ range.first_time = 0;
       G_write_range(output, &range);
    }
    else {
       struct FPRange fprange;
       fprange.min = info->range[0];
       fprange.max = info->range[1];
+ fprange.first_time = 0;
       G_write_fp_range(output, &fprange);
       write_fp_format(output, info);
       write_fp_quant(output);

Yes, it solved the problem. Backported to 6.4.svn.

I'm assuming that the inaccuracy in the range is due to the lack of
the -r flag. Without that flag, GDAL calculates an approximate range
from a sampling of cells, rather than processing the entire map.

Using the -r flag helps certainly, too:
r.info -r pat_dtm_5m
WARNING: category support for [pat_dtm_5m] in mapset [PERMANENT] missing
min=0.000000
max=3758.871094

Thanks again for this great module and assistance (so I can avoid to
by another terabyte for my imported (now linked) provincial geodata.

Markus

14

Hi,

using the DTM map registered with r.external, I get sometimes (not sure on
what it depends), a Segmentation fault:

GRASS 6.4.svn (patUTM32):~ > d.rast pat_dtm5m

100%
Segmentation fault

GRASS 6.4.svn (patUTM32):~ > gdb d.rast pat_dtm5m
GNU gdb 6.6-3mdv2008.0 (Mandriva Linux release 2008.0)
Copyright (C) 2006 Free Software Foundation, Inc.
...
This GDB was configured as "x86_64-mandriva-linux-gnu"...
Using host libthread_db library "/lib64/libthread_db.so.1".
/home/neteler/pat_dtm5m: No such file or directory.
(gdb) r pat_dtm5m
Starting program:
/home/neteler/grass64/dist.x86_64-unknown-linux-gnu/bin/d.rast
pat_dtm5m
[Thread debugging using libthread_db enabled]
[New Thread 47745198216448 (LWP 24661)]
warning: Lowest section in /usr/lib64/libicudata.so.36 is .gnu.hash at
0000000000000120

b 100%

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 47745198216448 (LWP 24661)]
0x00002b6c8556ebcb in free () from /lib64/libc.so.6
(gdb) bt
#0 0x00002b6c8556ebcb in free () from /lib64/libc.so.6
#1 0x00002b6c83f0c09b in G_free (buf=0x7c7c668c) at alloc.c:130
#2 0x00002b6c83f112e5 in close_old (fd=8) at closecell.c:144
#3 0x00002b6c83f11181 in G_close_cell (fd=8) at closecell.c:86
#4 0x0000000000401dd2 in cell_draw (name=0x608b20 "pat_dtm5m",
mapset=0x608eb0 "PERMANENT", colors=0x7fff26fd57a0, overlay=0,
invert=0, data_type=2)
    at display.c:97
#5 0x0000000000401c30 in display (name=0x608b20 "pat_dtm5m",
mapset=0x608eb0 "PERMANENT", overlay=0, bg=0x0, data_type=2, invert=0,
nocmd=0)
    at display.c:38
#6 0x00000000004021eb in main (argc=2, argv=0x7fff26fd5a18) at main.c:133

(gdb) bt full
#0 0x00002b6c8556ebcb in free () from /lib64/libc.so.6
No symbol table info available.
#1 0x00002b6c83f0c09b in G_free (buf=0x7c7c668c) at alloc.c:130
No locals.
#2 0x00002b6c83f112e5 in close_old (fd=8) at closecell.c:144
        fcb = (struct fileinfo *) 0x6ee5c0
        i = 8
#3 0x00002b6c83f11181 in G_close_cell (fd=8) at closecell.c:86
        fcb = (struct fileinfo *) 0x6ee5c0
#4 0x0000000000401dd2 in cell_draw (name=0x608b20 "pat_dtm5m",
mapset=0x608eb0 "PERMANENT", colors=0x7fff26fd57a0, overlay=0,
invert=0, data_type=2)
    at display.c:97
        cellfile = 8
        xarray = (void *) 0x62b060
        cur_A_row = -1
        t = 0
        b = 1177
        l = 0
        r = 1920
        ncols = 470
        nrows = 390
#5 0x0000000000401c30 in display (name=0x608b20 "pat_dtm5m",
mapset=0x608eb0 "PERMANENT", overlay=0, bg=0x0, data_type=2, invert=0,
nocmd=0)
    at display.c:38
        colors = {version = 0, shift = 0, invert = 0, is_float = 1,
null_set = 1, null_red = 255 '?', null_grn = 255 '?', null_blu = 255
'?',
  undef_set = 0, undef_red = 108 'l', undef_grn = 43 '+', undef_blu =
0 '\0', fixed = {rules = 0x609900, n_rules = 13, lookup = {
      red = 0xffffffff <Address 0xffffffff out of bounds>, grn = 0x0,
blu = 0x6038a0 "???\203l+", set = 0x0, nalloc = 654137872, active =
0},
    fp_lookup = {vals = 0x746240, rules = 0x746320, nalloc = 14,
active = 1}, min = -11000, max = 8850}, modular = {rules = 0x0,
n_rules = 0, lookup = {
      red = 0x606770 "x0", grn = 0x2b6c83efb988 "\024\a", blu =
0x2b6c83ceac00 "", set = 0x7fff26fd5930 "", nalloc = -2085735822,
active = 0},
    fp_lookup = {vals = 0x0, rules = 0x7fff26fd56d8, nalloc = 0,
active = 0}, min = 0, max = -1}, cmin = -11000, cmax = 8850}
        r = 0
        g = 0
        b = 0
#6 0x00000000004021eb in main (argc=2, argv=0x7fff26fd5a18) at main.c:133
        mapset = 0x608eb0 "PERMANENT"
        name = 0x608b20 "pat_dtm5m"
        overlay = 0
        invert = 0
        fp = 1
        module = (struct GModule *) 0x2b6c84160760
        map = (struct Option *) 0x2b6c841606c0
        catlist = (struct Option *) 0x6087f0
        vallist = (struct Option *) 0x608890
        bg = (struct Option *) 0x608950
        flag_o = (struct Flag *) 0x2b6c84160680
        flag_i = (struct Flag *) 0x608a30
        flag_x = (struct Flag *) 0x608a80
(gdb)

It seems to be some G_free() issue (?).
Any ideas?

thanks.
Markus

15

Markus Neteler wrote:

using the DTM map registered with r.external, I get sometimes (not sure on
what it depends), a Segmentation fault:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 47745198216448 (LWP 24661)]
0x00002b6c8556ebcb in free () from /lib64/libc.so.6
(gdb) bt
#0 0x00002b6c8556ebcb in free () from /lib64/libc.so.6
#1 0x00002b6c83f0c09b in G_free (buf=0x7c7c668c) at alloc.c:130
#2 0x00002b6c83f112e5 in close_old (fd=8) at closecell.c:144

143 if (fcb->cellhd.compressed)
144 G_free(fcb->row_ptr);

The compressed field should always be zero.

r.external should always set it to zero in the cellhd file, and for a
map opened for read, the cellhd structure should contain exactly what
is in the file.

--
Glynn Clements <glynn@gclements.plus.com>

16

On Wed, Sep 17, 2008 at 6:45 PM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Neteler wrote:

using the DTM map registered with r.external, I get sometimes (not sure on
what it depends), a Segmentation fault:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 47745198216448 (LWP 24661)]
0x00002b6c8556ebcb in free () from /lib64/libc.so.6
(gdb) bt
#0 0x00002b6c8556ebcb in free () from /lib64/libc.so.6
#1 0x00002b6c83f0c09b in G_free (buf=0x7c7c668c) at alloc.c:130
#2 0x00002b6c83f112e5 in close_old (fd=8) at closecell.c:144

143 if (fcb->cellhd.compressed)
144 G_free(fcb->row_ptr);

The compressed field should always be zero.

r.external should always set it to zero in the cellhd file, and for a
map opened for read, the cellhd structure should contain exactly what
is in the file.

This seems to be the case:

[neteler@fep grassdata]$ cd patUTM32/PERMANENT/cellhd/
[neteler@fep cellhd]$ cat pat_DTM_5m
proj: 1
zone: 32
north: 5157086
south: 5059541
east: 730098
west: 612488
cols: 23522
rows: 19509
e-w resol: 5
n-s resol: 5
format: -1
compressed: 0

Markus

17

Markus Neteler wrote:

>> using the DTM map registered with r.external, I get sometimes (not sure on
>> what it depends), a Segmentation fault:
>
>> Program received signal SIGSEGV, Segmentation fault.
>> [Switching to Thread 47745198216448 (LWP 24661)]
>> 0x00002b6c8556ebcb in free () from /lib64/libc.so.6
>> (gdb) bt
>> #0 0x00002b6c8556ebcb in free () from /lib64/libc.so.6
>> #1 0x00002b6c83f0c09b in G_free (buf=0x7c7c668c) at alloc.c:130
>> #2 0x00002b6c83f112e5 in close_old (fd=8) at closecell.c:144
>
> 143 if (fcb->cellhd.compressed)
> 144 G_free(fcb->row_ptr);
>
> The compressed field should always be zero.
>
> r.external should always set it to zero in the cellhd file, and for a
> map opened for read, the cellhd structure should contain exactly what
> is in the file.

This seems to be the case:

[neteler@fep grassdata]$ cd patUTM32/PERMANENT/cellhd/
[neteler@fep cellhd]$ cat pat_DTM_5m
proj: 1
zone: 32
north: 5157086
south: 5059541
east: 730098
west: 612488
cols: 23522
rows: 19509
e-w resol: 5
n-s resol: 5
format: -1
compressed: 0

So it appears that something is modifying fcb->cellhd.compressed.

But I have no idea what. For an input map, nothing explicitly modifies
that field after the structure has been read from the file, and I
can't see any plausible memory corruption scenario.

d.rast should only have a single raster open, so the preceding
G__.fileinfo slots should be unused (AFAICT, the fact that it's
getting fd=8 is due to additional descriptors used by gdb).

The only array in "struct fileinfo" is NULL_ROWS, but if the map is a
GDAL link, the only code which modifies that uses
"for (i = 0; i < NULL_ROWS_INMEM; i++)", so it can't overflow the
array.

--
Glynn Clements <glynn@gclements.plus.com>