[GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset

GDAL · July 3, 2017, 6:06pm

#3365: r.in.gdal: crash on large dataset
-----------------------+---------------------------------
Reporter: neteler | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.2.2
Component: Raster | Version: svn-releasebranch72
Keywords: r.in.gdal | CPU: x86-64
Platform: Linux |
-----------------------+---------------------------------
Trying to import a large VRT dataset, I get a crash:

{{{
GRASS 7.2.2svn (latlong):~/software/grass72_svn_debugmode/bin.x86_64-pc-
linux-gnu > r.in.gdal
input=/mnt/kvm7-storage/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Proceeding with import of 1 raster bands...
Importing raster map <loss_global_forest_cover_loss_2000_2015>...
Segmentation fault (core dumped)
}}}

Debugging:
{{{
GRASS 7.2.2svn (latlong):~/software/grass72_svn_debugmode/bin.x86_64-pc-
linux-gnu > gdb r.in.gdal
GNU gdb (GDB) Fedora 7.12.1-48.fc25
Copyright (C) 2017 Free Software Foundation, Inc.
[...]
Reading symbols from r.in.gdal...done.
(gdb) r
input=/mnt//GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Starting program:
/home/mundialis/software/grass72_svn_debugmode/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal
input=/mnt//GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Missing separate debuginfos, use: dnf debuginfo-install
glibc-2.24-8.fc25.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Proceeding with import of 1 raster bands...
Importing raster map <loss_global_forest_cover_loss_2000_2015>...

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff77a1ab9 in put_data (fd=0, null_buf=0x7fffffe9b020 "",
cell=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0) at put_row.c:370
370 compressed_buf[0] = work_buf[0] = nbytes;

(gdb) bt full
#0 0x00007ffff77a1ab9 in put_data (fd=0, null_buf=0x7fffffe9b020 "",
cell=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0) at put_row.c:370
         wk = 0x7fffff91cb31 ""
         nbytes = 1
         compressed_buf = 0x7fffff7bd220 <error: Cannot access memory at
address 0x7fffff7bd220>
         total = 1440000
         fcb = 0x691080
         compressed = 1
         len = 4
         work_buf = 0x7fffff91cb30 "\001"
         wk = 0x7fffff91cb31 ""
         nwrite = 0
#1 0x00007ffff77a1f98 in put_raster_data (fd=0, null_buf=0x7fffffe9b020
"", rast=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0, map_type=0)
     at put_row.c:476
         fcb = 0x691080
#2 0x00007ffff77a2bdb in put_raster_row (fd=0, buf=0x7fffc62ac010,
data_type=0, zeros_r_nulls=0) at put_row.c:699
         convert_and_write_FtypeOtype = {{0x0, 0x7ffff77a23d9
<convert_and_write_if>, 0x7ffff77a25f5 <convert_and_write_id>}, {
             0x7ffff77a280f <convert_and_write_fi>, 0x0, 0x7ffff77a26f3
<convert_and_write_fd>}, {0x7ffff77a2928 <convert_and_write_di>,
             0x7ffff77a24d7 <convert_and_write_df>, 0x0}}
         fcb = 0x691080
         null_buf = 0x7fffffe9b020 ""
#3 0x00007ffff77a0f8b in Rast_put_row (fd=0, buf=0x7fffc62ac010,
data_type=0) at put_row.c:67
No locals.
#4 0x0000000000406e4d in ImportBand (hBand=0x7dbcb0, output=0x627dc0
"loss_global_forest_cover_loss_2000_2015", group_ref=0x0) at main.c:1141
         data_type = 0
         eGDT = GDT_Int32
         eRawGDT = GDT_Byte
         row = 1
         nrows = 560000
         ncols = 1440000
         complex = 0
         cf = 0
         cfR = 32767
         cfI = -145625496
         bNoDataEnabled = 0
         indx = 32767
         cell = 0x7fffc62ac010
         cellReal = 0x77
         cellImg = 0x7fffffffac40
         bufComplex = 0x7fffffffab40
         dfNoData = -10000
         outputReal =
"0\341\377\367\377\177\000\000\260\254\377\377\377\177\000\000\071\031@\000\000\000\000\000\243\366\251\242\000\000\000\000\377\377\377\377",
'\000' <repeats 12 times>,
"H\017\064\366\377\177\000\000\000\240\374\367\377\177\000\000\000\000\000\000\000\000\360?",
'\000' <repeats 14 times>, "\360?", '\000' <repeats 30 times>,
"\360?\000\000\000\000\000\000\000\000\220\062@\000\000\000\000\000\314\345w\367\377\177\000\000P\255\377\377\377\177\000\000\350\262`\000\000\000\000\000\220\062@\000\000\000\000\000\320\332\377\377\377\177",
'\000' <repeats 18 times>, "\360\331\377\377\377\177\000\000"...
         outputImg = '\000' <repeats 32 times>,
"\377\000\000\000\000\000\377\377\000\000\000\000\000\000\377\377\000\000\000\000\000\000\377\377\377\377\377\377\000\377\377\377",
'\000' <repeats 22 times>, "\360?", '\000' <repeats 14 times>, "\360?",
'\000' <repeats 14 times>,
"\360?\000\000\000\000\000\000\000\000\210\345w\367\377\177", '\000'
<repeats 26 times>,
"\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\200\256\377\377\377\177\000\000\240\224z\367\377\177\000\000\000\273}",
'\000' <repeats 13 times>...
         nullFlags = 0x0
         history = {fields = {0x7ffff7fca000 "", 0x7fffffffabe8 "",
0x7fffffffabe4 "", 0x7ffff7ff7b78 "\350\343\377\367\377\177",
             0x7fffffffaba0 "\300\254\377\377\377\177", 0x401939
"GDALGetRasterBand", 0x400be8 "\261\a", 0x7fffffffabe8 ""}, nlines =
-1565919581,
           lines = 0x28aa7da}
         GDALmetadata = 0x7ffff77aa094
         have_colors = 0
         gdal_rat = 0x7ffff752f25f <find_file1+198>
#5 0x0000000000405231 in main (argc=3, argv=0x7fffffffdad8) at main.c:619
         nBand = 1
         input = 0x627d20
"/mnt/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt"
         output = 0x627dc0 "loss_global_forest_cover_loss_2000_2015"
         title = 0x0
         cellhd = {format = 0, compressed = 0, rows = 560000, rows3 =
560000, cols = 1440000, cols3 = 1440000, depths = 1, proj = 3, zone = 0,
           ew_res = 0.00025000000000000017, ew_res3 =
0.00025000000000000017, ns_res = 0.00025000000000000017, ns_res3 =
0.00025000000000000017, tb_res = 1,
           north = 80, south = -60.000000000000085, east =
180.00000000000023, west = -180, top = 1, bottom = 0}
         loc_wind = {format = 0, compressed = -1, rows = 1, rows3 = 1, cols
= 1, cols3 = 1, depths = 1, proj = 3, zone = 0, ew_res = 1, ew_res3 = 1,
           ns_res = 1, ns_res3 = 1, tb_res = 1, north = 1, south = 0, east
= 1, west = 0, top = 1, bottom = 0}
         cur_wind = {format = 0, compressed = 0, rows = 0, rows3 = 0, cols
= 0, cols3 = 0, depths = 0, proj = 0, zone = 0, ew_res = 0, ew_res3 = 0,
           ns_res = 0, ns_res3 = 0, tb_res = 0, north = 0, south = 0, east
= 0, west = 0, top = 0, bottom = 0}
         proj_info = 0x68ced0
         proj_units = 0x68cbf0
         loc_proj_info = 0x68e5f0
         loc_proj_units = 0x7db4f0
         hDS = 0x7dbb00
         hDriver = 0x62ef90
         hBand = 0x7dbcb0
         adfGeoTransform = {-180, 0.00025000000000000017, 0, 80, 0,
-0.00025000000000000017}
         n_bands = 0
         force_imagery = 0
         error_msg = "\253\000\000\000\022", '\000' <repeats 19 times>,
"i\000\000\000\022", '\000' <repeats 19 times>, "\222\000\000\000\022",
'\000' <repeats 19 times>, "\231\000\000\000\022", '\000' <repeats 19
times>, "\001\000\000\000
\000\000\000@\323\377\377\377\177\000\000\260\327\377\377\377\177\000\000\350
Y\372\367\377\177\000\000\060U\372\367\377\177\000\000\001\000\000\000\000\000\000\000\235\066\313\335\377\177\000\000\260Q\336\367\377\177\000\000\000\000\000\000\000\000\000\000D|\336\367\377\177\000\000`O\372\367\377\177\000\000\210\337\377\367\377\177\000\000\000\000\000\000\000\000\000\000"...
         projcomp_error = 1
         overwrite = 0
         offset = 0
         suffix = 0x629130 ""
         num_digits = 0
         module = 0x7ffff777ea28 <state+40>
         parm = {input = 0x7ffff777ea88 <state+136>, output = 0x626ea0,
target = 0x6270e0, title = 0x6271a0, outloc = 0x6274a0, band = 0x626f60,
           memory = 0x627020, offset = 0x627260, num_digits = 0x627320,
map_names_file = 0x6273e0, rat = 0x627560}
         flag_o = 0x7ffff777ea58 <state+88>
         flag_e = 0x626840
         flag_k = 0x627ab0
         flag_f = 0x627a10
         flag_l = 0x627a60
         flag_c = 0x627b00
         flag_p = 0x627b50
         flag_j = 0x626550
(gdb)
}}}

Doing a convert operation with GDAL, no apparent issues (I CTRL-C'd it
then):
{{{
gdal_translate
/mnt/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
/scratch/bla.tif
Input file size is 1440000, 560000
0..
}}}

**Data source:**
http://earthenginepartners.appspot.com/science-2013-global-
forest/download_v1.3.html

I created a VRT from the "loss" layer tiles which I then tried to import
as above:
{{{
gdalbuildvrt loss_global_forest_cover_loss_2000_2015.vrt *.tif

gdalinfo loss_global_forest_cover_loss_2000_2015.vrt
Driver: VRT/Virtual Raster
[...]
Size is 1440000, 560000
Coordinate System is:
GEOGCS["WGS 84",
     DATUM["WGS_1984",
         SPHEROID["WGS 84",6378137,298.257223563,
             AUTHORITY["EPSG","7030"]],
         AUTHORITY["EPSG","6326"]],
     PRIMEM["Greenwich",0],
     UNIT["degree",0.0174532925199433],
     AUTHORITY["EPSG","4326"]]
Origin = (-180.000000000000000,80.000000000000000)
Pixel Size = (0.000250000000000,-0.000250000000000)
Corner Coordinates:
Upper Left (-180.0000000, 80.0000000) (180d 0' 0.00"W, 80d 0' 0.00"N)
Lower Left (-180.0000000, -60.0000000) (180d 0' 0.00"W, 60d 0' 0.00"S)
Upper Right ( 180.0000000, 80.0000000) (180d 0' 0.00"E, 80d 0' 0.00"N)
Lower Right ( 180.0000000, -60.0000000) (180d 0' 0.00"E, 60d 0' 0.00"S)
Center ( 0.0000000, 10.0000000) ( 0d 0' 0.00"E, 10d 0' 0.00"N)
Band 1 Block=128x128 Type=Byte, ColorInterp=Gray
}}}

Test with GRASS GIS 7.3:

{{{
GRASS 7.3.svn (latlong):~/software/grass73_svn > r.in.gdal
input=/mnt/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Importing raster map <loss_global_forest_cover_loss_2000_2015>...
0% Segmentation fault (core dumped)
}}}

System:
{{{
gcc -v
...
gcc version 6.3.1 20161221 (Red Hat 6.3.1-1) (GCC)

uname -a
Linux fedora-calc 4.9.12-200.fc25.x86_64 #1 SMP Thu Feb 23 19:31:49 UTC
2017 x86_64 x86_64 x86_64 GNU/Linux
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3365>
GRASS GIS <https://grass.osgeo.org>

GDAL · July 3, 2017, 7:56pm

#3365: r.in.gdal: crash on large dataset
----------------------+---------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.2.2
Component: Raster | Version: svn-releasebranch72
Resolution: | Keywords: r.in.gdal
       CPU: x86-64 | Platform: Linux
----------------------+---------------------------------

Comment (by mmetz):

I have experienced problems with importing GDAL VRT raster maps before,
therefore decided to gdal_translate them before import.

Results of

1. running the import again with valgrind --tool=memcheck and posting the
valgrind log

2. gdal_translate'ing the VRT to tif, then import the tif with r.in.gdal
in order to test if this causes also a segfault

would help a lot.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3365#comment:1>
GRASS GIS <https://grass.osgeo.org>

GDAL · July 4, 2017, 12:51pm

#3365: r.in.gdal: crash on large dataset
----------------------+---------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.2.2
Component: Raster | Version: svn-releasebranch72
Resolution: | Keywords: r.in.gdal
       CPU: x86-64 | Platform: Linux
----------------------+---------------------------------

Old description:

Trying to import a large VRT dataset, I get a crash:

{{{
GRASS 7.2.2svn (latlong):~/software/grass72_svn_debugmode/bin.x86_64-pc-
linux-gnu > r.in.gdal
input=/mnt/kvm7-storage/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Proceeding with import of 1 raster bands...
Importing raster map <loss_global_forest_cover_loss_2000_2015>...
Segmentation fault (core dumped)
}}}

Debugging:
{{{
GRASS 7.2.2svn (latlong):~/software/grass72_svn_debugmode/bin.x86_64-pc-
linux-gnu > gdb r.in.gdal
GNU gdb (GDB) Fedora 7.12.1-48.fc25
Copyright (C) 2017 Free Software Foundation, Inc.
[...]
Reading symbols from r.in.gdal...done.
(gdb) r
input=/mnt//GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Starting program:
/home/mundialis/software/grass72_svn_debugmode/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal
input=/mnt//GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Missing separate debuginfos, use: dnf debuginfo-install
glibc-2.24-8.fc25.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Proceeding with import of 1 raster bands...
Importing raster map <loss_global_forest_cover_loss_2000_2015>...

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff77a1ab9 in put_data (fd=0, null_buf=0x7fffffe9b020 "",
cell=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0) at put_row.c:370
370 compressed_buf[0] = work_buf[0] = nbytes;

(gdb) bt full
#0 0x00007ffff77a1ab9 in put_data (fd=0, null_buf=0x7fffffe9b020 "",
cell=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0) at put_row.c:370
        wk = 0x7fffff91cb31 ""
        nbytes = 1
        compressed_buf = 0x7fffff7bd220 <error: Cannot access memory at
address 0x7fffff7bd220>
        total = 1440000
        fcb = 0x691080
        compressed = 1
        len = 4
        work_buf = 0x7fffff91cb30 "\001"
        wk = 0x7fffff91cb31 ""
        nwrite = 0
#1 0x00007ffff77a1f98 in put_raster_data (fd=0, null_buf=0x7fffffe9b020
"", rast=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0, map_type=0)
    at put_row.c:476
        fcb = 0x691080
#2 0x00007ffff77a2bdb in put_raster_row (fd=0, buf=0x7fffc62ac010,
data_type=0, zeros_r_nulls=0) at put_row.c:699
        convert_and_write_FtypeOtype = {{0x0, 0x7ffff77a23d9
<convert_and_write_if>, 0x7ffff77a25f5 <convert_and_write_id>}, {
            0x7ffff77a280f <convert_and_write_fi>, 0x0, 0x7ffff77a26f3
<convert_and_write_fd>}, {0x7ffff77a2928 <convert_and_write_di>,
            0x7ffff77a24d7 <convert_and_write_df>, 0x0}}
        fcb = 0x691080
        null_buf = 0x7fffffe9b020 ""
#3 0x00007ffff77a0f8b in Rast_put_row (fd=0, buf=0x7fffc62ac010,
data_type=0) at put_row.c:67
No locals.
#4 0x0000000000406e4d in ImportBand (hBand=0x7dbcb0, output=0x627dc0
"loss_global_forest_cover_loss_2000_2015", group_ref=0x0) at main.c:1141
        data_type = 0
        eGDT = GDT_Int32
        eRawGDT = GDT_Byte
        row = 1
        nrows = 560000
        ncols = 1440000
        complex = 0
        cf = 0
        cfR = 32767
        cfI = -145625496
        bNoDataEnabled = 0
        indx = 32767
        cell = 0x7fffc62ac010
        cellReal = 0x77
        cellImg = 0x7fffffffac40
        bufComplex = 0x7fffffffab40
        dfNoData = -10000
        outputReal =
"0\341\377\367\377\177\000\000\260\254\377\377\377\177\000\000\071\031@\000\000\000\000\000\243\366\251\242\000\000\000\000\377\377\377\377",
'\000' <repeats 12 times>,
"H\017\064\366\377\177\000\000\000\240\374\367\377\177\000\000\000\000\000\000\000\000\360?",
'\000' <repeats 14 times>, "\360?", '\000' <repeats 30 times>,
"\360?\000\000\000\000\000\000\000\000\220\062@\000\000\000\000\000\314\345w\367\377\177\000\000P\255\377\377\377\177\000\000\350\262`\000\000\000\000\000\220\062@\000\000\000\000\000\320\332\377\377\377\177",
'\000' <repeats 18 times>, "\360\331\377\377\377\177\000\000"...
        outputImg = '\000' <repeats 32 times>,
"\377\000\000\000\000\000\377\377\000\000\000\000\000\000\377\377\000\000\000\000\000\000\377\377\377\377\377\377\000\377\377\377",
'\000' <repeats 22 times>, "\360?", '\000' <repeats 14 times>, "\360?",
'\000' <repeats 14 times>,
"\360?\000\000\000\000\000\000\000\000\210\345w\367\377\177", '\000'
<repeats 26 times>,
"\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\200\256\377\377\377\177\000\000\240\224z\367\377\177\000\000\000\273}",
'\000' <repeats 13 times>...
        nullFlags = 0x0
        history = {fields = {0x7ffff7fca000 "", 0x7fffffffabe8 "",
0x7fffffffabe4 "", 0x7ffff7ff7b78 "\350\343\377\367\377\177",
            0x7fffffffaba0 "\300\254\377\377\377\177", 0x401939
"GDALGetRasterBand", 0x400be8 "\261\a", 0x7fffffffabe8 ""}, nlines =
-1565919581,
          lines = 0x28aa7da}
        GDALmetadata = 0x7ffff77aa094
        have_colors = 0
        gdal_rat = 0x7ffff752f25f <find_file1+198>
#5 0x0000000000405231 in main (argc=3, argv=0x7fffffffdad8) at
main.c:619
        nBand = 1
        input = 0x627d20
"/mnt/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt"
        output = 0x627dc0 "loss_global_forest_cover_loss_2000_2015"
        title = 0x0
        cellhd = {format = 0, compressed = 0, rows = 560000, rows3 =
560000, cols = 1440000, cols3 = 1440000, depths = 1, proj = 3, zone = 0,
          ew_res = 0.00025000000000000017, ew_res3 =
0.00025000000000000017, ns_res = 0.00025000000000000017, ns_res3 =
0.00025000000000000017, tb_res = 1,
          north = 80, south = -60.000000000000085, east =
180.00000000000023, west = -180, top = 1, bottom = 0}
        loc_wind = {format = 0, compressed = -1, rows = 1, rows3 = 1,
cols = 1, cols3 = 1, depths = 1, proj = 3, zone = 0, ew_res = 1, ew_res3
= 1,
          ns_res = 1, ns_res3 = 1, tb_res = 1, north = 1, south = 0, east
= 1, west = 0, top = 1, bottom = 0}
        cur_wind = {format = 0, compressed = 0, rows = 0, rows3 = 0, cols
= 0, cols3 = 0, depths = 0, proj = 0, zone = 0, ew_res = 0, ew_res3 = 0,
          ns_res = 0, ns_res3 = 0, tb_res = 0, north = 0, south = 0, east
= 0, west = 0, top = 0, bottom = 0}
        proj_info = 0x68ced0
        proj_units = 0x68cbf0
        loc_proj_info = 0x68e5f0
        loc_proj_units = 0x7db4f0
        hDS = 0x7dbb00
        hDriver = 0x62ef90
        hBand = 0x7dbcb0
        adfGeoTransform = {-180, 0.00025000000000000017, 0, 80, 0,
-0.00025000000000000017}
        n_bands = 0
        force_imagery = 0
        error_msg = "\253\000\000\000\022", '\000' <repeats 19 times>,
"i\000\000\000\022", '\000' <repeats 19 times>, "\222\000\000\000\022",
'\000' <repeats 19 times>, "\231\000\000\000\022", '\000' <repeats 19
times>, "\001\000\000\000
\000\000\000@\323\377\377\377\177\000\000\260\327\377\377\377\177\000\000\350
Y\372\367\377\177\000\000\060U\372\367\377\177\000\000\001\000\000\000\000\000\000\000\235\066\313\335\377\177\000\000\260Q\336\367\377\177\000\000\000\000\000\000\000\000\000\000D|\336\367\377\177\000\000`O\372\367\377\177\000\000\210\337\377\367\377\177\000\000\000\000\000\000\000\000\000\000"...
        projcomp_error = 1
        overwrite = 0
        offset = 0
        suffix = 0x629130 ""
        num_digits = 0
        module = 0x7ffff777ea28 <state+40>
        parm = {input = 0x7ffff777ea88 <state+136>, output = 0x626ea0,
target = 0x6270e0, title = 0x6271a0, outloc = 0x6274a0, band = 0x626f60,
          memory = 0x627020, offset = 0x627260, num_digits = 0x627320,
map_names_file = 0x6273e0, rat = 0x627560}
        flag_o = 0x7ffff777ea58 <state+88>
        flag_e = 0x626840
        flag_k = 0x627ab0
        flag_f = 0x627a10
        flag_l = 0x627a60
        flag_c = 0x627b00
        flag_p = 0x627b50
        flag_j = 0x626550
(gdb)
}}}

Doing a convert operation with GDAL, no apparent issues (I CTRL-C'd it
then):
{{{
gdal_translate
/mnt/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
/scratch/bla.tif
Input file size is 1440000, 560000
0..
}}}

**Data source:**
http://earthenginepartners.appspot.com/science-2013-global-
forest/download_v1.3.html

I created a VRT from the "loss" layer tiles which I then tried to import
as above:
{{{
gdalbuildvrt loss_global_forest_cover_loss_2000_2015.vrt *.tif

gdalinfo loss_global_forest_cover_loss_2000_2015.vrt
Driver: VRT/Virtual Raster
[...]
Size is 1440000, 560000
Coordinate System is:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433],
    AUTHORITY["EPSG","4326"]]
Origin = (-180.000000000000000,80.000000000000000)
Pixel Size = (0.000250000000000,-0.000250000000000)
Corner Coordinates:
Upper Left (-180.0000000, 80.0000000) (180d 0' 0.00"W, 80d 0' 0.00"N)
Lower Left (-180.0000000, -60.0000000) (180d 0' 0.00"W, 60d 0' 0.00"S)
Upper Right ( 180.0000000, 80.0000000) (180d 0' 0.00"E, 80d 0' 0.00"N)
Lower Right ( 180.0000000, -60.0000000) (180d 0' 0.00"E, 60d 0' 0.00"S)
Center ( 0.0000000, 10.0000000) ( 0d 0' 0.00"E, 10d 0' 0.00"N)
Band 1 Block=128x128 Type=Byte, ColorInterp=Gray
}}}

Test with GRASS GIS 7.3:

{{{
GRASS 7.3.svn (latlong):~/software/grass73_svn > r.in.gdal
input=/mnt/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Importing raster map <loss_global_forest_cover_loss_2000_2015>...
0% Segmentation fault (core dumped)
}}}

System:
{{{
gcc -v
...
gcc version 6.3.1 20161221 (Red Hat 6.3.1-1) (GCC)

uname -a
Linux fedora-calc 4.9.12-200.fc25.x86_64 #1 SMP Thu Feb 23 19:31:49 UTC
2017 x86_64 x86_64 x86_64 GNU/Linux
}}}

New description:

Trying to import a large VRT dataset, I get a crash:

{{{
GRASS 7.2.2svn (latlong):~/software/grass72_svn_debugmode/bin.x86_64-pc-
linux-gnu > r.in.gdal
input=/mnt/kvm7-storage/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Proceeding with import of 1 raster bands...
Importing raster map <loss_global_forest_cover_loss_2000_2015>...
Segmentation fault (core dumped)
}}}

Debugging:
{{{
GRASS 7.2.2svn (latlong):~/software/grass72_svn_debugmode/bin.x86_64-pc-
linux-gnu > gdb r.in.gdal
GNU gdb (GDB) Fedora 7.12.1-48.fc25
Copyright (C) 2017 Free Software Foundation, Inc.
[...]
Reading symbols from r.in.gdal...done.
(gdb) r
input=/mnt//GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Starting program:
/home/mundialis/software/grass72_svn_debugmode/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal
input=/mnt//GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Missing separate debuginfos, use: dnf debuginfo-install
glibc-2.24-8.fc25.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Proceeding with import of 1 raster bands...
Importing raster map <loss_global_forest_cover_loss_2000_2015>...

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff77a1ab9 in put_data (fd=0, null_buf=0x7fffffe9b020 "",
cell=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0) at put_row.c:370
370 compressed_buf[0] = work_buf[0] = nbytes;

(gdb) bt full
#0 0x00007ffff77a1ab9 in put_data (fd=0, null_buf=0x7fffffe9b020 "",
cell=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0) at put_row.c:370
         wk = 0x7fffff91cb31 ""
         nbytes = 1
         compressed_buf = 0x7fffff7bd220 <error: Cannot access memory at
address 0x7fffff7bd220>
         total = 1440000
         fcb = 0x691080
         compressed = 1
         len = 4
         work_buf = 0x7fffff91cb30 "\001"
         wk = 0x7fffff91cb31 ""
         nwrite = 0
#1 0x00007ffff77a1f98 in put_raster_data (fd=0, null_buf=0x7fffffe9b020
"", rast=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0, map_type=0)
     at put_row.c:476
         fcb = 0x691080
#2 0x00007ffff77a2bdb in put_raster_row (fd=0, buf=0x7fffc62ac010,
data_type=0, zeros_r_nulls=0) at put_row.c:699
         convert_and_write_FtypeOtype = {{0x0, 0x7ffff77a23d9
<convert_and_write_if>, 0x7ffff77a25f5 <convert_and_write_id>}, {
             0x7ffff77a280f <convert_and_write_fi>, 0x0, 0x7ffff77a26f3
<convert_and_write_fd>}, {0x7ffff77a2928 <convert_and_write_di>,
             0x7ffff77a24d7 <convert_and_write_df>, 0x0}}
         fcb = 0x691080
         null_buf = 0x7fffffe9b020 ""
#3 0x00007ffff77a0f8b in Rast_put_row (fd=0, buf=0x7fffc62ac010,
data_type=0) at put_row.c:67
No locals.
#4 0x0000000000406e4d in ImportBand (hBand=0x7dbcb0, output=0x627dc0
"loss_global_forest_cover_loss_2000_2015", group_ref=0x0) at main.c:1141
         data_type = 0
         eGDT = GDT_Int32
         eRawGDT = GDT_Byte
         row = 1
         nrows = 560000
         ncols = 1440000
         complex = 0
         cf = 0
         cfR = 32767
         cfI = -145625496
         bNoDataEnabled = 0
         indx = 32767
         cell = 0x7fffc62ac010
         cellReal = 0x77
         cellImg = 0x7fffffffac40
         bufComplex = 0x7fffffffab40
         dfNoData = -10000
         outputReal =
"0\341\377\367\377\177\000\000\260\254\377\377\377\177\000\000\071\031@\000\000\000\000\000\243\366\251\242\000\000\000\000\377\377\377\377",
'\000' <repeats 12 times>,
"H\017\064\366\377\177\000\000\000\240\374\367\377\177\000\000\000\000\000\000\000\000\360?",
'\000' <repeats 14 times>, "\360?", '\000' <repeats 30 times>,
"\360?\000\000\000\000\000\000\000\000\220\062@\000\000\000\000\000\314\345w\367\377\177\000\000P\255\377\377\377\177\000\000\350\262`\000\000\000\000\000\220\062@\000\000\000\000\000\320\332\377\377\377\177",
'\000' <repeats 18 times>, "\360\331\377\377\377\177\000\000"...
         outputImg = '\000' <repeats 32 times>,
"\377\000\000\000\000\000\377\377\000\000\000\000\000\000\377\377\000\000\000\000\000\000\377\377\377\377\377\377\000\377\377\377",
'\000' <repeats 22 times>, "\360?", '\000' <repeats 14 times>, "\360?",
'\000' <repeats 14 times>,
"\360?\000\000\000\000\000\000\000\000\210\345w\367\377\177", '\000'
<repeats 26 times>,
"\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\200\256\377\377\377\177\000\000\240\224z\367\377\177\000\000\000\273}",
'\000' <repeats 13 times>...
         nullFlags = 0x0
         history = {fields = {0x7ffff7fca000 "", 0x7fffffffabe8 "",
0x7fffffffabe4 "", 0x7ffff7ff7b78 "\350\343\377\367\377\177",
             0x7fffffffaba0 "\300\254\377\377\377\177", 0x401939
"GDALGetRasterBand", 0x400be8 "\261\a", 0x7fffffffabe8 ""}, nlines =
-1565919581,
           lines = 0x28aa7da}
         GDALmetadata = 0x7ffff77aa094
         have_colors = 0
         gdal_rat = 0x7ffff752f25f <find_file1+198>
#5 0x0000000000405231 in main (argc=3, argv=0x7fffffffdad8) at main.c:619
         nBand = 1
         input = 0x627d20
"/mnt/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt"
         output = 0x627dc0 "loss_global_forest_cover_loss_2000_2015"
         title = 0x0
         cellhd = {format = 0, compressed = 0, rows = 560000, rows3 =
560000, cols = 1440000, cols3 = 1440000, depths = 1, proj = 3, zone = 0,
           ew_res = 0.00025000000000000017, ew_res3 =
0.00025000000000000017, ns_res = 0.00025000000000000017, ns_res3 =
0.00025000000000000017, tb_res = 1,
           north = 80, south = -60.000000000000085, east =
180.00000000000023, west = -180, top = 1, bottom = 0}
         loc_wind = {format = 0, compressed = -1, rows = 1, rows3 = 1, cols
= 1, cols3 = 1, depths = 1, proj = 3, zone = 0, ew_res = 1, ew_res3 = 1,
           ns_res = 1, ns_res3 = 1, tb_res = 1, north = 1, south = 0, east
= 1, west = 0, top = 1, bottom = 0}
         cur_wind = {format = 0, compressed = 0, rows = 0, rows3 = 0, cols
= 0, cols3 = 0, depths = 0, proj = 0, zone = 0, ew_res = 0, ew_res3 = 0,
           ns_res = 0, ns_res3 = 0, tb_res = 0, north = 0, south = 0, east
= 0, west = 0, top = 0, bottom = 0}
         proj_info = 0x68ced0
         proj_units = 0x68cbf0
         loc_proj_info = 0x68e5f0
         loc_proj_units = 0x7db4f0
         hDS = 0x7dbb00
         hDriver = 0x62ef90
         hBand = 0x7dbcb0
         adfGeoTransform = {-180, 0.00025000000000000017, 0, 80, 0,
-0.00025000000000000017}
         n_bands = 0
         force_imagery = 0
         error_msg = "\253\000\000\000\022", '\000' <repeats 19 times>,
"i\000\000\000\022", '\000' <repeats 19 times>, "\222\000\000\000\022",
'\000' <repeats 19 times>, "\231\000\000\000\022", '\000' <repeats 19
times>, "\001\000\000\000
\000\000\000@\323\377\377\377\177\000\000\260\327\377\377\377\177\000\000\350
Y\372\367\377\177\000\000\060U\372\367\377\177\000\000\001\000\000\000\000\000\000\000\235\066\313\335\377\177\000\000\260Q\336\367\377\177\000\000\000\000\000\000\000\000\000\000D|\336\367\377\177\000\000`O\372\367\377\177\000\000\210\337\377\367\377\177\000\000\000\000\000\000\000\000\000\000"...
         projcomp_error = 1
         overwrite = 0
         offset = 0
         suffix = 0x629130 ""
         num_digits = 0
         module = 0x7ffff777ea28 <state+40>
         parm = {input = 0x7ffff777ea88 <state+136>, output = 0x626ea0,
target = 0x6270e0, title = 0x6271a0, outloc = 0x6274a0, band = 0x626f60,
           memory = 0x627020, offset = 0x627260, num_digits = 0x627320,
map_names_file = 0x6273e0, rat = 0x627560}
         flag_o = 0x7ffff777ea58 <state+88>
         flag_e = 0x626840
         flag_k = 0x627ab0
         flag_f = 0x627a10
         flag_l = 0x627a60
         flag_c = 0x627b00
         flag_p = 0x627b50
         flag_j = 0x626550
(gdb)
}}}

Doing a convert operation with GDAL, no apparent issues (I CTRL-C'd it
then):
{{{
gdal_translate
/mnt/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
/scratch/bla.tif
Input file size is 1440000, 560000
0..
}}}

**Data source:**
http://earthenginepartners.appspot.com/science-2013-global-
forest/download_v1.3.html

I created a VRT from the "loss" layer tiles which I then tried to import
as above:
{{{
gdalbuildvrt loss_global_forest_cover_loss_2000_2015.vrt *.tif

gdalinfo loss_global_forest_cover_loss_2000_2015.vrt
Driver: VRT/Virtual Raster
[...]
Size is 1440000, 560000
Coordinate System is:
GEOGCS["WGS 84",
     DATUM["WGS_1984",
         SPHEROID["WGS 84",6378137,298.257223563,
             AUTHORITY["EPSG","7030"]],
         AUTHORITY["EPSG","6326"]],
     PRIMEM["Greenwich",0],
     UNIT["degree",0.0174532925199433],
     AUTHORITY["EPSG","4326"]]
Origin = (-180.000000000000000,80.000000000000000)
Pixel Size = (0.000250000000000,-0.000250000000000)
Corner Coordinates:
Upper Left (-180.0000000, 80.0000000) (180d 0' 0.00"W, 80d 0' 0.00"N)
Lower Left (-180.0000000, -60.0000000) (180d 0' 0.00"W, 60d 0' 0.00"S)
Upper Right ( 180.0000000, 80.0000000) (180d 0' 0.00"E, 80d 0' 0.00"N)
Lower Right ( 180.0000000, -60.0000000) (180d 0' 0.00"E, 60d 0' 0.00"S)
Center ( 0.0000000, 10.0000000) ( 0d 0' 0.00"E, 10d 0' 0.00"N)
Band 1 Block=128x128 Type=Byte, ColorInterp=Gray
}}}

Test with GRASS GIS 7.3:

{{{
GRASS 7.3.svn (latlong):~/software/grass73_svn > r.in.gdal
input=/mnt/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Importing raster map <loss_global_forest_cover_loss_2000_2015>...
0% Segmentation fault (core dumped)
}}}

System:
{{{
gcc -v
...
gcc version 6.3.1 20161221 (Red Hat 6.3.1-1) (GCC)

uname -a
Linux fedora-calc 4.9.12-200.fc25.x86_64 #1 SMP Thu Feb 23 19:31:49 UTC
2017 x86_64 x86_64 x86_64 GNU/Linux
}}}

--

Comment (by neteler):

Using valgrind:

{{{
GRASS 7.3.svn (latlong):~ > valgrind $CMD
==30211== Memcheck, a memory error detector
==30211== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==30211== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright
info
==30211== Command: r.in.gdal
input=/mnt/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
==30211==
Importing raster map <loss_global_forest_cover_loss_2000_2015>...
==30211== Warning: client switching stacks? SP change: 0xffee9d140 -->
0xffe91ed30
==30211== to suppress, use: --max-stackframe=5760016 or greater
==30211== Invalid write of size 8
==30211== at 0x5273D89: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/lib/libgrass_raster.7.3.svn.so)
==30211== by 0x52744B4: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/lib/libgrass_raster.7.3.svn.so)
==30211== by 0x4063DE: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal)
==30211== by 0x404F64: main (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal)
==30211== Address 0xffe91ed28 is on thread 1's stack
==30211==
==30211== Invalid read of size 8
==30211== at 0x6D63E3F: lseek (in /usr/lib64/libc-2.24.so)
==30211== by 0x5273D8D: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/lib/libgrass_raster.7.3.svn.so)
==30211== by 0x52744B4: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/lib/libgrass_raster.7.3.svn.so)
==30211== by 0x4063DE: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal)
==30211== by 0x404F64: main (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal)
==30211== Address 0xffe91ed28 is on thread 1's stack
==30211== in frame #0, created by lseek (???
==30211==
==30211== Invalid write of size 1
==30211== at 0x527378C: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/lib/libgrass_raster.7.3.svn.so)
==30211== by 0x52744B4: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/lib/libgrass_raster.7.3.svn.so)
==30211== by 0x4063DE: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal)
==30211== by 0x404F64: main (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal)
==30211== Address 0xffe91ed34 is on thread 1's stack
==30211==
==30211== Invalid write of size 1
==30211== at 0x5273798: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/lib/libgrass_raster.7.3.svn.so)
==30211== by 0x52744B4: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/lib/libgrass_raster.7.3.svn.so)
==30211== by 0x4063DE: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal)
==30211== by 0x404F64: main (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal)
==30211== Address 0xffe91ed32 is on thread 1's stack
==30211==
==30211== Invalid read of size 1
==30211== at 0x527384E: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/lib/libgrass_raster.7.3.svn.so)
==30211== by 0x52744B4: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/lib/libgrass_raster.7.3.svn.so)
==30211== by 0x4063DE: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal)
==30211== by 0x404F64: main (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal)
==30211== Address 0xffe91ed31 is on thread 1's stack
==30211==
==30211==
==30211== More than 10000000 total errors detected. I'm not reporting any
more.
==30211== Final error counts will be inaccurate. Go fix your program!
==30211== Rerun with --error-limit=no to disable this cutoff. Note
==30211== that errors may occur in your program without prior warning from
==30211== Valgrind, because errors are no longer being displayed.
==30211==
==30211==
==30211== Process terminating with default action of signal 11 (SIGSEGV):
dumping core
==30211== Access not within mapped region at address 0xFFE7BF420
==30211== at 0x5273CBA: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/lib/libgrass_raster.7.3.svn.so)
==30211== If you believe this happened as a result of a stack
==30211== overflow in your program's main thread (unlikely but
==30211== possible), you can try to increase the size of the
==30211== main thread stack using the --main-stacksize= flag.
==30211== The main thread stack size used in this run was 8388608.
==30211==
==30211== Process terminating with default action of signal 11 (SIGSEGV)
==30211== Access not within mapped region at address 0xFFE7BF418
==30211== at 0x4A286C0: _vgnU_freeres (vg_preloaded.c:59)
==30211== If you believe this happened as a result of a stack
==30211== overflow in your program's main thread (unlikely but
==30211== possible), you can try to increase the size of the
==30211== main thread stack using the --main-stacksize= flag.
==30211== The main thread stack size used in this run was 8388608.
==30211==
==30211== HEAP SUMMARY:
==30211== in use at exit: 106,603,369 bytes in 11,066 blocks
==30211== total heap usage: 78,802 allocs, 67,736 frees, 136,593,845
bytes allocated
==30211==
==30211== LEAK SUMMARY:
==30211== definitely lost: 134 bytes in 5 blocks
==30211== indirectly lost: 353 bytes in 20 blocks
==30211== possibly lost: 0 bytes in 0 blocks
==30211== still reachable: 106,602,882 bytes in 11,041 blocks
==30211== suppressed: 0 bytes in 0 blocks
==30211== Rerun with --leak-check=full to see details of leaked memory
==30211==
==30211== For counts of detected and suppressed errors, rerun with: -v
==30211== ERROR SUMMARY: 10000000 errors from 5 contexts (suppressed: 0
from 0)
Segmentation fault (core dumped)
}}}

Next I converted the VRT to GeoTIFF and tried to import that:

{{{
GRASS 7.3.svn (latlong):~ > nice gdal_translate -co "COMPRESS=DEFLATE"
/mnt/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
/scratch/loss_global_forest_cover_loss_2000_2015.tif
Input file size is 1440000, 560000
0..10...20...30...40...50...60...70...80...90...100 - done.

GRASS 7.3.svn (latlong):~ > r.in.gdal
/scratch/loss_global_forest_cover_loss_2000_2015.tif
out=loss_global_forest_cover_loss_2000_2015
Importing raster map <loss_global_forest_cover_loss_2000_2015>...
Segmentation fault (core dumped)
}}}

The debug results are the same as before:

{{{
GRASS 7.2.2svn (latlong):~/software/grass72_svn_debugmode/bin.x86_64-pc-
linux-gnu > gdb r.in.gdal
r GNU gdb (GDB) Fedora 7.12.1-48.fc25
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>\.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>\.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from r.in.gdal...done.
(gdb) r /scratch/loss_global_forest_cover_loss_2000_2015.tif
out=loss_global_forest_cover_loss_2000_2015
Starting program:
/home/mundialis/software/grass72_svn_debugmode/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal /scratch/loss_global_forest_cover_loss_2000_2015.tif
out=loss_global_forest_cover_loss_2000_2015
Missing separate debuginfos, use: dnf debuginfo-install
glibc-2.24-8.fc25.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Proceeding with import of 1 raster bands...
Importing raster map <loss_global_forest_cover_loss_2000_2015>...

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff77a1ab9 in put_data (fd=0, null_buf=0x7fffffe9b080 "",
cell=0x7fffc57fd010, row=0, n=1440000, zeros_r_nulls=0) at put_row.c:370
370 compressed_buf[0] = work_buf[0] = nbytes;

}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3365#comment:2>
GRASS GIS <https://grass.osgeo.org>

GDAL · July 4, 2017, 7:41pm

#3365: r.in.gdal: crash on large dataset
----------------------+---------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.2.2
Component: Raster | Version: svn-releasebranch72
Resolution: | Keywords: r.in.gdal
       CPU: x86-64 | Platform: Linux
----------------------+---------------------------------

Comment (by neteler):

I got an idea how to more easily reproduce the problem with less data: it
is sufficient to only download the four corner tiles and create a VRT from
it:

{{{
# download tiles
wget -c https://storage.googleapis.com/earthenginepartners-
hansen/GFC-2015-v1.3/Hansen_GFC-2015-v1.3_loss_50S_170E.tif
wget -c https://storage.googleapis.com/earthenginepartners-
hansen/GFC-2015-v1.3/Hansen_GFC-2015-v1.3_loss_50S_180W.tif
wget -c https://storage.googleapis.com/earthenginepartners-
hansen/GFC-2015-v1.3/Hansen_GFC-2015-v1.3_loss_80N_170E.tif
wget -c https://storage.googleapis.com/earthenginepartners-
hansen/GFC-2015-v1.3/Hansen_GFC-2015-v1.3_loss_80N_180W.tif

# create VRT
gdalbuildvrt forest_loss_corner_tiles.vrt
Hansen_GFC-2015-v1.3_loss_50S_170E.tif
Hansen_GFC-2015-v1.3_loss_50S_180W.tif
Hansen_GFC-2015-v1.3_loss_80N_170E.tif
Hansen_GFC-2015-v1.3_loss_80N_180W.tif

# import
r.in.gdal input=forest_loss_corner_tiles.vrt
output=loss_global_forest_corner_tiles
Segmentation fault (core dumped)
}}}

Looks like an overflow with the large amount of rows/cols.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3365#comment:3>
GRASS GIS <https://grass.osgeo.org>

GDAL · July 4, 2017, 7:42pm

#3365: r.in.gdal: crash on large dataset
----------------------+---------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.2.2
Component: Raster | Version: svn-releasebranch72
Resolution: | Keywords: r.in.gdal
       CPU: x86-64 | Platform: Linux
----------------------+---------------------------------

Comment (by mmetz):

Replying to [comment:2 neteler]:
> Using valgrind:
>
> {{{
> GRASS 7.3.svn (latlong):~ > valgrind $CMD
> ==30211== Memcheck, a memory error detector
> ==30211== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et
al.
> ==30211== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright
info
> ==30211== Command: r.in.gdal
input=/mnt/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
> ==30211==
> Importing raster map <loss_global_forest_cover_loss_2000_2015>...
> ==30211== Warning: client switching stacks? SP change: 0xffee9d140 -->
0xffe91ed30
> ==30211== to suppress, use: --max-stackframe=5760016 or greater
> ==30211== Invalid write of size 8
> ==30211== at 0x5273D89: ??? (in
/home/mundialis/software/grass73_svn/dist.x86_64-pc-linux-
gnu/lib/libgrass_raster.7.3.svn.so)
> }}}

Maybe you need to recompile GRASS with -g in order to replace "???" with
some real information.

I could reproduce the problem, the segfault is caused by stack overflow
because alloca allocates memory in the stack, while malloc and calloc
allocate memory in the heap. I could fix the segfault by replacing
G_alloca()/G_freea() with G_malloc()/G_free(). Please try the attached
patch against lib/raster/.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3365#comment:4>
GRASS GIS <https://grass.osgeo.org>

GDAL · July 4, 2017, 7:43pm

#3365: r.in.gdal: crash on large dataset
----------------------+---------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.2.2
Component: Raster | Version: svn-releasebranch72
Resolution: | Keywords: r.in.gdal
       CPU: x86-64 | Platform: Linux
----------------------+---------------------------------
Changes (by mmetz):

* Attachment "rasterlib.patch" added.

patch for rasterlib to replace G_alloca for reading rows (avoids stack
overflow)

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3365>
GRASS GIS <https://grass.osgeo.org>

GDAL · July 4, 2017, 10:26pm

#3365: r.in.gdal: crash on large dataset
----------------------+---------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.2.2
Component: Raster | Version: svn-releasebranch72
Resolution: | Keywords: r.in.gdal
       CPU: x86-64 | Platform: Linux
----------------------+---------------------------------

Comment (by neteler):

Replying to [comment:4 mmetz]:
...
> Maybe you need to recompile GRASS with -g in order to replace "???" with
some real information.

My bad, I was calling the wrong grass72 version without debugging symbols
in...

> I could reproduce the problem, the segfault is caused by stack overflow
because alloca allocates memory in the stack, while malloc and calloc
allocate memory in the heap. I could fix the segfault by replacing
G_alloca()/G_freea() with G_malloc()/G_free(). Please try the attached
patch against lib/raster/.

Thanks! r.in.gdal now runs, no more immediate crash. In some hours the
dataset may be imported, I'll report.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3365#comment:5>
GRASS GIS <https://grass.osgeo.org>

GDAL · July 5, 2017, 6:02am

#3365: r.in.gdal: crash on large dataset
----------------------+---------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.2.2
Component: Raster | Version: svn-releasebranch72
Resolution: | Keywords: r.in.gdal
       CPU: x86-64 | Platform: Linux
----------------------+---------------------------------

Comment (by neteler):

Works!

{{{
# note: mounted disk connected from a different server via NFS
GRASS 7.2.2svn (latlong) > date ; r.in.gdal
input=/mnt/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015 ; date
Wed Jul 5 00:22:36 CEST 2017
Proceeding with import of 1 raster bands...
Importing raster map <loss_global_forest_cover_loss_2000_2015>...
100%
Wed Jul 5 03:33:54 CEST 2017

GRASS 7.2.2svn (latlong): > g.region
raster=loss_global_forest_cover_loss_2000_2015 -p
projection: 3 (Latitude-Longitude)
zone: 0
datum: wgs84
ellipsoid: wgs84
north: 80N
south: 60S
west: 180W
east: 180E
nsres: 0:00:00.9
ewres: 0:00:00.9
rows: 560000
cols: 1440000
cells: 806400000000

# d.rast shows this easily (20sec over ssh connected d.mon wx0)
}}}

Looks like a very useful patch thanks.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3365#comment:6>
GRASS GIS <https://grass.osgeo.org>

GDAL · July 5, 2017, 6:48am

#3365: r.in.gdal: crash on large dataset
----------------------+---------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.2.2
Component: Raster | Version: svn-releasebranch72
Resolution: | Keywords: r.in.gdal
       CPU: x86-64 | Platform: Linux
----------------------+---------------------------------

Comment (by mmetz):

Replying to [comment:6 neteler]:
> Works!
>
> [...]
>
> Looks like a very useful patch thanks.

Patch applied to trunk and relbr72 in r71241,2.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3365#comment:7>
GRASS GIS <https://grass.osgeo.org>

GDAL · July 5, 2017, 4:32pm

#3365: r.in.gdal: crash on large dataset
----------------------+---------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: closed
  Priority: normal | Milestone: 7.2.2
Component: Raster | Version: svn-releasebranch72
Resolution: fixed | Keywords: r.in.gdal
       CPU: x86-64 | Platform: Linux
----------------------+---------------------------------
Changes (by neteler):

* status: new => closed
* resolution: => fixed

Comment:

Thanks a lot, closing.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3365#comment:8>
GRASS GIS <https://grass.osgeo.org>