[GRASS-dev] [GRASS GIS] #1006: r.terraflow fails to create temp file on Windows when launched from wxgui

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
----------------------------------------+-----------------------------------
Reporter: marisn | Owner: grass-dev@lists.osgeo.org
     Type: defect | Status: new
Priority: normal | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Keywords: wxgui, wingrass, terraflow | Platform: MSWindows Vista
      Cpu: x86-32 |
----------------------------------------+-----------------------------------
Temporary file folder (specified as STREAM_DIR) exists and is writeable
for current user.

Run r.terraflow with default STREAM_DIR option within wxgui:

{{{
Sun Mar 14 18:47:32 2010)
r.terraflow elevation=reljefs_rst_t100@zonde filled=floddeed
direction=dirrectionss swatershed=sinkss accumulation=akumulacija2
tci=tcii
stats.out: Permission denied
STREAM temporary files in C:/Users/Maris/AppData/Local/Temp
(THESE INTERMEDIATE STREAMS WILL NOT BE DELETED IN CASE OF
ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE
DELETE THESE FILES MANUALLY!)
(Sun Mar 14 18:47:32 2010) Command finished (0 sec)
}}}

Running the same from CLI:

{{{
GRASS 6.4.0svn (latvija3)> r.terraflow elevation=reljefs_rst_t100
filled=fffllll
l direction=diiiirrr swatershed=siiiknk accumulation=aacccum tci=tttopoc
STREAM temporary files in C:/Users/Maris/AppData/Local/Temp (THESE
INTERMEDIATE
  STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE
PROGRAM. TO
SAVE SPACE PLEASE DELETE THESE FILES MANUALLY!)
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 300.00M (314572800) bytes
Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
Assertion failed: nrows * ncols == str->stream_len(), file grass2str.h,
line 144

This application has requested the Runtime to terminate it in an unusual
way.
Please contact the application's support team for more information.
}}}

WinGRASS-6.4.SVN-r41428 on Windows Vista

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1006&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by marisn):

Aaah - just noticed - could this be related to #520?

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1006#comment:1&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hellik):

Replying to [ticket:1006 marisn]:
> Temporary file folder (specified as STREAM_DIR) exists and is writeable
for current user.
>
> Run r.terraflow with default STREAM_DIR option within wxgui:
>
> {{{
> Sun Mar 14 18:47:32 2010)
> r.terraflow elevation=reljefs_rst_t100@zonde filled=floddeed
direction=dirrectionss swatershed=sinkss accumulation=akumulacija2
tci=tcii
> stats.out: Permission denied
> STREAM temporary files in C:/Users/Maris/AppData/Local/Temp
> (THESE INTERMEDIATE STREAMS WILL NOT BE DELETED IN CASE OF
> ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE
> DELETE THESE FILES MANUALLY!)
> (Sun Mar 14 18:47:32 2010) Command finished (0 sec)
> }}}
>
> Running the same from CLI:
>
> {{{
> GRASS 6.4.0svn (latvija3)> r.terraflow elevation=reljefs_rst_t100
filled=fffllll
> l direction=diiiirrr swatershed=siiiknk accumulation=aacccum tci=tttopoc
> STREAM temporary files in C:/Users/Maris/AppData/Local/Temp (THESE
INTERMEDIATE
> STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE
PROGRAM. TO
> SAVE SPACE PLEASE DELETE THESE FILES MANUALLY!)
> MFD flow direction
> D8CUT=999999986991104.000000
> Memory size: 300.00M (314572800) bytes
> Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
> Assertion failed: nrows * ncols == str->stream_len(), file grass2str.h,
line 144
>
>
> This application has requested the Runtime to terminate it in an unusual
way.
> Please contact the application's support team for more information.
> }}}
>
> WinGRASS-6.4.SVN-r41428 on Windows Vista

I've tried this on my WinVista32-box, from command line and wx-gui, the
intermediate temp files are partly written in
C:\Users\syringia\AppData\Local\Temp, but r.terraflow fails with

{{{
Assertion failed: nrows * ncols == str->stream_len(), file grass2str.h,
line 144
}}}

Helmut

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1006#comment:2&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hamish):

what does "r.terraflow --help" say ?

(specifically the STREAM_DIR section)

{{{
   /* temporary STREAM path */
   struct Option *streamdir;
   streamdir = G_define_option() ;
   streamdir->key = "STREAM_DIR";
   streamdir->type = TYPE_STRING;
   streamdir->required = NO;
#ifdef __MINGW32__
   streamdir->answer =
G_convert_dirseps_from_host(G_store(getenv("TEMP")));
#else
   streamdir->answer = G_store("/var/tmp/");
#endif
   streamdir->description=
      _("Directory to hold temporary files (they can be large)");
}}}

Maris:
> Aaah - just noticed - could this be related to #520?

probably not as that should be already fixed. (just not ideal)

can you try on a small (file size) raster like elevation.dem @ Spearfish?
(existing 300mb doesn't seem so bad though)

can you try with --verbose so percent done is shown for nrows and we get
more details?

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:3&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hellik):

Replying to [comment:3 hamish]:
> what does "r.terraflow --help" say ?

{{{
GRASS 6.4> r.terraflow --help

Beschreibung:
  Flow computation for massive grids (Float version).

Schlüsselwörter:
  Raster

Benutzung:
  r.terraflow [-sq] elevation=name filled=name direction=name
    swatershed=name accumulation=name tci=name [d8cut=value]
    [memory=value] [STREAM_DIR=string] [stats=string] [--overwrite]
    [--verbose] [--quiet]

Flags:
   -s SFD (D8) flow (default is MFD)
   -q schweigsam
  --o Ausgabedateien dürfen bereits existierende Dateien überschreiben.
  --v Detaillierter Ausgabemodus
  --q Schweigsame Modulausgabe

Parameter:
      elevation Name der Höhenkarte.
         filled Output filled (flooded) elevation raster map
      direction Output flow direction raster map
     swatershed Output sink-watershed raster map
   accumulation Output flow accumulation raster map
            tci Output topographic convergence index (tci) raster map
          d8cut If flow accumulation is larger than this value it is
routed using SFD (D8) direction
                  (meaningfull only for MFD flow)
                   Standard: infinity
         memory Maximum runtime memory size (in MB)
                   Standard: 300
     STREAM_DIR Directory to hold temporary files (they can be large)
                   Standard: C:/Users/syringia/AppData/Local/Temp
          stats Name of file containing runtime statistics
                   Standard: stats.out
}}}

>
> (specifically the STREAM_DIR section)
>
>
> {{{
> /* temporary STREAM path */
> struct Option *streamdir;
> streamdir = G_define_option() ;
> streamdir->key = "STREAM_DIR";
> streamdir->type = TYPE_STRING;
> streamdir->required = NO;
> #ifdef __MINGW32__
> streamdir->answer =
G_convert_dirseps_from_host(G_store(getenv("TEMP")));
> #else
> streamdir->answer = G_store("/var/tmp/");
> #endif
> streamdir->description=
> _("Directory to hold temporary files (they can be large)");
> }}}
>
>
> Maris:
> > Aaah - just noticed - could this be related to #520?
>
> probably not as that should be already fixed. (just not ideal)
>
>
> can you try on a small (file size) raster like elevation.dem @
Spearfish? (existing 300mb doesn't seem so bad though)

nc-data set:

{{{
g.region -p -a rast=el_D782_6m@PERMANENT
projection: 99 (Lambert Conformal Conic)
zone: 0
datum: nad83
ellipsoid: a=6378137 es=0.006694380022900787
north: 222504.44509601
south: 219456.439
west: 633985.268
east: 637033.27409601
nsres: 6.09601219
ewres: 6.09601219
rows: 500
cols: 500
cells: 250000
}}}

>
>
> can you try with --verbose so percent done is shown for nrows and we get
more details?
>
>
> Hamish

{{{
g.gisenv set=DEBUG=3
}}}

{{{
r.terraflow --verbose elevation=el_D782_6m@PERMANENT filled=floodeda
direction=dirr swatershed=swat accumulation=accr tci=topgr
D2/3: G__read_Cell_head
D2/3: G__read_Cell_head_array
D3/3: region item: proj: 99
D3/3: region item: zone: 0
D3/3: region item: north: 222504.44509601
D3/3: region item: south: 219456.439
D3/3: region item: east: 637033.27409601
D3/3: region item: west: 633985.268
D3/3: region item: cols: 500
D3/3: region item: rows: 500
D3/3: region item: e-w resol: 6.09601219
D3/3: region item: n-s resol: 6.09601219
D3/3: region item: top: 1
D3/3: region item: bottom: 0
D3/3: region item: cols3: 500
D3/3: region item: rows3: 500
D3/3: region item: depths: 1
D3/3: region item: e-w resol3: 6.09601219
D3/3: region item: n-s resol3: 6.09601219
D3/3: region item: t-b resol: 1
D2/3: G__read_Cell_head
D2/3: G__read_Cell_head_array
D3/3: region item: proj: 99
D3/3: region item: zone: 0
D3/3: region item: north: 222504.44509601
D3/3: region item: south: 219456.439
D3/3: region item: east: 637033.27409601
D3/3: region item: west: 633985.268
D3/3: region item: cols: 500
D3/3: region item: rows: 500
D3/3: region item: e-w resol: 6.09601219
D3/3: region item: n-s resol: 6.09601219
D3/3: region item: format: -1
D3/3: region item: compressed: 1
cell el_D782_6m header compatible with region header
Elevation stored as FLOAT (4B)
Region size is 500 x 500
STREAM temporary files in
C:/Users/syringia/AppData/Local/Temp (THESE INTERMEDIATE
STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION
OF THE PROGRAM. TO SAVE SPACE PLEASE DELETE THESE FILES
MANUALLY!)
file stats.out exists - renaming.
reading data from el_D782_6m to stream
C:/Users/syringia/AppData/Local/Temp/STREAM_a05332: D2/3:
G__read_Cell_head
D2/3: G__read_Cell_head_array
D3/3: region item: proj: 99
D3/3: region item: zone: 0
D3/3: region item: north: 222504.44509601
D3/3: region item: south: 219456.439
D3/3: region item: east: 637033.27409601
D3/3: region item: west: 633985.268
D3/3: region item: cols: 500
D3/3: region item: rows: 500
D3/3: region item: e-w resol: 6.09601219
D3/3: region item: n-s resol: 6.09601219
D3/3: region item: format: -1
D3/3: region item: compressed: 1
D3/3: create window mapping (500 columns)
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 300.00M (314572800) bytes
Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
Assertion failed: nrows * ncols == str->stream_len(), file
grass2str.h, line 144
This application has requested the Runtime to terminate it
in an unusual way.
Please contact the application's support team for more
information.
}}}

the only maybe usefull information?

{{{
Assertion failed: nrows * ncols == str->stream_len(), file
grass2str.h, line 144
}}}

Helmut

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:4&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hamish):

{{{
STREAM temporary files in
C:/Users/syringia/AppData/Local/Temp (THESE INTERMEDIATE
STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION
OF THE PROGRAM. TO SAVE SPACE PLEASE DELETE THESE FILES
MANUALLY!)
file stats.out exists - renaming.
}}}

could you clean out the temp dir? in the original report you had:
{{{
stats.out: Permission denied
}}}

at that stage. does your user account have write permission in that dir?

this isn't related to the bug AFAIK, but I notice the D8CUT variable can't
seem to decide if it wants to be an integer, float, or double:

{{{
module output: D8CUT=999999986991104.000000
option.h: float d8cut; /* flow value where flow accu comp
switches to D8 */
main.cc: d8cut->type = TYPE_DOUBLE;
main.cc: opt->d8cut = atoi(d8cut->answer);
main.cc: sprintf(buf, "D8CUT=%f", opt->d8cut);
types.h:static const flowaccumulation_type MAX_ACCU = 1e+15;
}}}

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:5&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by marisn):

Replying to [comment:5 hamish]:
> could you clean out the temp dir? in the original report you had:
> {{{
> stats.out: Permission denied
> }}}
>
> at that stage. does your user account have write permission in that dir?

This isn't Slashdot where You don't have to read original report before
replying.

{{{
(Thu Mar 18 13:55:51 2010)
r.terraflow elevation=elevation.dem@PERMANENT filled=flo1 direction=dir1
swatershed=sink1 accumulation=accu1 tci=tci1
stats.out: Permission denied
raster elevation.dem is of type CELL_TYPE --you should use
r.terraflow.short
STREAM temporary files in C:/Users/Maris/AppData/Local/Temp
(THESE INTERMEDIATE STREAMS WILL NOT BE DELETED IN CASE OF
ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE
DELETE THESE FILES MANUALLY!)
(Thu Mar 18 13:55:51 2010) Command finished (0 sec)
(Thu Mar 18 13:55:58 2010)
r.terraflow --verbose elevation=elevation.dem@PERMANENT filled=flo1
direction=dir1 swatershed=sink1 accumulation=accu1 tci=tci1
cell elevation.dem header compatible with region header
stats.out: Permission denied
Elevation stored as FLOAT (4B)
raster elevation.dem is of type CELL_TYPE --you should use
r.terraflow.short
Region size is 466 x 633
STREAM temporary files in C:/Users/Maris/AppData/Local/Temp
(THESE INTERMEDIATE STREAMS WILL NOT BE DELETED IN CASE OF
ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE
DELETE THESE FILES MANUALLY!)
(Thu Mar 18 13:55:58 2010) Command finished (0 sec)
}}}

{{{
GRASS 6.4.0svn (Spearfish60)> cd C:/Users/Maris/AppData/Local/Temp

GRASS 6.4.0svn (Spearfish60)> pwd
/tmp

GRASS 6.4.0svn (Spearfish60)> echo "11111">stats.out

GRASS 6.4.0svn (Spearfish60)> ls
Comodo plugtmp tmp8dmank tmphq36zm.ppm tmpu1igd0.ppm
Low stats.out tmp8dmank.pgm tmpjwcn5a tmpulgzoq.ppm
Maris.bmp tmp38qqbr.ppm tmp8dmank.ppm tmpmvlbga tmpv1ri6t.ppm
STREAM_a03888 tmp6qna7p tmp_qcrpo.ppm tmpqnwipi tmpx9g0rx.ppm
WPDNSE tmp6qna7p.pgm tmpg45dva.ppm tmpqnwipi.pgm tmpyxrqeo.ppm
jusched.log tmp6qna7p.ppm tmpghekic.ppm tmpqnwipi.ppm ~nsu.tmp

GRASS 6.4.0svn (Spearfish60)> cat stats.out
"11111"
}}}

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1006#comment:6&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Changes (by hamish):

  * priority: normal => major

Comment:

{{{
Assertion failed: nrows * ncols == str->stream_len(), file
grass2str.h, line 144
}}}

Hi, I can reproduce this in Windows XP Prof. with the 6.4.0rc6 native
wingrass installer.

The temp dir option seems to be respected.

Interestingly the stats.out file ends up in C:\Program
Files\GRASS-6.4\msys

it contains:
{{{
Free Memory=-1
[0.0] Fri Mar 26 00:57:42 2010

Command Line:
r.terraflow --verbose elevation=elevation.dem@PERMANENT
filled=terra.flooded \
    direction=terra.dir swatershed=terra.sink accumulation=terra.accum \
    tci=terra.tci memory=600
input elevation grid: elevation.dem
output (flooded) elevations grid: terra.flooded
output directions grid: terra.dir
output sinkwatershed grid: terra.sink
output accumulation grid: terra.accum
output tci grid: terra.tci
[0.0] MFD flow direction
[0.0] D8CUT=999999986991104.000000
[0.0] Memory size: 600.00M (629145600) bytes
region size = 288.06K (294978) elts (466 rows x 633 cols)
internal nodata value: -9999
reading raster map elevation.demto stream
C:/DOCUME~1/HAMISH/LOCALS~1/Temp/STREAM_a02052
}}}

Note the '''Free Memory=-1''' at the top of the file ...

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:7&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hamish):

Replying to [comment:7 hamish]:
> Note the '''Free Memory=-1''' at the top of the file ...

that's nothing. freeMem() is hardcoded to that on Windows.

in r41535 I've added a new debug message to devbr6.5 to show the values
tested in that assert(). So we'll see what tomorrow's nightly build says
about it.

On linux for elevation.dem@spearfish I get:
{{{
D3/3: nrows=466 ncols=633 stream_len()=294978
}}}
as expected.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:8&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hellik):

Replying to [comment:8 hamish]:
> Replying to [comment:7 hamish]:
> > Note the '''Free Memory=-1''' at the top of the file ...
>
> that's nothing. freeMem() is hardcoded to that on Windows.
>
> in r41535 I've added a new debug message to devbr6.5 to show the values
tested in that assert(). So we'll see what tomorrow's nightly build says
about it.
>
> On linux for elevation.dem@spearfish I get:
> {{{
> D3/3: nrows=466 ncols=633 stream_len()=294978
> }}}
> as expected.
>
>
> Hamish

grass65, self compiled in osgeo4w-stack, WinVista32
r41535

{{{
g.gisenv set=DEBUG=3
}}}

{{{
g.region -p -a rast=elevation@PERMANENT
projection: 99 (Lambert Conformal Conic)
zone: 0
datum: nad83
ellipsoid: a=6378137 es=0.006694380022900787
north: 228500
south: 215000
west: 630000
east: 645000
nsres: 10
ewres: 10
rows: 1350
cols: 1500
cells: 2025000
}}}

{{{
r.terraflow --verbose elevation=elevation@PERMANENT filled=elevflood1
direction=elevdir1 swatershed=elevsinkw1 accumulation=elevacc1
tci=elevtci1 memory=900
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 900.00M (943718400) bytes
Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
D2/3: G__read_Cell_head
D2/3: G__read_Cell_head_array
D3/3: region item: proj: 99
D3/3: region item: zone: 0
D3/3: region item: north: 228500
D3/3: region item: south: 215000
D3/3: region item: east: 645000
D3/3: region item: west: 630000
D3/3: region item: cols: 1500
D3/3: region item: rows: 1350
D3/3: region item: e-w resol: 10
D3/3: region item: n-s resol: 10
D3/3: region item: top: 1
D3/3: region item: bottom: 0
D3/3: region item: cols3: 1500
D3/3: region item: rows3: 1350
D3/3: region item: depths: 1
D3/3: region item: e-w resol3: 10
D3/3: region item: n-s resol3: 10
D3/3: region item: t-b resol: 1
D2/3: G__read_Cell_head
D2/3: G__read_Cell_head_array
D3/3: region item: proj: 99
D3/3: region item: zone: 0
D3/3: region item: north: 228500
D3/3: region item: south: 215000
D3/3: region item: east: 645000
D3/3: region item: west: 630000
D3/3: region item: cols: 1500
D3/3: region item: rows: 1350
D3/3: region item: e-w resol: 10
D3/3: region item: n-s resol: 10
D3/3: region item: format: -1
D3/3: region item: compressed: 1
cell elevation header compatible with region header
Elevation stored as FLOAT (4B)
Region size is 1350 x 1500
STREAM temporary files in
C:/Users/syringia/AppData/Local/Temp (THESE INTERMEDIATE
STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION
OF THE PROGRAM. TO SAVE SPACE PLEASE DELETE THESE FILES
MANUALLY!)
reading data from elevation to stream
C:/Users/syringia/AppData/Local/Temp/STREAM_a06112: D2/3:
G__read_Cell_head
D2/3: G__read_Cell_head_array
D3/3: region item: proj: 99
D3/3: region item: zone: 0
D3/3: region item: north: 228500
D3/3: region item: south: 215000
D3/3: region item: east: 645000
D3/3: region item: west: 630000
D3/3: region item: cols: 1500
D3/3: region item: rows: 1350
D3/3: region item: e-w resol: 10
D3/3: region item: n-s resol: 10
D3/3: region item: format: -1
D3/3: region item: compressed: 1
D3/3: create window mapping (1500 columns)
D3/3: nrows=1350 ncols=1500 stream_len()=0
'''Assertion failed: nrows * ncols == str->stream_len(), file
grass2str.h, line 146'''
This application has requested the Runtime to terminate it
in an unusual way.
Please contact the application's support team for more
information.
}}}

{{{
stats.out:
Free Memory=-1
[0.0] Thu Mar 25 20:26:58 2010

Command Line:
r.terraflow --verbose elevation=elevation@PERMANENT filled=elevflood1
direction=elevdir1 swatershed=elevsinkw1 accumulation=elevacc1
tci=elevtci1 memory=900
input elevation grid: elevation
output (flooded) elevations grid: elevflood1
output directions grid: elevdir1
output sinkwatershed grid: elevsinkw1
output accumulation grid: elevacc1
output tci grid: elevtci1
[0.0] MFD flow direction
[0.0] D8CUT=999999986991104.000000
[0.0] Memory size: 900.00M (943718400) bytes
region size = 1.93M (2025000) elts (1350 rows x 1500 cols)
internal nodata value: -9999
reading raster map elevationto stream
C:/Users/syringia/AppData/Local/Temp/STREAM_a06112
}}}

Helmut

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:9&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hellik):

Replying to [comment:9 hellik]:
> Replying to [comment:8 hamish]:
> > Replying to [comment:7 hamish]:
> > > Note the '''Free Memory=-1''' at the top of the file ...
> >
> > that's nothing. freeMem() is hardcoded to that on Windows.
> >
> > in r41535 I've added a new debug message to devbr6.5 to show the
values tested in that assert(). So we'll see what tomorrow's nightly build
says about it.
> >
> > On linux for elevation.dem@spearfish I get:
> > {{{
> > D3/3: nrows=466 ncols=633 stream_len()=294978
> > }}}
> > as expected.
> >
> >
> > Hamish
>
> grass65, self compiled in osgeo4w-stack, WinVista32
> r41535
>
> {{{
> g.gisenv set=DEBUG=3
> }}}
>

now it's

Assertion failed: nrows * ncols == str->stream_len(), file
grass2str.h, '''line 146'''

Helmut

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:10&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hamish):

Hamish:
> in r41535 I've added a new debug message to devbr6.5 to show
> the values tested in that assert().

Replying to [comment:10 hellik]:
> now it's
>
> Assertion failed: nrows * ncols == str->stream_len(), file
> grass2str.h, '''line 146'''

yes, because I added two lines to the source code so now it fails two
lines later.

turn on debug messages by running `g.gisenv set="DEBUG=3"`.
(set that back to 0 to turn them off again once you are done)

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:11&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hamish):

Replying to [comment:9 hellik]:
> D3/3: nrows=1350 ncols=1500 stream_len()=0
> [[BR]]
> '''Assertion failed: nrows * ncols == str->stream_len(), file
> grass2str.h, line 146'''

(assert() complains because the two sides are not the same)

ok, so str->stream_len() is not getting set correctly.

I wonder if this works:
{{{
   /* create output stream */
   AMI_STREAM<T>* str = new AMI_STREAM<T>();
}}}

stream_len() is defined in include/iostream/ami_stream.h
(number of items in each stream)

more debug messages now turned on in that r41548 (6.5svn) ...

hmmm I get "Free Memory=-1" in stats.out on 32bit linux too.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:12&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hellik):

Replying to [comment:12 hamish]:
> [...]
> more debug messages now turned on in that r41548 (6.5svn) ...

r41567, self compiled in the osgeo4w/

{{{
g.gisenv set=DEBUG=3
}}}

{{{
r.terraflow --verbose elevation=elevation@PERMANENT filled=elevflood
direction=elevdir swatershed=elevsinkwater accumulation=elevflowacc
tci=elevtci memory=900
D2/3: G__read_Cell_head
D2/3: G__read_Cell_head_array
D3/3: region item: proj: 99
D3/3: region item: zone: 0
D3/3: region item: north: 228500
D3/3: region item: south: 215000
D3/3: region item: east: 645000
D3/3: region item: west: 630000
D3/3: region item: cols: 1500
D3/3: region item: rows: 1350
D3/3: region item: e-w resol: 10
D3/3: region item: n-s resol: 10
D3/3: region item: top: 1
D3/3: region item: bottom: 0
D3/3: region item: cols3: 1500
D3/3: region item: rows3: 1350
D3/3: region item: depths: 1
D3/3: region item: e-w resol3: 10
D3/3: region item: n-s resol3: 10
D3/3: region item: t-b resol: 1
D2/3: G__read_Cell_head
D2/3: G__read_Cell_head_array
D3/3: region item: proj: 99
D3/3: region item: zone: 0
D3/3: region item: north: 228500
D3/3: region item: south: 215000
D3/3: region item: east: 645000
D3/3: region item: west: 630000
D3/3: region item: cols: 1500
D3/3: region item: rows: 1350
D3/3: region item: e-w resol: 10
D3/3: region item: n-s resol: 10
D3/3: region item: format: -1
D3/3: region item: compressed: 1
cell elevation header compatible with region header
Elevation stored as FLOAT (4B)
Region size is 1350 x 1500
STREAM temporary files in
C:/Users/syringia/AppData/Local/Temp (THESE INTERMEDIATE
STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION
OF THE PROGRAM. TO SAVE SPACE PLEASE DELETE THESE FILES
MANUALLY!)
Reading data from <elevation> to stream
<C:/Users/syringia/AppData/Local/Temp/STREAM_a07132>
D2/3: G__read_Cell_head
D2/3: G__read_Cell_head_array
D3/3: region item: proj: 99
D3/3: region item: zone: 0
D3/3: region item: north: 228500
D3/3: region item: south: 215000
D3/3: region item: east: 645000
D3/3: region item: west: 630000
D3/3: region item: cols: 1500
D3/3: region item: rows: 1350
D3/3: region item: e-w resol: 10
D3/3: region item: n-s resol: 10
D3/3: region item: format: -1
D3/3: region item: compressed: 1
D3/3: create window mapping (1500 columns)
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 900.00M (943718400) bytes
Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
C:/Users/syringia/AppData/Local/Temp/STREAM_a07132: length =
17179869184 sizeof(T)=1
D3/3: nrows=1350 ncols=1500 stream_len()=0
C:/Users/syringia/AppData/Local/Temp/STREAM_a07132: length =
17179869184 sizeof(T)=0
Assertion failed: nrows * ncols == str->stream_len(), file
grass2str.h, line 146
This application has requested the Runtime to terminate it
in an unusual way.
Please contact the application's support team for more
information.
}}}

{{{
Free Memory=-1
[0.0] Sat Mar 27 03:38:21 2010

Command Line:
r.terraflow --verbose elevation=elevation@PERMANENT filled=elevflood
direction=elevdir swatershed=elevsinkwater accumulation=elevflowacc
tci=elevtci memory=900
input elevation grid: elevation
output (flooded) elevations grid: elevflood
output directions grid: elevdir
output sinkwatershed grid: elevsinkwater
output accumulation grid: elevflowacc
output tci grid: elevtci
[0.0] MFD flow direction
[0.0] D8CUT=999999986991104.000000
[0.0] Memory size: 900.00M (943718400) bytes
region size = 1.93M (2025000) elts (1350 rows x 1500 cols)
internal nodata value: -9999
Reading raster map <elevation> to stream
<C:/Users/syringia/AppData/Local/Temp/STREAM_a07132>.
}}}

{{{
g.gisenv set=DEBUG=0
}}}

{{{
r.terraflow --verbose elevation=elevation@PERMANENT filled=elevflood
direction=elevdir swatershed=elevsinkwater accumulation=elevflowacc
tci=elevtci memory=900
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 900.00M (943718400) bytes
Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
cell elevation header compatible with region header
Elevation stored as FLOAT (4B)
Region size is 1350 x 1500
STREAM temporary files in
C:/Users/syringia/AppData/Local/Temp (THESE INTERMEDIATE
STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION
OF THE PROGRAM. TO SAVE SPACE PLEASE DELETE THESE FILES
MANUALLY!)
Reading data from <elevation> to stream
<C:/Users/syringia/AppData/Local/Temp/STREAM_a01828>
C:/Users/syringia/AppData/Local/Temp/STREAM_a01828: length =
17179869184 sizeof(T)=1
C:/Users/syringia/AppData/Local/Temp/STREAM_a01828: length =
17179869184 sizeof(T)=1
Assertion failed: nrows * ncols == str->stream_len(), file
grass2str.h, line 146
This application has requested the Runtime to terminate it
in an unusual way.
Please contact the application's support team for more
information.
}}}

{{{
Free Memory=-1
[0.0] Sat Mar 27 03:40:39 2010

Command Line:
r.terraflow --verbose elevation=elevation@PERMANENT filled=elevflood
direction=elevdir swatershed=elevsinkwater accumulation=elevflowacc
tci=elevtci memory=900
input elevation grid: elevation
output (flooded) elevations grid: elevflood
output directions grid: elevdir
output sinkwatershed grid: elevsinkwater
output accumulation grid: elevflowacc
output tci grid: elevtci
[0.0] MFD flow direction
[0.0] D8CUT=999999986991104.000000
[0.0] Memory size: 900.00M (943718400) bytes
region size = 1.93M (2025000) elts (1350 rows x 1500 cols)
internal nodata value: -9999
Reading raster map <elevation> to stream
<C:/Users/syringia/AppData/Local/Temp/STREAM_a01828>.
}}}

best regards
Helmut

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:13&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hamish):

are you using the elevation raster map from the NC dataset?

Helmut wrote:
{{{
Reading data from <elevation> to stream
<C:/Users/syringia/AppData/Local/Temp/STREAM_a01828>
C:/Users/syringia/AppData/Local/Temp/STREAM_a01828: length =
17179869184 sizeof(T)=1
C:/Users/syringia/AppData/Local/Temp/STREAM_a01828: length =
17179869184 sizeof(T)=1
Assertion failed: nrows * ncols == str->stream_len(), file
grass2str.h, line 146
}}}

so the value of the buf.st_size variable is filled with garbage.
and sizeof(T) is different from what I get on linux.

for me on 32bit linux I get this output for that line (spearfish's
elevation.dem)

{{{
evelation.dem:
rows: 466
cols: 633
...
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 300.00M (314572800) bytes
Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
Reading data from <elevation.dem> to stream </var/tmp//STREAM_qylbyz>
  100%

/var/tmp//STREAM_qylbyz: length = 1179912 sizeof(T)=4
/var/tmp//STREAM_qylbyz: length = 1179912 sizeof(T)=4

total elements=294978, nodata elements=2661
largest temporary files:
FILL: 15.75M (16518768) [294978 elements, 56B each]
FLOW: 22.30M (23385360) [292317 elements, 80B each]
Will need at least 44.60M (46770720) space available in /var/tmp/
------------------------------
COMPUTING FLOW DIRECTIONS
...
}}}

is stat() on Windows the same as it is on UNIX?

on my version of linux .st_size should be type off_t: "total size, in
bytes", and stat() should return 0 on success and -1 on failure.

is `%lld` the right printf() setting for off_t?

I'm not sure how big <class T> should be, but probably bigger than 1
byte.. ?

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:14&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hellik):

Replying to [comment:14 hamish]:
> are you using the elevation raster map from the NC dataset?

yes, it is the NC-dataset.

Helmut

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:15&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hellik):

Replying to [comment:14 hamish]:

> for me on 32bit linux I get this output for that line (spearfish's
elevation.dem)
>
  {{{
  evelation.dem:
  rows: 466
  cols: 633
  ...
  MFD flow direction
  D8CUT=999999986991104.000000
  Memory size: 300.00M (314572800) bytes
  Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
  Reading data from <elevation.dem> to stream </var/tmp//STREAM_qylbyz>
   100%

  /var/tmp//STREAM_qylbyz: length = 1179912 sizeof(T)=4
  /var/tmp//STREAM_qylbyz: length = 1179912 sizeof(T)=4

  total elements=294978, nodata elements=2661
  largest temporary files:
  FILL: 15.75M (16518768) [294978 elements, 56B each]
  FLOW: 22.30M (23385360) [292317 elements, 80B each]
  Will need at least 44.60M (46770720) space available in /var/tmp/
  ------------------------------
  COMPUTING FLOW DIRECTIONS
  ...
}}}

and with spearfish's elevation.dem on WinVista32

{{{
r.terraflow --verbose elevation=elevation.dem@PERMANENT
filled=floodedelevdem direction=flowdirrast swatershed=sinkwatershrast
accumulation=flowaccrast tci=topoconvrast memory=900
...
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 900.00M (943718400) bytes
Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
C:/Users/syringia/AppData/Local/Temp/STREAM_a03800: length =
17179869184 sizeof(T)=1
D3/3: nrows=466 ncols=633 stream_len()=0
C:/Users/syringia/AppData/Local/Temp/STREAM_a03800: length =
17179869184 sizeof(T)=0
Assertion failed: nrows * ncols == str->stream_len(), file
grass2str.h, line 146
...
}}}

Helmut

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:16&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hamish):

could you try r.in.bin? it also uses stat()'s .st_size.

(I notice it uses int,%d for FILE_SIZE too, so will not be LFS-safe)

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:17&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Comment (by hellik):

Replying to [comment:17 hamish]:
> could you try r.in.bin? it also uses stat()'s .st_size.

I'm not at my laptop at the moment, so I've tried WinGRASS-6.5.SVN-
r41581-1-Setup.exe on a WinVista32-box with the nc-dataset:

{{{
r.out.bin -h --verbose input=elevation@PERMANENT
output=J:\wd\grassoutbin\elevroutbin3.bin
}}}

{{{
r.in.bin -h input=J:\wd\grassoutbin\elevroutbin3.bin output=elevrinbin
bytes=4
D2/3: G__read_Cell_head
D2/3: G__read_Cell_head_array
D3/3: region item: proj: 99
D3/3: region item: zone: 0
D3/3: region item: north: 228500
D3/3: region item: south: 215000
D3/3: region item: east: 645000
D3/3: region item: west: 630000
D3/3: region item: cols: 1500
D3/3: region item: rows: 1350
D3/3: region item: e-w resol: 10
D3/3: region item: n-s resol: 10
D3/3: region item: top: 1
D3/3: region item: bottom: 0
D3/3: region item: cols3: 1500
D3/3: region item: rows3: 1350
D3/3: region item: depths: 1
D3/3: region item: e-w resol3: 10
D3/3: region item: n-s resol3: 10
D3/3: region item: t-b resol: 1
D1/3: Creating support files for elevrinbin
D1/3: close elevrinbin compressed
D3/3: G_recreate_command()
}}}

import with r.in.bin is working

{{{
+----------------------------------------------------------------------------+
  | Layer: elevrinbin@user1 Date: Sun Mar 28 20:14:13 2010
|
  | Mapset: user1 Login of Creator: normal
|
  | Location: nc_spm_07
|
  | DataBase: J:\gisdata
|
  | Title: ( elevrinbin )
|
  | Timestamp: none
|
|----------------------------------------------------------------------------|
  |
|
  | Type of Map: raster Number of Categories: 1125930098
|
  | Data Type: CELL
|
  | Rows: 1350
|
  | Columns: 1500
|
  | Total Cells: 2025000
|
  | Projection: Lambert Conformal Conic
|
  | N: 228500 S: 215000 Res: 10
|
  | E: 645000 W: 630000 Res: 10
|
  | Range of data: min = 1113477295 max = 1125930098
|
  |
|
  | Data Description:
|
  | generated by r.in.bin
|
  |
|
  | Comments:
|
  | r.in.bin -h input="J:\wd\grassoutbin\elevroutbin3.bin"
output="elevr\ |
  | inbin" bytes=4
|
  |
|
+----------------------------------------------------------------------------+
}}}

Helmut

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:18&gt;
GRASS GIS <http://grass.osgeo.org>

#1006: r.terraflow fails to create temp file on Windows when launched from wxgui
------------------------------+---------------------------------------------
  Reporter: marisn | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: critical | Milestone: 6.4.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: wxgui, wingrass, terraflow
  Platform: MSWindows Vista | Cpu: x86-32
------------------------------+---------------------------------------------
Changes (by hamish):

  * priority: major => critical

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1006#comment:19&gt;
GRASS GIS <http://grass.osgeo.org>