[GRASS-dev] [GRASS GIS] #1903: r.watershed fails with "Subprocess failed with exit code 6"

#1903: r.watershed fails with "Subprocess failed with exit code 6"
-------------------------+--------------------------------------------------
Reporter: marcosc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 6.4.3
Component: Default | Version:
Keywords: | Platform: Linux
      Cpu: Unspecified |
-------------------------+--------------------------------------------------
Some Information about the raster:

rows: 38728
cols: 55611
cells: 2153702808

r.watershed was used with the flags -f and -m.

Using openSuse as Platform.

The error-messages, which are repeated very often:

WARNING: segment_pagein: read EOF[[BR]]
WARNING: seg_put(): could not write segment file[[BR]]
&[[BR]]
WARNING: segment_pagein: read EOF[[BR]]
WARNING: bseg_put(): could not read segment file[[BR]]
&[[BR]]
WARNING: segment_seek: Invalid argument[[BR]]
&[[BR]]
WARNING: Subprocess failed with exit code 6[[BR]]
WARNING: category information for [flow_accumulation] in[[BR]]
[r.watershed][[BR]]
          missing or invalid[[BR]]
WARNING: category information for [drainage_direction] in[[BR]]
[r.watershed][[BR]]
          missing or invalid[[BR]]
WARNING: category information for [basins] in [r.watershed] missing
or[[BR]]
          invalid[[BR]]
WARNING: category information for [streams] in [r.watershed][[BR]] missing
or[[BR]]
          invalid[[BR]]

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1903&gt;
GRASS GIS <http://grass.osgeo.org>

#1903: r.watershed fails with "Subprocess failed with exit code 6"
-------------------------+--------------------------------------------------
Reporter: marcosc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 6.4.3
Component: Raster | Version: svn-releasebranch64
Keywords: r.watershed | Platform: Linux
      Cpu: Unspecified |
-------------------------+--------------------------------------------------
Changes (by neteler):

  * keywords: => r.watershed
  * version: => svn-releasebranch64
  * component: Default => Raster

Comment:

You went over the file limit of 32bit (is your installation LFS = large
file enabled?)

{{{
Your size: 2153702808
      2^31: 2147483648
}}}

See also
  * http://grasswiki.osgeo.org/wiki/Large_File_Support
  * http://grasswiki.osgeo.org/wiki/Large_raster_data_processing

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1903#comment:1&gt;
GRASS GIS <http://grass.osgeo.org>

#1903: r.watershed fails with "Subprocess failed with exit code 6"
-------------------------+--------------------------------------------------
Reporter: marcosc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 6.4.3
Component: Raster | Version: svn-releasebranch64
Keywords: r.watershed | Platform: Linux
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by marcosc):

LFS is enabled, but we are working on a 64-bit platform anyway.

The command:
{{{
r.watershed -fm elevation=... accumulation=... drainage=... basin=basins
stream=... threshold=10000 memory=25000 --overwrite --verbose
  }}}
... with:
{{{
g.region -p

rows: 38728
cols: 55611
cells: 2153702808
  }}}
...results in:
  {{{
ERROR: G_malloc: unable to allocate 596992000 bytes at setup.c:64
WARNING: Subprocess failed with exit code 256
WARNING: category information for [flow_accumulation] in [r.watershed]
          missing or invalid
WARNING: category information for [drainage_direction] in [r.watershed]
          missing or invalid
WARNING: category information for [basins] in [r.watershed] missing or
          invalid
WARNING: category information for [streams] in [r.watershed] missing or
          invalid
Removing raster <MASK>
[H[2JClosing monitors ...
Cleaning up temporary files ...
Done.
  }}}

The system runs with 32 gb of RAM, so ~600 MB shouldn't be a problem.

We checked the setup.c in /lib/segment/setup.c but could not locate the
problem.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1903#comment:2&gt;
GRASS GIS <http://grass.osgeo.org>

#1903: r.watershed fails with "Subprocess failed with exit code 6"
-------------------------+--------------------------------------------------
Reporter: marcosc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 6.4.3
Component: Raster | Version: svn-releasebranch64
Keywords: r.watershed | Platform: Linux
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by mmetz):

Replying to [comment:2 marcosc]:
> LFS is enabled, but we are working on a 64-bit platform anyway.
>
> The command:
{{{
  r.watershed -fm elevation=... accumulation=... drainage=... basin=basins
stream=... threshold=10000 memory=25000 --overwrite --verbose
}}}
> ... with:
{{{
  g.region -p

  rows: 38728
  cols: 55611
  cells: 2153702808
}}}
>
The number of cells can cause integer overflow because the largest 32bit
integer is 2147483647 < 2153702808. This limit has been removed in GRASS
7.

Markus M

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1903#comment:3&gt;
GRASS GIS <http://grass.osgeo.org>

#1903: r.watershed fails with "Subprocess failed with exit code 6"
-------------------------+--------------------------------------------------
Reporter: marcosc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 6.4.3
Component: Raster | Version: svn-releasebranch64
Keywords: r.watershed | Platform: Linux
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by neteler):

Replying to [comment:2 marcosc]:
> LFS is enabled, but we are working on a 64-bit platform anyway.

Please post here the output of

uname -a

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1903#comment:4&gt;
GRASS GIS <http://grass.osgeo.org>

#1903: r.watershed fails with "Subprocess failed with exit code 6"
-------------------------+--------------------------------------------------
Reporter: marcosc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 6.4.3
Component: Raster | Version: svn-releasebranch64
Keywords: r.watershed | Platform: Linux
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by marcosc):

{{{
uname -a
Linux voss-mod02 2.6.32.59-0.7-xen #1 SMP 2012-07-13 15:50:56 +0200 x86_64
x86_64 x86_64 GNU/Linux
}}}

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1903#comment:5&gt;
GRASS GIS <http://grass.osgeo.org>

#1903: r.watershed fails with "Subprocess failed with exit code 6"
-------------------------+--------------------------------------------------
Reporter: marcosc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 6.4.3
Component: Raster | Version: svn-releasebranch64
Keywords: r.watershed | Platform: Linux
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by glynn):

Replying to [comment:2 marcosc]:
> LFS is enabled, but we are working on a 64-bit platform anyway.

That doesn't necessarily help. There are lots of places where sizes are
stored or calculated using "int", which is still only 32-bit on most
64-bit platforms. Many such issues have been fixed in 7.0, but I wouldn't
assume that we have found all of them.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1903#comment:6&gt;
GRASS GIS <http://grass.osgeo.org>

#1903: r.watershed fails with "Subprocess failed with exit code 6"
------------------------------+---------------------------------------------
Reporter: marcosc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 6.4.3
Component: Raster | Version: svn-releasebranch64
Keywords: r.watershed, LFS | Platform: Linux
      Cpu: Unspecified |
------------------------------+---------------------------------------------
Changes (by neteler):

  * keywords: r.watershed => r.watershed, LFS

Comment:

see also ticket #2036

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1903#comment:7&gt;
GRASS GIS <http://grass.osgeo.org>