Hi folks,
I have a quick troubleshooting question. I’m trying to run r.watershed to extract stream segments and I’m receiving the following error;
GRASS 6.4.0RC5+39438 (ccarleton.minanha.utm):~ > r.watershed elevation=aster_srtm_dem_filled@CloudRemoval threshold=10 stream=aster_srtm_stream memory=300
SECTION 1a (of 5): Initiating Memory.
SECTION 1b (of 5): Determining Offmap Flow.
100%
SECTION 2: A * Search.
100%
SECTION 3: Accumulating Surface Flow.
100%
SECTION 4: Watershed determination.
Segmentation fault
WARNING: Subprocess failed with exit code 35584
WARNING: category information for [aster_srtm_stream] in [geophys] missing or invalid
I haven’t been able to find something in the mailing lists about this, but if you know of somewhere that I can find a solution I’d appreciate the link. I’m running GRASS 6.4 RC5 on Ubuntu LTS 10.04 with a Dell Precision T7500 Workstation. The region settings are as follows;
projection: 1 (UTM)
zone: 16
datum: wgs84
ellipsoid: wgs84
north: 1942292.77488498
south: 1798172.56898554
west: 174812.49733566
east: 382055.4579663
nsres: 15.00002143
ewres: 15.00021429
rows: 9608
cols: 13816
cells: 132744128
The actual DEM is much smaller than this (a composite ASTER/SRTM DEM) and I have 12GB of memory so I’d be surprised if this was a memory problem. Before I start working with GDB or other in-depth debugging I was hoping someone would have a short answer for this already. Thanks for your time,
Chris
Hi,
2010/8/23 Chris Carleton <w_chris_carleton@hotmail.com>:
I haven't been able to find something in the mailing lists about this, but
if you know of somewhere that I can find a solution I'd appreciate the link.
I'm running GRASS 6.4 RC5 on Ubuntu LTS 10.04 with a Dell Precision T7500
Workstation. The region settings are as follows;
RC5 is quite old - please try RC6 or better fresh SVN.
Martin
--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa
Martin Landa:
Chris Carleton:
I haven't been able to find something in the mailing lists about this, but
if you know of somewhere that I can find a solution I'd appreciate the link.
I'm running GRASS 6.4 RC5 on Ubuntu LTS 10.04 with a Dell Precision T7500
Workstation. The region settings are as follows;
RC5 is quite old - please try RC6 or better fresh SVN.
The Ubuntu version and grass version used are both true 64bit?
Processing over 130 million cells with r.watershed in memory and 12GB
is possible, but only if grass is compiled as a 64bit application.
SECTION 4: Watershed determination is pretty much unchanged since RC5,
the segfault might also happen in current 6.4.
The combination of threshold=10 and over 130 million cells is a bit
unusual, producing a very large number of stream segments, but AFAICT
not large enough to trigger a segfault.
Markus M
Chris Carleton wrote:
Okay thanks. I was using a 'trial-and-error' method for determining the
threshold, but I think I ought to start large and proceed toward smaller
values. I haven't come across any analytical method for determining the best
threshold value for stream extraction.
There are conceptually different approaches to determine a reasonable
threshold. One would be to use the DEM only and estimate what
threshold provides a reasonable stream network. Obviously you can get
vastly different stream networks from a 1m LiDAR DEM and a 90m SRTM
DEM. Such an approach gives an answer to what is possible with the
DEM, which does not necessarily match reality (most of the time I
guess it does not match reality). A different approach would be to
look at rainfall patterns, soil infiltration rates etc, and decide
whether you are interested in ephemeral, intermediate or permanent
streams. For permanent streams, a decision must be made whether you
want to get all, also very small permanent streams, or only major
streams.
Markus M
Some smart person needs to develop a
statistically sound method for assessing the uncertainty associated with
geophysical feature extraction so that multiple results can be compared
meaningfully and we're not left to assess the validity of a result on the
basis of whether or not it 'looks okay' and preferably that doesn't involve
going into the field to count streams. I'm going to compile GRASS from
source with the latest stable release and configure for 64bit support. I
think the current install was done through the repository so it's behind and
not optimized. On that note, I'm trying to figure out what the appropriate
configure flag(s) is(are) for HDF4 and HDF5 support in GDAL so that I can
open HDF files in GRASS. The only like that I can find that seems to be
discussing it formally is here:
http://trac.osgeo.org/gdal/wiki/HDF
Does anyone have the answer handy or another link with instructions that are
more clear? Thanks,
Chris
On 24 August 2010 05:39, Markus Metz <markus.metz.giswork@googlemail.com>
wrote:
Martin Landa:
>
> Chris Carleton:
>> I haven't been able to find something in the mailing lists about this,
>> but
>> if you know of somewhere that I can find a solution I'd appreciate the
>> link.
>> I'm running GRASS 6.4 RC5 on Ubuntu LTS 10.04 with a Dell Precision
>> T7500
>> Workstation. The region settings are as follows;
>
> RC5 is quite old - please try RC6 or better fresh SVN.
>
The Ubuntu version and grass version used are both true 64bit?
Processing over 130 million cells with r.watershed in memory and 12GB
is possible, but only if grass is compiled as a 64bit application.
SECTION 4: Watershed determination is pretty much unchanged since RC5,
the segfault might also happen in current 6.4.
The combination of threshold=10 and over 130 million cells is a bit
unusual, producing a very large number of stream segments, but AFAICT
not large enough to trigger a segfault.
Markus M
Chris Carleton wrote:
As for the r.watershed problem, it's occurring
again and I'm now using GRASS 6.4 RC6 with 64bit support.
Apparently you are comfortable to build from source. No need to use
outdated RC6, use the latest weekly snapshot instead, available at [1]
Here are the region settings I'm using;
zone: 16
datum: wgs84
ellipsoid: wgs84
north: 1942292.77488498
south: 1798172.56898554
west: 174812.49733566
east: 382055.4579663
nsres: 15.00002143
ewres: 15.00021429
rows: 9608
cols: 13816
cells: 132744128
And here is the output in the terminal from strace and r.watershed;
GRASS 6.4.0RC6 (ccarleton.minanha.utm):~/ > strace r.watershed
elevation=aster_srtm_dem_filled@CloudRemoval threshold=100
stream=aster_srtm_streams
...
SECTION 1b (of 5): Determining Offmap Flow.
100%
SECTION 2: A * Search.
100%
SECTION 3: Accumulating Surface Flow.
100%
SECTION 4: Watershed determination.
Segmentation fault
I will test it myself next week, currently on the road.
Markus M
[1] http://grass.osgeo.org/grass64/source/snapshot/
I've attached the whole strace stdout to this email. Again, I'm using Ubuntu
10.04 LTS 64bit. Does anyone have any ideas about where to start
troubleshooting this? Thanks,
Chris
On 24 August 2010 09:55, Chris Carleton <w.ccarleton@gmail.com> wrote:
Okay thanks. I was using a 'trial-and-error' method for determining the
threshold, but I think I ought to start large and proceed toward smaller
values. I haven't come across any analytical method for determining the best
threshold value for stream extraction. Some smart person needs to develop a
statistically sound method for assessing the uncertainty associated with
geophysical feature extraction so that multiple results can be compared
meaningfully and we're not left to assess the validity of a result on the
basis of whether or not it 'looks okay' and preferably that doesn't involve
going into the field to count streams. I'm going to compile GRASS from
source with the latest stable release and configure for 64bit support. I
think the current install was done through the repository so it's behind and
not optimized. On that note, I'm trying to figure out what the appropriate
configure flag(s) is(are) for HDF4 and HDF5 support in GDAL so that I can
open HDF files in GRASS. The only like that I can find that seems to be
discussing it formally is here:
http://trac.osgeo.org/gdal/wiki/HDF
Does anyone have the answer handy or another link with instructions that
are more clear? Thanks,
Chris
On 24 August 2010 05:39, Markus Metz <markus.metz.giswork@googlemail.com>
wrote:
Martin Landa:
>
> Chris Carleton:
>> I haven't been able to find something in the mailing lists about this,
>> but
>> if you know of somewhere that I can find a solution I'd appreciate the
>> link.
>> I'm running GRASS 6.4 RC5 on Ubuntu LTS 10.04 with a Dell Precision
>> T7500
>> Workstation. The region settings are as follows;
>
> RC5 is quite old - please try RC6 or better fresh SVN.
>
The Ubuntu version and grass version used are both true 64bit?
Processing over 130 million cells with r.watershed in memory and 12GB
is possible, but only if grass is compiled as a 64bit application.
SECTION 4: Watershed determination is pretty much unchanged since RC5,
the segfault might also happen in current 6.4.
The combination of threshold=10 and over 130 million cells is a bit
unusual, producing a very large number of stream segments, but AFAICT
not large enough to trigger a segfault.
Markus M
Tested with a 144 million cells DEM and threshold=10, r.watershed
completes successfully here. Can you try to debug with gdb or
valgrind? See [1] for some info on GRASS debugging.
Markus M
[1] http://grass.osgeo.org/wiki/GRASS_Debugging
Markus Metz wrote:
Chris Carleton wrote:
As for the r.watershed problem, it's occurring
again and I'm now using GRASS 6.4 RC6 with 64bit support.
Apparently you are comfortable to build from source. No need to use
outdated RC6, use the latest weekly snapshot instead, available at [1]
Here are the region settings I'm using;
zone: 16
datum: wgs84
ellipsoid: wgs84
north: 1942292.77488498
south: 1798172.56898554
west: 174812.49733566
east: 382055.4579663
nsres: 15.00002143
ewres: 15.00021429
rows: 9608
cols: 13816
cells: 132744128
And here is the output in the terminal from strace and r.watershed;
GRASS 6.4.0RC6 (ccarleton.minanha.utm):~/ > strace r.watershed
elevation=aster_srtm_dem_filled@CloudRemoval threshold=100
stream=aster_srtm_streams
...
SECTION 1b (of 5): Determining Offmap Flow.
100%
SECTION 2: A * Search.
100%
SECTION 3: Accumulating Surface Flow.
100%
SECTION 4: Watershed determination.
Segmentation fault
I will test it myself next week, currently on the road.
Markus M
[1] http://grass.osgeo.org/grass64/source/snapshot/
I've attached the whole strace stdout to this email. Again, I'm using Ubuntu
10.04 LTS 64bit. Does anyone have any ideas about where to start
troubleshooting this? Thanks,
Chris
On 24 August 2010 09:55, Chris Carleton <w.ccarleton@gmail.com> wrote:
Okay thanks. I was using a 'trial-and-error' method for determining the
threshold, but I think I ought to start large and proceed toward smaller
values. I haven't come across any analytical method for determining the best
threshold value for stream extraction. Some smart person needs to develop a
statistically sound method for assessing the uncertainty associated with
geophysical feature extraction so that multiple results can be compared
meaningfully and we're not left to assess the validity of a result on the
basis of whether or not it 'looks okay' and preferably that doesn't involve
going into the field to count streams. I'm going to compile GRASS from
source with the latest stable release and configure for 64bit support. I
think the current install was done through the repository so it's behind and
not optimized. On that note, I'm trying to figure out what the appropriate
configure flag(s) is(are) for HDF4 and HDF5 support in GDAL so that I can
open HDF files in GRASS. The only like that I can find that seems to be
discussing it formally is here:
http://trac.osgeo.org/gdal/wiki/HDF
Does anyone have the answer handy or another link with instructions that
are more clear? Thanks,
Chris
On 24 August 2010 05:39, Markus Metz <markus.metz.giswork@googlemail.com>
wrote:
Martin Landa:
>
> Chris Carleton:
>> I haven't been able to find something in the mailing lists about this,
>> but
>> if you know of somewhere that I can find a solution I'd appreciate the
>> link.
>> I'm running GRASS 6.4 RC5 on Ubuntu LTS 10.04 with a Dell Precision
>> T7500
>> Workstation. The region settings are as follows;
>
> RC5 is quite old - please try RC6 or better fresh SVN.
>
The Ubuntu version and grass version used are both true 64bit?
Processing over 130 million cells with r.watershed in memory and 12GB
is possible, but only if grass is compiled as a 64bit application.
SECTION 4: Watershed determination is pretty much unchanged since RC5,
the segfault might also happen in current 6.4.
The combination of threshold=10 and over 130 million cells is a bit
unusual, producing a very large number of stream segments, but AFAICT
not large enough to trigger a segfault.
Markus M
Please upgrade to grass 6.4.0 stable, you are using out-of-date grass 6.4.RC6
Also note that r.watershed is really just a front-end for two other
programs, r.watershed.ram and r.watershed.seg. The segfault occurs in
r.watershed.ram, not in r.watershed. After you've updated your grass
version, and if you still experience the segfault, you could try to
debug r.watershed.ram e.g. with gdb
(gdb) run /usr/local/grass-6.4.0RC6/etc/r.watershed.ram
el="aster_srtm_dem_filled@CloudRemoval" t=100 se="aster_stream"
instead of
(gdb) run elevation=aster_srtm_dem_filled@CloudRemoval threshold=100
stream=aster_stream memory=300
Markus M
On Fri, Sep 17, 2010 at 8:24 PM, Chris Carleton <w.ccarleton@gmail.com> wrote:
Hi All,
I'm back at this problem again. I could really use a hand tracking this
segfault down and fixing this all up. I don't have much experience debugging
code that I didn't write myself so I'm a bit lost (or large projects like
GRASS). Attached to this email are reports from strace and valgrind. Here's
what I've got from GDB,
(gdb) set print inferior-events on
(gdb) run elevation=aster_srtm_dem_filled@CloudRemoval threshold=100
stream=aster_stream memory=300
Starting program: /usr/local/grass-6.4.0RC6/bin/r.watershed
elevation=aster_srtm_dem_filled@CloudRemoval threshold=100
stream=aster_stream memory=300
D1/1: Mode: All in RAM
D1/1: Running: /usr/local/grass-6.4.0RC6/etc/r.watershed.ram
el="aster_srtm_dem_filled@CloudRemoval" t=100 se="aster_stream"
SECTION 1a (of 5): Initiating Memory.
SECTION 1b (of 5): Determining Offmap Flow.
100%
SECTION 2: A * Search.
100%
SECTION 3: Accumulating Surface Flow.
100%
SECTION 4: Watershed determination.
Segmentation fault
WARNING: Subprocess failed with exit code 35584
WARNING: category information for [aster_stream] in [geophys] missing or
invalid
Program exited normally.
[Inferior 14967 exited]
I'm using GRASS on a Dell Precision workstation with Ubuntu 10,04 LTS 64bit.
I compiled GRASS a couple of weeks ago with 64 bit support (including large
files) and I don't know where to go from here. Any help is greatly
appreciated,
Chris
On 30 August 2010 07:55, Markus Metz <markus.metz.giswork@googlemail.com>
wrote:
Tested with a 144 million cells DEM and threshold=10, r.watershed
completes successfully here. Can you try to debug with gdb or
valgrind? See [1] for some info on GRASS debugging.
Markus M
[1] http://grass.osgeo.org/wiki/GRASS_Debugging
Markus Metz wrote:
> Chris Carleton wrote:
>>
>> As for the r.watershed problem, it's occurring
>> again and I'm now using GRASS 6.4 RC6 with 64bit support.
>
> Apparently you are comfortable to build from source. No need to use
> outdated RC6, use the latest weekly snapshot instead, available at [1]
>
>>
>> Here are the region settings I'm using;
>> zone: 16
>> datum: wgs84
>> ellipsoid: wgs84
>> north: 1942292.77488498
>> south: 1798172.56898554
>> west: 174812.49733566
>> east: 382055.4579663
>> nsres: 15.00002143
>> ewres: 15.00021429
>> rows: 9608
>> cols: 13816
>> cells: 132744128
>>
>> And here is the output in the terminal from strace and r.watershed;
>>
>> GRASS 6.4.0RC6 (ccarleton.minanha.utm):~/ > strace r.watershed
>> elevation=aster_srtm_dem_filled@CloudRemoval threshold=100
>> stream=aster_srtm_streams
>>
>> ...
>>
>> SECTION 1b (of 5): Determining Offmap Flow.
>> 100%
>> SECTION 2: A * Search.
>> 100%
>> SECTION 3: Accumulating Surface Flow.
>> 100%
>> SECTION 4: Watershed determination.
>> Segmentation fault
>
> I will test it myself next week, currently on the road.
>
> Markus M
>
> [1] http://grass.osgeo.org/grass64/source/snapshot/
>
>
>>
>> I've attached the whole strace stdout to this email. Again, I'm using
>> Ubuntu
>> 10.04 LTS 64bit. Does anyone have any ideas about where to start
>> troubleshooting this? Thanks,
>>
>> Chris
>>
>>
>> On 24 August 2010 09:55, Chris Carleton <w.ccarleton@gmail.com> wrote:
>>>
>>> Okay thanks. I was using a 'trial-and-error' method for determining
>>> the
>>> threshold, but I think I ought to start large and proceed toward
>>> smaller
>>> values. I haven't come across any analytical method for determining
>>> the best
>>> threshold value for stream extraction. Some smart person needs to
>>> develop a
>>> statistically sound method for assessing the uncertainty associated
>>> with
>>> geophysical feature extraction so that multiple results can be
>>> compared
>>> meaningfully and we're not left to assess the validity of a result on
>>> the
>>> basis of whether or not it 'looks okay' and preferably that doesn't
>>> involve
>>> going into the field to count streams. I'm going to compile GRASS from
>>> source with the latest stable release and configure for 64bit support.
>>> I
>>> think the current install was done through the repository so it's
>>> behind and
>>> not optimized. On that note, I'm trying to figure out what the
>>> appropriate
>>> configure flag(s) is(are) for HDF4 and HDF5 support in GDAL so that I
>>> can
>>> open HDF files in GRASS. The only like that I can find that seems to
>>> be
>>> discussing it formally is here:
>>>
>>> http://trac.osgeo.org/gdal/wiki/HDF
>>>
>>> Does anyone have the answer handy or another link with instructions
>>> that
>>> are more clear? Thanks,
>>>
>>> Chris
>>>
>>> On 24 August 2010 05:39, Markus Metz
>>> <markus.metz.giswork@googlemail.com>
>>> wrote:
>>>>
>>>> Martin Landa:
>>>> >
>>>> > Chris Carleton:
>>>> >> I haven't been able to find something in the mailing lists about
>>>> >> this,
>>>> >> but
>>>> >> if you know of somewhere that I can find a solution I'd appreciate
>>>> >> the
>>>> >> link.
>>>> >> I'm running GRASS 6.4 RC5 on Ubuntu LTS 10.04 with a Dell
>>>> >> Precision
>>>> >> T7500
>>>> >> Workstation. The region settings are as follows;
>>>> >
>>>> > RC5 is quite old - please try RC6 or better fresh SVN.
>>>> >
>>>> The Ubuntu version and grass version used are both true 64bit?
>>>> Processing over 130 million cells with r.watershed in memory and 12GB
>>>> is possible, but only if grass is compiled as a 64bit application.
>>>>
>>>> SECTION 4: Watershed determination is pretty much unchanged since
>>>> RC5,
>>>> the segfault might also happen in current 6.4.
>>>>
>>>> The combination of threshold=10 and over 130 million cells is a bit
>>>> unusual, producing a very large number of stream segments, but AFAICT
>>>> not large enough to trigger a segfault.
>>>>
>>>> Markus M
>>>>
>>>
>>
>>
>