due to strange misbehaviours, we are doing some stress tests on
r.profile under Grass 6.4 and Windows 7
Case A (less interesting, IMHO):
r.profile fails with ERROR: G_malloc when the number of pixel queried
is more than 1E6
our test: profile lenght: 300km, res=3 OK, res=2 failure ( example a
river analysis on a lidar-derived DTM)
Case B (more interesting):
r.profile fails when "profile" input is a string larger than 8100
characters (more or less)
I suppose the problem comes from a limit of cmd.exe (8191 characters)
see [0]
Please, consider that using UTM integer coordinates that means only
300 nodes, and this is not so unusual for profile analysis.
Probably, using optionally a text file as "profile" input could solve
the problem without refactoring the module...
Could be this bug solved in Grass 7?
Hello Enrico,
please open a trac wish ticket for case A, as it should be possible to
sample large datasets as to obtain the result of module it does not
require to keep whole data set in RAM.
JFYI - one solution would be to share code with v.profile add-on that
allows to use a line from existing vector layer as the profiling line.
due to strange misbehaviours, we are doing some stress tests on
r.profile under Grass 6.4 and Windows 7
Case A (less interesting, IMHO):
r.profile fails with ERROR: G_malloc when the number of pixel queried
is more than 1E6
our test: profile lenght: 300km, res=3 OK, res=2 failure ( example a
river analysis on a lidar-derived DTM)
Case B (more interesting):
r.profile fails when "profile" input is a string larger than 8100
characters (more or less)
I suppose the problem comes from a limit of cmd.exe (8191 characters)
see [0]
Please, consider that using UTM integer coordinates that means only
300 nodes, and this is not so unusual for profile analysis.
Probably, using optionally a text file as "profile" input could solve
the problem without refactoring the module...
Could be this bug solved in Grass 7?
due to strange misbehaviours, we are doing some stress tests on
r.profile under Grass 6.4 and Windows 7
Case A (less interesting, IMHO):
r.profile fails with ERROR: G_malloc when the number of pixel queried
is more than 1E6
our test: profile lenght: 300km, res=3 OK, res=2 failure ( example a
river analysis on a lidar-derived DTM)
can you provide the current region settings (g.region -p) and the exact command line parameters used?
r.profile counts "n" as integer, but that should survive well past 1 million.
?
note WinGRASS binaries are still 32 bit, and I see some float rows,cols work around in the r.profile code's do_profile() function.
it would be good to also test in 64 bit linux on a machine with the same amount of RAM.
As Maris noted, please open a bug report for this.
Case B (more interesting):
r.profile fails when "profile" input is a string larger than 8100
characters (more or less)
I suppose the problem comes from a limit of cmd.exe (8191 characters)
see [0]
Please, consider that using UTM integer coordinates that means only
300 nodes, and this is not so unusual for profile analysis.
Probably, using optionally a text file as "profile" input could
solve the problem without refactoring the module...
will get you past the command line length limit (and is easier to work
with too). what we should do is explicitly look for "profile=-" for
reading from stdin, to make it more clear that it exists. another
more GUI friendly way is to add a new option for an input file, which
would be mutually exclusive with the profile=[east,north[,east,north,...]
method of using the module.
On Thu, Feb 13, 2014 at 9:19 PM, Hamish <hamish_b@yahoo.com> wrote:
will get you past the command line length limit (and is easier to work
with too). what we should do is explicitly look for "profile=-" for
reading from stdin, to make it more clear that it exists. another
more GUI friendly way is to add a new option for an input file, which
would be mutually exclusive with the profile=[east,north[,east,north,...]
method of using the module.
I agree with more GUI friendly way. Once done in r.profile, this
functionality will be instantly available for MS Windows users (using only
GUI).
I updated the very outdated manual of r.profile by removing -i flag
documentation (which even has a replacement in GRASS 7). I noted also the
possibility of standard input.
Note that currently when you forget to fill `profile` in GUI and press Run,
you will get strange behavior. Two distinct options will avoid this.
what we should do is explicitly look for "profile=-" for
reading from stdin, to make it more clear that it exists. another
more GUI friendly way is to add a new option for an input file, which
would be mutually exclusive with the profile=[east,north[,east,north,...]
method of using the module.
Vaclav:
I agree with more GUI friendly way.
Actually it was the only way, since profile= expects comma separated
pairs, and the parser (if it was working!) refuses to accept just
one "-" for the option.
So now in trunk r59033 profile= only accepts sets of coord pairs,
and for stdin you have to use coordinate_file="-".
Once done in r.profile, this functionality will be instantly
available for MS Windows users (using only GUI).
fwiw both <stdin and command line are just as available and work in
the same way on MS Windows as in Mac and UNIX. That the MS Windows
users are not currently getting a command line dos box from the
default desktop icon is debatably a mis-feature (there's a long
discussion in a trac ticket about that).
Note that currently when you forget to fill `profile` in GUI and
press Run, you will get strange behavior.
Now it should be better, you have to leave profile= blank and also
add "-" to the coordinate_file= option. Still possible to get stuck waiting for stdin to close, but much
less probable.
Once done in r.profile, this functionality will be instantly
available for MS Windows users (using only GUI).
fwiw both <stdin and command line are just as available and work in
the same way on MS Windows as in Mac and UNIX. That the MS Windows
users are not currently getting a command line dos box from the
default desktop icon is debatably a mis-feature (there's a long
discussion in a trac ticket about that).
in very truth, I need r.profile in a Python script and <profile=>
option value is given using a string variable
I think "<stdin" is not an option but "coordinate_file" is OK ..
thanks
Note that currently when you forget to fill `profile` in GUI and
press Run, you will get strange behavior.
Now it should be better, you have to leave profile= blank and also
add "-" to the coordinate_file= option. Still possible to get stuck waiting for stdin to close, but much
less probable.
On Fri, Feb 14, 2014 at 8:08 AM, Enrico Gallo <enrico.gallo@gmail.com>wrote:
I think "<stdin" is not an option
Actually, it is for Python (m.proj example):
from grass.script import core as gcore
...
proc = gcore.start_command('m.proj', input='-', separator=' , ',
flags='od',
stdin=gcore.PIPE, stdout=gcore.PIPE,
stderr=gcore.PIPE)
proc.stdin.write(proj_in) # put string to the module stdin
proc.stdin.close()
proc.stdin = None # not sure if this really necessary
proj_out, errors = proc.communicate()
if proc.returncode:
raise RuntimeError("m.proj error: %s" % errors)
# process stdout written into the proj_out string
...
But coordinate_file might be better in your case (I'm not sure how to
rewrite code above to easily handle large amounts of coordinates).