I am running a large number of processing jobs on a
cluster and the job manager output is cluttered with
control chars from G_percent().
It would be great to have a switch (or detection magic)
to output the plain ASCII style 0..10...20.. in this cases.
Maybe it's already there?
Markus
I am running a large number of processing jobs on a
cluster and the job manager output is cluttered with
control chars from G_percent().
It would be great to have a switch (or detection magic)
to output the plain ASCII style 0..10...20.. in this cases.Maybe it's already there?
set GRASS_VERBOSE=0 ?
Hamish
On Tue, Nov 4, 2008 at 8:41 AM, Hamish <hamish_b@yahoo.com> wrote:
I am running a large number of processing jobs on a
cluster and the job manager output is cluttered with
control chars from G_percent().
It would be great to have a switch (or detection magic)
to output the plain ASCII style 0..10...20.. in this cases.Maybe it's already there?
set GRASS_VERBOSE=0 ?
No, because some of these jobs runs for many hours and I
would like to see the progress (to estimate the total time).
In this case I am running v.vol.rst...
Markus
On Tue, Nov 4, 2008 at 8:23 AM, Markus Neteler <neteler@osgeo.org> wrote:
I am running a large number of processing jobs on a
cluster and the job manager output is cluttered with
control chars from G_percent().
It would be great to have a switch (or detection magic)
to output the plain ASCII style 0..10...20.. in this cases.
Suggestion: Could this be conditionalized upon the presence of the
GRASS_BATCH_JOB
variable?
Markus
On Tue, 4 Nov 2008, Markus Neteler wrote:
On Tue, Nov 4, 2008 at 8:41 AM, Hamish <hamish_b@yahoo.com> wrote:
I am running a large number of processing jobs on a
cluster and the job manager output is cluttered with
control chars from G_percent().
It would be great to have a switch (or detection magic)
to output the plain ASCII style 0..10...20.. in this cases.Maybe it's already there?
set GRASS_VERBOSE=0 ?
No, because some of these jobs runs for many hours and I
would like to see the progress (to estimate the total time).
Hi Markus,
I'm interested as to what your log file looks like and how you discern which percentage value is coming from which process. Are they not all mixed together anyway? I was thinking perhaps you could filter the stderr of each GRASS module through a short C program that could strip the backspace characters and print the name of the module with it's progress value on a new line every time there is a new percentage point? Or is the problem not complicated enough to need such a solution?
Paul
On Tue, Nov 4, 2008 at 1:52 PM, Paul Kelly
<paul-grass@stjohnspoint.co.uk> wrote:
On Tue, 4 Nov 2008, Markus Neteler wrote:
On Tue, Nov 4, 2008 at 8:41 AM, Hamish <hamish_b@yahoo.com> wrote:
I am running a large number of processing jobs on a
cluster and the job manager output is cluttered with
control chars from G_percent().
It would be great to have a switch (or detection magic)
to output the plain ASCII style 0..10...20.. in this cases.Maybe it's already there?
set GRASS_VERBOSE=0 ?
No, because some of these jobs runs for many hours and I
would like to see the progress (to estimate the total time).Hi Markus,
Hello Paul,
I'm interested as to what your log file looks like and how you discern which
percentage value is coming from which process. Are they not all mixed
together anyway?
The job manager ([SUN] Grid Engine) separates stderr and stdout into
two files. The stderr file contains
...
0%^H^H^H^H^H 3%^H^H^H^H^H 6%^H^H^H^H^H 9%^H^H^H^H^H
12%^H^H^H^H^H 15%^H^H^H^H^H 18%^H^H^H^H^H 21%^H^H^H^H^H
24%^H^H^H^H^H 27%^H^H^H^H^H 30%^H^H^H^H^H 33%^H^H^H^H^H
36%^H^H^H^H^H 39%^H^H^H^H^H 42%^H^H^H^H^H 45%^H^H^H^H^H
48%^H^H^H^H^H 51%^H^H^H^H^H 54%^H^H^H^H^H 57%^H^H^H^H^H
60%^H^H^H^H^H 63%^H^H^H^H^H 66%^H^H^H^H^H 69%^H^H^H^H^H
72%^H^H^H^H^H 75%^H^H^H^H^H 78%^H^H^H^H^H 81%^H^H^H^H^H
84%^H^H^H^H^H 87%^H^H^H^H^H 90%^H^H^H^H^H 93%^H^H^H^H^H
96%^H^H^H^H^H 99%^H^H^H^H^H 100%^H^H^H^H^H
0%^H^H^H^H^H 3%^H^H^H^H^H 6%^H^H^H^H^H 9%^H^H^H^H^H
12%^H^H^H^H^H 15%^H^H^H^H^H 18%^H^H^H^H^H 21%^H^H^H^H^H
24%^H^H^H^H^H 27%^H^H^H^H^H 30%^H^H^H^H^H 33%^H^H^H^H^H
36%^H^H^H^H^H 39%^H^H^H^H^H 42%^H^H^H^H^H 45%^H^H^H^H^H
48%^H^H^H^H^H 51%^H^H^H^H^H 54%^H^H^H^H^H 57%^H^H^H^H^H
60%^H^H^H^H^H 63%^H^H^H^H^H 66%^H^H^H^H^H 69%^H^H^H^H^H
72%^H^H^H^H^H 75%^H^H^H^H^H 78%^H^H^H^H^H 81%^H^H^H^H^H
84%^H^H^H^H^H 87%^H^H^H^H^H 90%^H^H^H^H^H 93%^H^H^H^H^H
96%^H^H^H^H^H 99%^H^H^H^H^H 100%^H^H^H^H^H
Color table for <aqua_lst1km20060416.LST_Night_1km.filt2_lstmodel> set to
celsius
Collecting Stats...
0%^H^H^H^H^H 3%^H^H^H^H^H 6%^H^H^H^H^H 9%^H^H^H^H^H
12%^H^H^H^H^H 15%^H^H^H^H^H 18%^H^H^H^H^H 21%^H^H^H^H^H
24%^H^H^H^H^H 27%^H^H^H^H^H 30%^H^H^H^H^H 33%^H^H^H^H^H
36%^H^H^H^H^H 39%^H^H^H^H^H 42%^H^H^H^H^H 45%^H^H^H^H^H
48%^H^H^H^H^H 51%^H^H^H^H^H 54%^H^H^H^H^H 57%^H^H^H^H^H
60%^H^H^H^H^H 63%^H^H^H^H^H 66%^H^H^H^H^H 69%^H^H^H^H^H
72%^H^H^H^H^H 75%^H^H^H^H^H 78%^H^H^H^H^H 81%^H^H^H^H^H
84%^H^H^H^H^H 87%^H^H^H^H^H 90%^H^H^H^H^H 93%^H^H^H^H^H
96%^H^H^H^H^H 99%^H^H^H^H^H 100%^H^H^H^H^H
Writing vector map <aqua_lst1km20060416_LST_Night_1km_filt2> ...
...
... stuff like that.
I was thinking perhaps you could filter the stderr of each
GRASS module through a short C program that could strip the backspace
characters and print the name of the module with it's progress value on a
new line every time there is a new percentage point? Or is the problem not
complicated enough to need such a solution?
I guess I would have to modify Grid engine... not possible.
I can certainly do a local hack to suppress all these ^H chars but would prefer
a general solution.
It's not a dramatic issue, just to get rid of these chars would be helpful
to debug such script output files.
Markus
Markus Neteler wrote:
I am running a large number of processing jobs on a
cluster and the job manager output is cluttered with
control chars from G_percent().
It would be great to have a switch (or detection magic)
to output the plain ASCII style 0..10...20.. in this cases.Maybe it's already there?
The closest is:
export GRASS_MESSAGE_FORMAT=gui
That's a bit noisy, but it does eliminate the BS characters.
--
Glynn Clements <glynn@gclements.plus.com>
>>>> control chars from G_percent().
>>>> It would be great to have a switch (or detection magic)
>>>> to output the plain ASCII style 0..10...20.. in this cases.
because the way it is called it can't be like "0..10...20.." as with
gdal tools, but there could be a version which replaced ^H^H^H with " ".
Hamish
On Tue, Nov 4, 2008 at 8:31 PM, Glynn Clements <glynn@gclements.plus.com> wrote:
Markus Neteler wrote:
I am running a large number of processing jobs on a
cluster and the job manager output is cluttered with
control chars from G_percent().
It would be great to have a switch (or detection magic)
to output the plain ASCII style 0..10...20.. in this cases.Maybe it's already there?
The closest is:
export GRASS_MESSAGE_FORMAT=gui
That's a bit noisy, but it does eliminate the BS characters.
Right, but it's too noisy, too.
Attached a patch proposal which outputs:
export GRASS_MESSAGE_FORMAT=batch
r.slope.aspect elevation as=myaspect
2..5..8..11..14..17..20..23..26..29..32..35..38..41..44..47..50..53..56..59..62..65..68..71..74..77..80..83..86..89..92..95..98..100..
Aspect raster map <aspect> complete
Looks ok?
Markus
percent_batch.diff (1.94 KB)
Markus Neteler wrote:
>> I am running a large number of processing jobs on a
>> cluster and the job manager output is cluttered with
>> control chars from G_percent().
>> It would be great to have a switch (or detection magic)
>> to output the plain ASCII style 0..10...20.. in this cases.
>>
>> Maybe it's already there?
>
> The closest is:
>
> export GRASS_MESSAGE_FORMAT=gui
>
> That's a bit noisy, but it does eliminate the BS characters.Right, but it's too noisy, too.
Attached a patch proposal which outputs:
export GRASS_MESSAGE_FORMAT=batch
r.slope.aspect elevation as=myaspect
2..5..8..11..14..17..20..23..26..29..32..35..38..41..44..47..50..53..56..59..62..65..68..71..74..77..80..83..86..89..92..95..98..100..
Aspect raster map <aspect> completeLooks ok?
Yes, except that I'm not sure about the name; maybe "plain"?
This isn't just an issue for batch jobs; my usual "terminal" is
XEmacs' shell-mode, which doesn't process ^H (or ^M or ^G for that
matter).
[For programs which use ^M, you can at least use "stty ocrnl".]
Maybe a better solution is to allow the specification of an arbitrary
string which is used to "rewind" percent output.
--
Glynn Clements <glynn@gclements.plus.com>
On Wed, Nov 5, 2008 at 11:31 PM, Glynn Clements
<glynn@gclements.plus.com> wrote:
Markus Neteler wrote:
>> I am running a large number of processing jobs on a
>> cluster and the job manager output is cluttered with
>> control chars from G_percent().
>> It would be great to have a switch (or detection magic)
>> to output the plain ASCII style 0..10...20.. in this cases.
>>
>> Maybe it's already there?
>
> The closest is:
>
> export GRASS_MESSAGE_FORMAT=gui
>
> That's a bit noisy, but it does eliminate the BS characters.Right, but it's too noisy, too.
Attached a patch proposal which outputs:
export GRASS_MESSAGE_FORMAT=batch
r.slope.aspect elevation as=myaspect
2..5..8..11..14..17..20..23..26..29..32..35..38..41..44..47..50..53..56..59..62..65..68..71..74..77..80..83..86..89..92..95..98..100..
Aspect raster map <aspect> completeLooks ok?
Yes, except that I'm not sure about the name; maybe "plain"?
OK, submitted with this change.
Markus
On Thu, Nov 6, 2008 at 11:21 AM, Markus Neteler <neteler@osgeo.org> wrote:
On Wed, Nov 5, 2008 at 11:31 PM, Glynn Clements wrote:
Markus Neteler wrote:
>> I am running a large number of processing jobs on a
>> cluster and the job manager output is cluttered with
>> control chars from G_percent().
>> It would be great to have a switch (or detection magic)
>> to output the plain ASCII style 0..10...20.. in this cases.
This was submitted and one obtains now:
export GRASS_MESSAGE_FORMAT=plain
r.slope.aspect elevation as=myaspect
2..5..8..11..14..17..20..23..26..29..32..35..38..41..44..47..50..53..56..59..62..65..68..71..74..77..80..83..86..89..92..95..98..100..
Aspect raster map <aspect> complete
Missing is a likewise change for lib/vector/Vlib/build_nat.c - (rather
trivial) patch
attached. Objections that I submit it?
Markus
build_nat.diff (991 Bytes)
On Sun, Dec 28, 2008 at 12:54 PM, Markus Neteler <neteler@osgeo.org> wrote:
On Thu, Nov 6, 2008 at 11:21 AM, Markus Neteler <neteler@osgeo.org> wrote:
On Wed, Nov 5, 2008 at 11:31 PM, Glynn Clements wrote:
Markus Neteler wrote:
>> I am running a large number of processing jobs on a
>> cluster and the job manager output is cluttered with
>> control chars from G_percent().
>> It would be great to have a switch (or detection magic)
>> to output the plain ASCII style 0..10...20.. in this cases.This was submitted and one obtains now:
export GRASS_MESSAGE_FORMAT=plain
r.slope.aspect elevation as=myaspect
2..5..8..11..14..17..20..23..26..29..32..35..38..41..44..47..50..53..56..59..62..65..68..71..74..77..80..83..86..89..92..95..98..100..
Aspect raster map <aspect> completeMissing is a likewise change for lib/vector/Vlib/build_nat.c - (rather
trivial) patch attached. Objections that I submit it?
thinking about it I have conditionalized it upon G_INFO_FORMAT_PLAIN
to keep behaviour as closest to the previous behaviour.
New patch attached.
Markus
build_nat.diff (985 Bytes)
On Mon, Dec 29, 2008 at 4:41 PM, Markus Neteler <neteler@osgeo.org> wrote:
On Sun, Dec 28, 2008 at 12:54 PM, Markus Neteler <neteler@osgeo.org> wrote:
On Thu, Nov 6, 2008 at 11:21 AM, Markus Neteler <neteler@osgeo.org> wrote:
Markus Neteler wrote:
>> I am running a large number of processing jobs on a
>> cluster and the job manager output is cluttered with
>> control chars from G_percent().
>> It would be great to have a switch (or detection magic)
>> to output the plain ASCII style 0..10...20.. in this cases.This was submitted and one obtains now:
export GRASS_MESSAGE_FORMAT=plain
r.slope.aspect elevation as=myaspect
2..5..8..11..14..17..20..23..26..29..32..35..38..41..44..47..50..53..56..59..62..65..68..71..74..77..80..83..86..89..92..95..98..100..
Aspect raster map <aspect> completeMissing is a likewise change for lib/vector/Vlib/build_nat.c - (rather
trivial) patch attached. Objections that I submit it?thinking about it I have conditionalized it upon G_INFO_FORMAT_PLAIN
to keep behaviour as closest to the previous behaviour.New patch attached.
... and submitted. Now also the vector modules should report
percentages in batch mode as seen above.
Markus