[GRASS-dev] [GRASS GIS] #2509: d.mon output overwrite

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------
`d.mon` checks output file only if `output` option is given

{{{
d.mon cairo out=map.png
ERROR: option <output>: <map.png> exists.
}}}

but not when using default file name of given display driver.

{{{
d.mon cairo
}}}

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------
Changes (by martinl):

* cc: grass-dev@… (added)
  * owner: grass-dev@… => martinl
  * status: new => assigned

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:1&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by wenzeslaus):

This applies to any `d.*` command which is drawing something.

Start GRASS and use in terminal:

{{{
d.rast elevation
}}}

`map.png` now contains image of elevation raster.

{{{
d.vect streams
}}}

`map.png` now contains image of streams vector.

You don't need to call `d.mon` in order to write to a file named
`map.png`. So, I'm not sure if what you are reporting, is an bug. It is
perhaps just undocumented feature. The unchecked overwrite is a bit
inconsistent but it also makes sense in this case. I recently learned
about this behavior in [http://lists.osgeo.org/pipermail/grass-
dev/2014-December/072250.html discussion related to r63227 - in
grass/trunk/scripts] ([http://osgeo-org.1560.x6.nabble.com/Re-GRASS-
SVN-r63227-in-grass-trunk-scripts-d-correlate-d-out-file-d-rast-
leg-d-redraw-d-shadedmap-d-t-td5175707.html nabble],
[http://comments.gmane.org/gmane.comp.gis.grass.devel/61537 gmane]).

I don't have any idea what should be the right behavior. It would be good
to have different use cases to understand how it is actually used (or
supposed to be used).

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:2&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by martinl):

Replying to [comment:2 wenzeslaus]:
> This applies to any `d.*` command which is drawing something.

This ticket is just related to `d.mon` (when you start a new monitor).

> Start GRASS and use in terminal:
>
> {{{
> d.rast elevation
> }}}
>
> `map.png` now contains image of elevation raster.
>
> {{{
> d.vect streams
> }}}
>
> `map.png` now contains image of streams vector.

`map.png` contains both raster and vector map, which is OK.

> You don't need to call `d.mon` in order to write to a file named
`map.png`. So, I'm not sure

I am not sure what you are referring to. Anyway `d.mon out=` checks if the
file exists. If `--overwrite` is given than the file is overwritten. If
you use `--overwrite` together with `-u` flag that the file is open
'update' mode, ie. new data are rendered on the top of the existing file.

This ticket is related to the fact that `d.mon` behaves differently when
launched without `output` option (when using default display driver output
file).

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:3&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by neteler):

Replying to [comment:2 wenzeslaus]:
> You don't need to call `d.mon` in order to write to a file named
`map.png`.

Is this a known feature and the default? I just discovered this recently,
especially since the d.* modules do not print a message that "map.png" was
generated. So I find these files scattered all over my disk :slight_smile:

In essence, is the default driver the cairo or png driver, and controlled
by the variables listed in

http://grass.osgeo.org/grass70/manuals/variables.html#list-of-selected-
grass-environment-variables-for-rendering

?

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:4&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by wenzeslaus):

Replying to [comment:4 neteler]:
> So I find these files scattered all over my disk :slight_smile:

This might be something different but probably related. Call of `d.rast`
when `d.mon wx0` is active causes creation of `map.png` in the current
directory:

{{{
> ls map.png
ls: cannot access map.png: No such file or directory
> d.mon wx0
> ls map.png
ls: cannot access map.png: No such file or directory
> d.rast elevation
> ls map.png
map.png
}}}

I observe this for a long time already but until recently I wasn't able to
find a cause. No further investigation done, however.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:5&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by neteler):

Replying to [comment:5 wenzeslaus]:
> Replying to [comment:4 neteler]:
> > So I find these files scattered all over my disk :slight_smile:
>
> This might be something different but probably related. Call of `d.rast`
when `d.mon wx0` is active causes creation of `map.png` in the current
directory:
> I observe this for a long time already but until recently I wasn't able
to find a cause.

I tried as well - must be display driver related of course. A rather
annoying problem...

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:6&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by martinl):

Summary: originally reported bug has been fixed in r63466 and related
commits (r63467 and r63465).

{{{
ls map.png ; d.mon cairo
map.png
ERROR: option <output>: <map.png> exists.
}}}

{{{
d.mon cairo --o
WARNING: File 'map.png' already exists and will be overwritten
Output file: /opt/src/grass_trunk/map.png
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2509#comment:7&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by martinl):

Replying to [comment:6 neteler]:

> I tried as well - must be display driver related of course. A rather
annoying problem...

yes, it comes from `D_close_driver()`, will look at it ASAP.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2509#comment:8&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by martinl):

Replying to [comment:6 neteler]:

> > This might be something different but probably related. Call of
`d.rast` when `d.mon wx0` is active causes creation of `map.png` in the
current directory:
> > I observe this for a long time already but until recently I wasn't
able to find a cause.
>
> I tried as well - must be display driver related of course. A rather
annoying problem...

related to this issue something changed in display architecture (probably
Glynn will know more), now you don't need to start any monitor to render
data

{{{
d.mon -p
No monitor selected

d.vect streams --v
Using display driver <cairo>...
cairo: collecting to file 'map.png'
cairo: image size 640x480
Plotting...
8554 lines plotted
d.vect complete.
}}}

I would see this behaviour as misleading from the user point of view. In
GRASS 7.0:

{{{
d.mon -p
No monitor selected
d.vect streams
ERROR: No graphics device selected. Use d.mon to select graphics device.
}}}

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:9&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by martinl):

Replying to [comment:9 martinl]:
> Replying to [comment:6 neteler]:
> related to this issue something changed in display architecture
(probably Glynn will know more), now you don't need to start any monitor
to render data

it's r63361 (btw, the log doesn't explain why it bogus...). I really don't
see the current behaviour of display commands in trunk as reasonable. The
data are rendered automatically by default display driver without any
noticing to the user. I strongly suggest to change it back as it works in
G70 - to require to start any monitor - otherwise the user will very
confused. As we are now.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:10&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by glynn):

Replying to [comment:10 martinl]:

> > related to this issue something changed in display architecture
(probably Glynn will know more), now you don't need to start any monitor
to render data
>
> it's r63361 (btw, the log doesn't explain why it bogus...).

It's bogus because there's simply no reason for that check to be there.
Everything worked fine before someone mistakenly added it, and works fine
now that it has been removed. The only consequence of that check was to
generate an error for entirely valid usage.

> I really don't see the current behaviour of display commands in trunk as
reasonable.

The current behaviour is exactly the same as the old behaviour before
someone decided to try to reincarnate the "monitor" concept from prior
versions and introduced a bug in the process.

> The data are rendered automatically by default display driver without
any noticing to the user. I strongly suggest to change it back as it works
in G70 - to require to start any monitor - otherwise the user will very
confused. As we are now.

Just because someone managed to introduce a bug which went un-noticed for
3 years, that doesn't make it correct behaviour. GRASS 7.0 hasn't been
released yet, so there's no imperative for compatibility.

To re-iterate: GRASS_RENDER_IMMEDIATE is not required for normal operation
(generating image files). It isn't supposed to be required, it wasn't
required until someone introduced a bug (probably because they thought
that it was required), it is no longer required now that the bug has been
fixed.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:11&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by neteler):

Replying to [comment:11 glynn]:
> Replying to [comment:10 martinl]:
> > I really don't see the current behaviour of display commands in trunk
as reasonable.
>
> The current behaviour is exactly the same as the old behaviour before
someone decided to try to reincarnate the "monitor" concept from prior
versions and introduced a bug in the process.

This happens.

However, the monitor concept is essential also for GRASS 7. At least for
power users who stick to the command line. It is sad enough that the fast
x0 monitor from GRASS 6 hasn't been replaced with a fast alternative.

The current behaviour of d.* to silently write into a PNG file is
unhelpful. At least a message about the newly generated/modified file
should be generated, otherwise the d.* behaviour remains obscure.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:12&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by martinl):

Replying to [comment:11 glynn]:

> It's bogus because there's simply no reason for that check to be there.
Everything worked fine before someone mistakenly added it, and works fine
now that it has been removed. The only consequence of that check was to
generate an error for entirely valid usage.

well, it worked fine also before you reverted it. The reason of this
condition was simple. Not to generate output by the default display driver
(ie. to write map.png to the current directory) without noticing to the
user. Such behaviour is really obscure. The user should be asked to start
monitor (`d.mon`) or to specify display driver by environmental variable
(GRASS_RENDER_IMMEDIATE).

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:13&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by martinl):

Replying to [comment:11 glynn]:

> The current behaviour is exactly the same as the old behaviour before
someone decided to try to reincarnate the "monitor" concept from prior
versions and introduced a bug in the process.

please could you elaborate which kind of bug you fixed? From that you
wrote it seems to me that you "reincarnate" old behaviour - to silently
produce output from display driver without asking user or even notifying
him/her.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:14&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by glynn):

> However, the monitor concept is essential also for GRASS 7. At least for
power users who stick to the command line.

Why? Someone who "sticks to the command line" won't even have wxGUI
running, in which case d.mon etc don't do anything for them.

> The current behaviour of d.* to silently write into a PNG file is
unhelpful.

Yet, that's how those commands behaved since the display system was re-
written. I don't recall any discussion about the behaviour; it got changed
by accident and now that's how it's "supposed" to be? When did we agree to
this change, and why?

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:15&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by neteler):

Replying to [comment:15 glynn]:
> > However, the monitor concept is essential also for GRASS 7. At least
for power users who stick to the command line.
>
> Why? Someone who "sticks to the command line" won't even have wxGUI
running, in which case d.mon etc don't do anything for them.

OK, then let's call it a "hybrid use mode": Doing everything from command
line including opening and closing the wx0/wxN monitor and displaying.
Just for map queries and zooming it is handy to use the mouse as in GRASS
6.

> > The current behaviour of d.* to silently write into a PNG file is
unhelpful.
>
> Yet, that's how those commands behaved since the display system was re-
written.

Maybe that was the idea but for sure not clearly communicated or
implemented in all d.* modules.

> I don't recall any discussion about the behaviour; it got changed by
accident and now that's how it's "supposed" to be? When did we agree to
this change, and why?

Now which change? I just asked that the d.* modules please say what they
do since I don't check the file system for a new file every time I issue a
command.

If it matters, more people are interested in a working d.mon/command line
approach: see #1719,#2233, #2234 and many email discussions in the mailing
lists (or http://gis.stackexchange.com/questions/71124/is-d-mon-xn-
deprecated-in-grass-in-favor-of-something-in-the-wxgui or elsewhere).

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:16&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by martinl):

Replying to [comment:7 martinl]:
> Summary: originally reported bug has been fixed in r63466 and related
commits (r63467 and r63465).

For record: backported to relbr70 in r63533.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:17&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by glynn):

Replying to [comment:16 neteler]:

> > > The current behaviour of d.* to silently write into a PNG file is
unhelpful.
> >
> > Yet, that's how those commands behaved since the display system was
re-written.
>
> Maybe that was the idea but for sure not clearly communicated or
implemented in all d.* modules.

I'm not sure how it could have been communicated more clearly. And the
behaviour is consistent across all display[*] modules. It cannot be
otherwise, as the behaviour comes from lib/display, not any particular
module.

[*] I'd like to say "all d.* modules", but there are a number of cases
where people have used that prefix for modules which are unrelated to the
display system, e.g. d.mon, d.out.file, etc.

> > I don't recall any discussion about the behaviour; it got changed by
accident and now that's how it's "supposed" to be? When did we agree to
this change, and why?
>
> Now which change?

r46984 and r46999.

When the support for 6.x-style monitors was removed in r32584, immediate
rendering became the only supported mode of operation (as was already the
case on Windows). Thus, it was no longer necessary to set
GRASS_RENDER_IMMEDIATE; executing any display command would simply
generate an image file (default map.png, configurable via $GRASS_PNGFILE,
later changed to $GRASS_RENDER_FILE).

r46984 and r46999 re-introduce a form of the "monitor" concept (although
I'm not clear on the details, it has something to do with $MONITOR).
AFAICT, r46984 effectively disabled direct rendering. r46999 re-enabled
it, but forced $GRASS_RENDER_IMMEDIATE to be explicitly set (if neither
$MONITOR nor $GRASS_RENDER_IMMEDIATE are set, it generated a fatal error).

In case it wasn't clear from [http://lists.osgeo.org/pipermail/grass-
dev/2014-December/072274.html this thread], the requirement for
GRASS_RENDER_IMMEDIATE to be set was news to me (I had explicitly set it
while investigating differences between the cairo and PNG drivers, and
forgotten to remove the setting).

> I just asked that the d.* modules please say what they do since I don't
check the file system for a new file every time I issue a command.

You were unaware of immediate rendering? Or of it having been the default
since the earliest days of GRASS 7?

> If it matters, more people are interested in a working d.mon/command
line approach:

I don't doubt it. I've tried to have discussions on such things in the
past. But some people prefer to just implement hacks like $MONITOR without
any discussion. Ultimately, that's likely to be counter-productive in
terms of any reasonable solution.

FWIW, I don't actually object to requiring $GRASS_RENDER_IMMEDIATE to be
set. I do object to it being done by /fait accompli/. I'm also less than
keen on the way that wxGUI seems to be slowly getting shoehorned into
somehow being "part of" the display system (e.g. using the d.* prefix for
commands which have nothing to do with the display system per se).

If the wxGUI developers need additional functionality from the display
system, that should be resolved through discussion rather than "edit
wars". Otherwise, it's likely to result in a display system which is of no
use for anything but wxGUI.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:18&gt;
GRASS GIS <http://grass.osgeo.org>

#2509: d.mon output overwrite
-------------------------+--------------------------------------------------
Reporter: martinl | Owner: martinl
     Type: defect | Status: assigned
Priority: normal | Milestone: 7.0.0
Component: Display | Version: unspecified
Keywords: d.mon | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by martinl):

Replying to [comment:18 glynn]:

> > If it matters, more people are interested in a working d.mon/command
line approach:
>
> I don't doubt it. I've tried to have discussions on such things in the
past. But some people prefer to just implement hacks like $MONITOR without
any discussion. Ultimately, that's likely to be counter-productive in
terms of any reasonable solution.
>
> FWIW, I don't actually object to requiring $GRASS_RENDER_IMMEDIATE to be
set. I do object to it being done by /fait accompli/. I'm also less than
keen on the way that wxGUI seems to be

Let's start with fundamental points:

  * some users like monitor concept (GUI monitors, and "file-based"
monitors), we should keep it since there is relatively strong request to
have them
  * direct rendering is a nice feature, but should be enabled only on
demand (eg. when $GRASS_RENDER_IMMEDIATE is set), otherwise the most of
users will be confused why display modules are producing `map.png` (or
`map.ps`...) files in the current directory just without asking

Can we have consensus on these points above? If so let's focus on
implementation of the second issue and after releasing 7.0 we can discuss
better implementation of monitors rather than using hacks like $MONITOR.
Make sense to you?

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2509#comment:19&gt;
GRASS GIS <http://grass.osgeo.org>