[GRASS-dev] [GRASS GIS] #3033: Cairo and PS drivers display only one raster or vector for SVG and PS

GDAL · May 10, 2016, 2:16am

#3033: Cairo and PS drivers display only one raster or vector for SVG and PS
-------------------------------------------------+-------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: defect | Status: new
Priority: major | Milestone: 7.2.0
Component: Display | Version: svn-trunk
Keywords: d.mon, cairo, ps, SVG, vector | CPU: Unspecified
graphics |
Platform: Unspecified |
-------------------------------------------------+-------------------------
Running the following creates a image with elevation raster and roadsmajor
vector on top of it:

{{{
g.region raster=elevation
d.mon cairo o=o.png
d.rast elevation
d.vect roadsmajor
d.mon stop=cairo
}}}

However, using Cairo driver with SVG output, you get only the vector which
is rendered last:

{{{
g.region raster=elevation
d.mon cairo o=o.svg
d.rast elevation
d.vect roadsmajor
d.mon stop=cairo
}}}

Same applies for the PS driver:

{{{
g.region raster=elevation
d.mon ps o=o.ps
d.rast elevation
d.vect roadsmajor
d.mon stop=ps
}}}

It seems that there is no way in GRASS to get full vector graphics output
with display monitors.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3033>
GRASS GIS <https://grass.osgeo.org>

GDAL · May 17, 2016, 7:18am

#3033: Cairo and PS drivers display only one raster or vector for SVG and PS
-------------------------+-------------------------------------------------
  Reporter: wenzeslaus | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: major | Milestone: 7.2.0
Component: Display | Version: svn-trunk
Resolution: | Keywords: d.mon, cairo, ps, SVG, vector
       CPU: | graphics
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by glynn):

Replying to [ticket:3033 wenzeslaus]:

> However, using Cairo driver with SVG output, you get only the vector
which is rendered last:

This is a limitation of cairo's SVG (and PostScript) "surface" and the
fact that d.* commands run as distinct processes. It isn't possible to
initialise a surface from an existing SVG file so that subsequent drawing
operations will be appended to the document structure, and you can't
simply concatenate the files.

In other words, GRASS_RENDER_FILE_READ=TRUE doesn't work for vector
formats. You need to generate multiple output files then combine them with
external tools.

> Same applies for the PS driver:

The PS driver does support combining multiple commands into a single
stream, via the environment variables GRASS_RENDER_PS_HEADER and
GRASS_RENDER_PS_TRAILER (although I have no idea whether these can be
controlled via d.mon).

With GRASS_RENDER_PS_HEADER=FALSE, the output file is opened for append
rather than write (i.e. the existing contents are retained) and no header
(prologue) is written.

With GRASS_RENDER_PS_TRAILER=FALSE, no trailer is written.

Both variables default to TRUE, meaning that each command will replace any
previous file and a header and trailer will be included. To combine the
output from multiple commands into a single file, all commands but the
first should have GRASS_RENDER_PS_HEADER=FALSE, while all commands but the
last should have GRASS_RENDER_PS_TRAILER=FALSE.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3033#comment:1>
GRASS GIS <https://grass.osgeo.org>

GDAL · May 17, 2016, 10:24pm

#3033: Cairo and PS drivers display only one raster or vector for SVG and PS
-------------------------+-------------------------------------------------
  Reporter: wenzeslaus | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: major | Milestone: 7.2.0
Component: Display | Version: svn-trunk
Resolution: | Keywords: d.mon, cairo, ps, SVG, vector
       CPU: | graphics
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by wenzeslaus):

Replying to [comment:1 glynn]:
> It isn't possible to initialise a surface from an existing SVG file so
that subsequent drawing operations will be appended to the document
structure, and you can't simply concatenate the files.

This is serious limitation. Is there a way to change the rendering to not
have this problem.

> You need to generate multiple output files then combine them with
external tools.

I was able to use HTML:

{{{
g.region rast=elevation
d.mon cairo output=elevation.svg
d.rast elevation
d.mon stop=cairo
d.mon cairo output=streams.svg
d.vect streams color=blue
d.mon stop=cairo
d.mon cairo output=roadsmajor.svg
d.vect roadsmajor
d.mon stop=cairo
}}}

{{{
#!text/html
<html>
<head>
<style>
object {
     position: absolute;
     top: 0;
     left: 0;
}
</style>
<head>
<body>
# also svg element can be used
<object type="image/svg+xml" data="elevation.svg"></object>
<object type="image/svg+xml" data="streams.svg"></object>
<object type="image/svg+xml" data="roadsmajor.svg"></object>
</body>
</html>
}}}

Unfortunately, this is not a sufficient workaround if you need to modify
the file.

Combining manually in Inkscape was not possible; I was not able to align
the layers. Command line Inkscape doesn't read multiple files. !ImageMagic
creates a raster.

> To combine the output from multiple commands into a single file, all
commands but the first should have GRASS_RENDER_PS_HEADER=FALSE, while all
commands but the last should have GRASS_RENDER_PS_TRAILER=FALSE.

Thanks for the tip, this worked:

{{{
g.region rast=elevation
export GRASS_RENDER_PS_HEADER=TRUE
export GRASS_RENDER_PS_TRAILER=FALSE
d.mon ps output=all.ps
d.rast elevation
export GRASS_RENDER_PS_HEADER=FALSE
d.vect streams color=blue
export GRASS_RENDER_PS_TRAILER=TRUE
d.vect roadsmajor
# export here worked as well
d.mon stop=ps
}}}

I followed with

{{{
pdf2svg all.pdf all.svg
ps2pdf all.ps all.pdf
}}}

to get SVG. But it seems that PS driver is not as powerful as Cairo and
same things are lost during the subsequent conversions, page size was
wrong as well.

Unfortunately, this doesn't seem to be applicable for GUI which would be
likely used when one wants to edit the file afterwards.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3033#comment:2>
GRASS GIS <https://grass.osgeo.org>

GDAL · May 19, 2016, 5:07am

#3033: Cairo and PS drivers display only one raster or vector for SVG and PS
-------------------------+-------------------------------------------------
  Reporter: wenzeslaus | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: major | Milestone: 7.2.0
Component: Display | Version: svn-trunk
Resolution: | Keywords: d.mon, cairo, ps, SVG, vector
       CPU: | graphics
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by glynn):

Replying to [comment:2 wenzeslaus]:

> This is serious limitation. Is there a way to change the rendering to
not have this problem.

Write SVG to a temporary file, read and parse the original file and
temporary file, merge SVG documents, write XML.

Merging the documents isn't exactly trivial, e.g. you need to rename
elements in <defs> sections (and any references to them) to avoid name
collisions. This might be feasible if we can figure out some bounds on
what we can expect cairo to generate, but it's still a fair amount of work
and seems fragile (e.g. future versions of cairo may change the document
structure).

> Combining manually in Inkscape was not possible; I was not able to align
the layers.

Inkscape is garbage for anything with technical constraints. I usually end
up writing Python scripts to manipulate the XML.

> But it seems that PS driver is not as powerful as Cairo and same things
are lost during the subsequent conversions, page size was wrong as well.

Converting !PostScript invariably loses information. Being a programming
language rather than a data format means that you can't extract high-
levels structure, you just have to execute it and capture the rendering
primitives.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3033#comment:3>
GRASS GIS <https://grass.osgeo.org>

GDAL · May 19, 2016, 9:02am

#3033: Cairo and PS drivers display only one raster or vector for SVG and PS
-------------------------+-------------------------------------------------
  Reporter: wenzeslaus | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: major | Milestone: 7.2.0
Component: Display | Version: svn-trunk
Resolution: | Keywords: d.mon, cairo, ps, SVG, vector
       CPU: | graphics
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by mlennert):

Just thinking out loud here: it seems to me that by trying to do all this
we will end up with a second ps.map... Wouldn't it be easier to try to
develop a way to translate the d. commands in the current display to a
ps.map instruction file and then just run that ?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3033#comment:4>
GRASS GIS <https://grass.osgeo.org>

GDAL · May 19, 2016, 1:45pm

#3033: Cairo and PS drivers display only one raster or vector for SVG and PS
-------------------------+-------------------------------------------------
  Reporter: wenzeslaus | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: major | Milestone: 7.2.0
Component: Display | Version: svn-trunk
Resolution: | Keywords: d.mon, cairo, ps, SVG, vector
       CPU: | graphics
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by wenzeslaus):

Replying to [comment:3 glynn]:
> Merging the documents isn't exactly trivial...

That's why I wanted to avoid that. It seems that having the rendering
functionality split into commands creates many problems.

> Inkscape is garbage for anything with technical constraints. I usually
end up writing Python scripts to manipulate the XML.

So, it seems. Having such a script in GRASS and also incorporated in GUI
would be nice. The role of Inkscape here is really the manual fine tuning
afterwards (the main motivation for this ticket).

> Converting !PostScript invariably loses information.

Makes sense, of course. That's why we need Cairo driver to support it.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3033#comment:5>
GRASS GIS <https://grass.osgeo.org>

GDAL · May 19, 2016, 2:00pm

#3033: Cairo and PS drivers display only one raster or vector for SVG and PS
-------------------------+-------------------------------------------------
  Reporter: wenzeslaus | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: major | Milestone: 7.2.0
Component: Display | Version: svn-trunk
Resolution: | Keywords: d.mon, cairo, ps, SVG, vector
       CPU: | graphics
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by wenzeslaus):

Replying to [comment:4 mlennert]:
> Just thinking out loud here: it seems to me that by trying to do all
this we will end up with a second ps.map...

I'm not sure what you mean by ''all this'', but my original idea before
creating this ticket was to create a map (ideally in GUI) and instead of
saving it as PNG, save it in a SVG, so I can edit it Inkscape.

> Wouldn't it be easier to try to develop a way to translate the d.
commands in the current display to a ps.map instruction file and then just
run that ?

That's of course interesting (as well as the other way around is), not
even mentioning never finished [https://grasswiki.osgeo.org/wiki/Ps.output
ps.output]. However, PS outputs are not ideal for further editing in
Inkscape.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3033#comment:6>
GRASS GIS <https://grass.osgeo.org>

GDAL · May 21, 2016, 4:56am

#3033: Cairo and PS drivers display only one raster or vector for SVG and PS
-------------------------+-------------------------------------------------
  Reporter: wenzeslaus | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: major | Milestone: 7.2.0
Component: Display | Version: svn-trunk
Resolution: | Keywords: d.mon, cairo, ps, SVG, vector
       CPU: | graphics
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by glynn):

Replying to [comment:5 wenzeslaus]:

> That's why I wanted to avoid that. It seems that having the rendering
functionality split into commands creates many problems.

It also provides flexibility that a monolithic approach such as ps.map
inherently lacks.

> Makes sense, of course. That's why we need Cairo driver to support it.

Another option, which isn't entirely trivial but possibly simpler than
(reliably) merging SVG documents, would be a "capture" driver which
serialises the driver calls and their parameters to a file for subsequent
rendering. This way, you could capture the output from multiple d.*
commands then render the final result via the cairo driver (or any other
driver, but it's only really useful in conjunction with cairo's vector
output formats).

Structurally, this would be quite similar to the PS driver. Actually,
parsing the output from the PS driver would probably work (mostly), and
avoid the need for an additional driver.

Except, we'd need to add the text methods (the cairo driver supports
"driver" fonts which cause text rendering to be passed to the driver; the
PS driver currently only supports stroke and freetype fonts which are
converted to lines or rasters by the driver library then rendered using
the driver's line/raster methods).

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3033#comment:7>
GRASS GIS <https://grass.osgeo.org>

GDAL · June 28, 2016, 11:09am

#3033: Cairo and PS drivers display only one raster or vector for SVG and PS
-------------------------+-------------------------------------------------
  Reporter: wenzeslaus | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: major | Milestone: 7.2.0
Component: Display | Version: svn-trunk
Resolution: | Keywords: d.mon, cairo, ps, SVG, vector
       CPU: | graphics
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by kuszinger):

Hello,

Starting from the utf-8 ps.map problem I arrived to the very same issues
your are talking about here.
I think that PS (PDF) output is essential when print is the final target.
I happened to me quite often so I worked with ps.map (manual assembly of
ps-map command files, no gui, but this is my taste only). When talking ps
embedded font or even glyphs converted to paths (lines, strokes, whatever
you call them) is also essential for quality. Also, lines as lines, vector
as vector. I never missed opacity options in final products however I like
it on screen visualizations.

Creating ps layers and merge them would be enough for me in case of utf-8
handling and font->path conversion.

Until then I started to create some scripts which runs well and generates
bitmap. I chose oversampling (600dpi) for 300 dpi target. But this is not
for a book, so print quality (=unlimited zoom and vector where possible)
is not essential.

I'd take part in any effort to make good PS output solution. I'll also
share my bitmap map generator once it is good enough.

What does 'good enough' means: I'd like to make it read workspace files
since compositions are already there, so no double effort is necessary.
Input will be: workspace file, target layout (landscape A4 for example) a
saved region and a dpi value. Options and gui later...

It will create one or more BMP fileas at a given resolution also applying
opacity as set in the workspace file and then merge them into one file. At
this point I use imagemagick. It is available on all platforms as far as I
know.

Anyway, I wanted to add one vote for PS output enhancement...

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3033#comment:8>
GRASS GIS <https://grass.osgeo.org>

GDAL · August 30, 2016, 10:20am

#3033: Cairo and PS drivers display only one raster or vector for SVG and PS
-------------------------+-------------------------------------------------
  Reporter: wenzeslaus | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: major | Milestone: 7.2.0
Component: Display | Version: svn-trunk
Resolution: | Keywords: d.mon, cairo, ps, SVG, vector
       CPU: | graphics
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by mlennert):

Is this really a major bug, or rather an enhancement wish ?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3033#comment:9>
GRASS GIS <https://grass.osgeo.org>

GDAL · August 30, 2016, 2:49pm

#3033: Cairo and PS drivers display only one raster or vector for SVG and PS
-------------------------+-------------------------------------------------
  Reporter: wenzeslaus | Owner: grass-dev@…
      Type: | Status: new
  enhancement |
  Priority: major | Milestone: 7.2.0
Component: Display | Version: svn-trunk
Resolution: | Keywords: d.mon, cairo, ps, SVG, vector
       CPU: | graphics
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------
Changes (by wenzeslaus):

* type: defect => enhancement

Comment:

All things considered this is an enhancement ticket.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3033#comment:10>
GRASS GIS <https://grass.osgeo.org>

GDAL · August 30, 2016, 2:56pm

#3033: Cairo and PS drivers display only one raster or vector for SVG and PS
-------------------------+-------------------------------------------------
  Reporter: wenzeslaus | Owner: grass-dev@…
      Type: | Status: new
  enhancement |
  Priority: major | Milestone: 7.2.0
Component: Display | Version: svn-trunk
Resolution: | Keywords: d.mon, cairo, ps, SVG, vector
       CPU: | graphics
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by wenzeslaus):

For people intersted in alternatives:

Replying to [comment:8 kuszinger]:
> Starting from the utf-8 ps.map problem ... scripts which runs well and
generates bitmap. I chose oversampling (600dpi) for 300 dpi target. ...
I'll also share my bitmap map generator once it is good enough. ... read
workspace files since compositions are already there, so no double effort
is necessary. Input will be: workspace file, target layout (landscape A4
for example) a saved region and a dpi value. ... BMP files at a given
resolution also applying opacity as set in the workspace file and then
merge them into one file. At this point I use ImageMagick.

This is now available as `m.printws` in GRASS Addons repository thanks to
kuszinger:

* https://grass.osgeo.org/grass70/manuals/addons/m.printws.html

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3033#comment:11>
GRASS GIS <https://grass.osgeo.org>

GDAL · September 8, 2016, 3:50am

#3033: Cairo and PS drivers display only one raster or vector for SVG and PS
-------------------------+-------------------------------------------------
  Reporter: wenzeslaus | Owner: grass-dev@…
      Type: | Status: new
  enhancement |
  Priority: major | Milestone: 7.4.0
Component: Display | Version: svn-trunk
Resolution: | Keywords: d.mon, cairo, ps, SVG, vector
       CPU: | graphics
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------
Changes (by wenzeslaus):

* milestone: 7.2.0 => 7.4.0

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3033#comment:12>
GRASS GIS <https://grass.osgeo.org>