{{{
d.mon cairo
d.rast elevation
ERROR: Cairo_begin_raster(): Failed to create surface (invalid value
(typically too big) for the size of the input (surface, pattern, etc.)).
Using rows: 135000, cols: 150000
}}}
>
{{{
ERROR: Cairo_begin_raster(): Failed to create surface (invalid value
(typically too big) for the size of the input (surface, pattern, etc.)).
Using rows: 135000, cols: 150000
}}}
It would be possible to change the cairo driver (lib/cairodriver/Raster.c)
to render each row separately.
However, this would effectively preclude adding certain features (e.g.
anti-aliasing or rotation). It would also make a mess of the file
structure for formats such as SVG.
For those reasons, I'd suggest making row-by-row rendering an alternative
to the existing method rather than simply replacing it.
Replying to [comment:3 martinl]:
> Replying to [comment:2 glynn]:
> > It would be possible to change the cairo driver
(lib/cairodriver/Raster.c) to render each row separately.
>
> for record https://bugs.freedesktop.org/show_bug.cgi?id=85561
the problem is that cairo limits number of rows and cols to 32767 (1). Any
idea how this bug could be resolved in GRASS?
> the problem is that cairo limits number of rows and cols to 32767 (1).
Any idea how this bug could be resolved in GRASS?
One option is to require clients (d.rast, d.his, d.rgb) to limit the
region resolution. Another is to modify the driver(s) to decimate the data
if it exceeds certain limits. Yet another is to modify the cairo driver to
down-sample the data itself (this avoids the issue that a source region of
32767x32767 will require 4 GiB of memory to store the surface regardless
of the output resolution).
All of these have the advantage that the client won't waste time reading
rows which will be discarded by the driver (when down-sampling, the driver
reports the next row which will be used).
> another is to modify the cairo driver to down-sample the data itself
(this avoids the issue that a source region of 32767x32767 will require 4
GiB of memory to store the surface regardless of the output resolution).
I was thinking about something similar, btw, is there any display driver
in GRASS which already doing down-sampling? Thanks for any hint.
> I was thinking about something similar, btw, is there any display driver
in GRASS which already doing down-sampling? Thanks for any hint.
The PNG driver resamples each row and stores it directly in the image.
The PS driver writes the data to the PostScript file as-is (after
conversion to hex). This can potentially result in huge files, but it
doesn't affect memory requirements (PostScript "streams" image data into
the frame-buffer from the input file; it doesn't store it in memory prior
to rendering).
> The PNG driver resamples each row and stores it directly in the image.
Thanks for the hint. Could you please review
attachment:cairo_down_sampling.diff which do down-sampling for Cairo in
similar as already does PNG driver?
Down-sampling has of course a significant speed effect when rendering
larger rasters using their original region, eg.
> Could you please review attachment:cairo_down_sampling.diff which do
down-sampling for Cairo in similar as already does PNG driver?
The main issue was that it neglected the inner Y loop when up-scaling.
r63582 makes the cairo driver behave almost identically to the PNG driver.
For reasons which I've been unable to determine so far, the result is one
destination pixel wider than with the PNG driver.
Ultimately, the code should be modified to use an intermediate surface
which has the same dimensions as the destination rectangle, to avoid the
overhead of copying the (empty) borders.
> r63582 makes the cairo driver behave almost identically to the PNG
driver.
I took liberty to backport it in r63595.
> For reasons which I've been unable to determine so far, the result is
one destination pixel wider than with the PNG driver.
>
> Ultimately, the code should be modified to use an intermediate surface
which has the same dimensions as the destination rectangle, to avoid the
overhead of copying the (empty) borders.
Since the originally reported issue has been fixed, I took liberty to
close this ticket.
This (formerly failing) example was done over NFS mounted /grassdata over
a 1Gb/s intranet connection with local GRASS GIS installation and remote
network data repository:
GRASS 7.0.0svn (eu_laea):~ > time -p d.rast lst_2014_03_minimum
100%
real 34.12
user 33.13
sys 0.67
##### comparison to see how long data reading itself takes:
GRASS 7.0.0svn (eu_laea):~ > time -p r.stats lst_2014_03_minimum
...
real 12.45
user 12.26
sys 0.12
}}}
>
{{{
GRASS 7.0.0svn (eu_laea):~ > time -p d.rast lst_2014_03_minimum
##### comparison to see how long data reading itself takes:
GRASS 7.0.0svn (eu_laea):~ > time -p r.stats lst_2014_03_minimum
}}}
FWIW, the comparison isn't particularly meaningful, as d.rast now only
needs to read the rows which are actually rendered, while r.stats needs to
read the entire map.
Beyond the need to uncompress entire rows, the rendering time should be
dictated by the output width, not the input width. However, d.rast's
values= feature adds overhead proportional to the input width.
Can you try commenting out the mask_raster_array() call at line 71 of
display/d.rast/display.c? If that has a significant impact, we should try
to skip that call altogether when values= isn't being used.
Replying to [comment:12 glynn]:
> Can you try commenting out the mask_raster_array() call at line 71 of
display/d.rast/display.c? If that has a significant impact, we should try
to skip that call altogether when values= isn't being used.
Sure. Test setting: connected from home via ssh/tsocks tunnel over ADSL
connection. Opening GRASS 7.0.svn (r63585) remotely, tunnelling wx0 over
network:
{{{
rows: 18711
cols: 22195
cells: 415290645
# original
d.mon wx0
time -p d.rast europe_altitude_map.shade
100%
real 4.64
user 0.93
sys 0.03
# with commented line of mask_raster_array() call
d.mon wx0 # ... previously opened monitor closed
time -p d.rast europe_altitude_map.shade
100%
real 0.92
user 0.91
sys 0.01
}}}
Probably that's significant.
Note: This is the time when d.rast finishes and returns to the command
line. The display in d.mon wx0 comes many seconds later but that's of
course a different issue.
Replying to [comment:9 glynn]:
> r63582 makes the cairo driver behave almost identically to the PNG
driver.
>
> For reasons which I've been unable to determine so far, the result is
one destination pixel wider than with the PNG driver.
Not sure if related but here I remember ticket #72.
> Ultimately, the code should be modified to use an intermediate surface
which has the same dimensions as the destination rectangle, to avoid the
overhead of copying the (empty) borders.
> > Ultimately, the code should be modified to use an intermediate surface
which has the same dimensions as the destination rectangle, to avoid the
overhead of copying the (empty) borders.
>
> Should the ticket be reopened for this?
If so, please open a new ticket. It's hard to follow tickets with multiple
issues...