Hi,
there seems to be missing a function which reports the presence
of NULL data in a raster map. This functions is needed for
interfacing GRASS to GDAL, especially for 8 bit data.
At time the entire map must be scanned, right? This
can become pretty time consuming.
Is it sufficient to test the presence of the 'null' file
within this potential function? A bit dirty, but otherwise the
cellhd file needed an addition (null: true or whatever).
Suggestions (or code)?
Markus
Markus Neteler wrote:
there seems to be missing a function which reports the presence
of NULL data in a raster map. This functions is needed for
interfacing GRASS to GDAL, especially for 8 bit data.
At time the entire map must be scanned, right?
Right.
This can become pretty time consuming.
Probably; but that doesn't automatically make the alternatives
feasible.
Is it sufficient to test the presence of the 'null' file
within this potential function?
I don't know.
If there isn't a null file, and it isn't a reclass map, and there
isn't a mask, and the current region doesn't extend outside the map's
boundaries, then I don't think that there can be any null cells.
OTOH, even if there is a null file, it is presumably still possible
that none of the cells are actually null.
In any case, we should arguably be getting rid of the null file and
just storing null values in the cell file. ISTR that the use of a
separate null bitmap was identified as a significant performance hit.
Iis this for GDAL's --with-grass functionality, or for r.out.gdal? If
it's the latter, it definitely *should* be honouring the mask and
region. For the former, you can presumably ignore the mask and region,
although there may still be the issue of reclass maps (a reclass map
won't have its own null file, and may have null cells even if the
underlying map doesn't).
A bit dirty, but otherwise the
cellhd file needed an addition (null: true or whatever).
Or you just accept the cost of scanning for nulls, or you assume that
all maps conceptually have null cells, even if, for some maps, the
number of null cells happens to be zero.
Regarding the approach of adding a null flag to the cellhd file, I
suspect that it may be harder to get right than it might initially
appear. I.e. you could spend the next few years occasionally bumping
into cases where the flag ends up out of sync with reality.
--
Glynn Clements <glynn.clements@virgin.net>
On Friday 02 July 2004 23:38, Markus Neteler wrote:
Hi,
there seems to be missing a function which reports the presence
of NULL data in a raster map. This functions is needed for
interfacing GRASS to GDAL, especially for 8 bit data.
At time the entire map must be scanned, right?
Yes
This can become pretty time consuming.
Is it sufficient to test the presence of the 'null' file
within this potential function?
Following the developer's manual yes.
BTW, the null file could be scanned to contain only null-bytes. It's 8 times
less time consuming... 
A bit dirty, but otherwise the
cellhd file needed an addition (null: true or whatever).Suggestions (or code)?
I think the best thing would be to check existence of null file. But I don't
know exaclty which changes that would need... hmm...
Cheers
Andrea
Markus
_______________________________________________
grass5 mailing list
grass5@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass5
--
____________________________________________________________________________
"Let it be as much a great honour to take as to give learning,
if you want to be called wise."
Skuggsja' - The King's mirror - 1240 Reykjavik
University of Trento
Department of Civil and Environmental Engineering
Via Mesiano, 77 - Trento (ITALY)
Andrea Antonello
tel: +393288497722
fax: +390461882672
____________________________________________________________________________
there seems to be missing a function which reports the presence
of NULL data in a raster map. This functions is needed for
interfacing GRASS to GDAL, especially for 8 bit data.
At time the entire map must be scanned, right? This
can become pretty time consuming.Is it sufficient to test the presence of the 'null' file
within this potential function? A bit dirty, but otherwise the
cellhd file needed an addition (null: true or whatever).Suggestions (or code)?
It would be easy to add a flag to continue; the loop in r.univar(2) at
line 120 just after the n++; to quickly report a count of total cells
vs. number of null cells in a map for a compare..
Hamish
On Sat, Jul 03, 2004 at 12:02:29AM +0100, Glynn Clements wrote:
Markus Neteler wrote:
> there seems to be missing a function which reports the presence
> of NULL data in a raster map. This functions is needed for
> interfacing GRASS to GDAL, especially for 8 bit data.
> At time the entire map must be scanned, right?Right.
> This can become pretty time consuming.
Probably; but that doesn't automatically make the alternatives
feasible.> Is it sufficient to test the presence of the 'null' file
> within this potential function?I don't know.
If there isn't a null file, and it isn't a reclass map, and there
isn't a mask, and the current region doesn't extend outside the map's
boundaries, then I don't think that there can be any null cells.OTOH, even if there is a null file, it is presumably still possible
that none of the cells are actually null.In any case, we should arguably be getting rid of the null file and
just storing null values in the cell file. ISTR that the use of a
separate null bitmap was identified as a significant performance hit.
This was proposed a long time ago, I can't remember why the change
wasn't done. Maybe the person lost track on the idea.
Iis this for GDAL's --with-grass functionality, or for r.out.gdal?
Currently it's for GDAL's --with-grass functionality.
The r.out.gdal script is just a workaround unless someone is willing
to contribute a C version.
If it's the latter, it definitely *should* be honouring the mask and
region.
It should, but it will do only in a C version.
For the former, you can presumably ignore the mask and region,
although there may still be the issue of reclass maps (a reclass map
won't have its own null file, and may have null cells even if the
underlying map doesn't).> A bit dirty, but otherwise the
> cellhd file needed an addition (null: true or whatever).Or you just accept the cost of scanning for nulls, or you assume that
all maps conceptually have null cells, even if, for some maps, the
number of null cells happens to be zero.
Scanning all maps first is practically a problem. Imagine all
QGIS users waiting such long for every GRASS raster map...
That's why we should find a better way.
Regarding the approach of adding a null flag to the cellhd file, I
suspect that it may be harder to get right than it might initially
appear. I.e. you could spend the next few years occasionally bumping
into cases where the flag ends up out of sync with reality.
Embedding the NULL values seems to be the best solution.
Would it be complicated (in fact it might simplify the GRASS
raster lib)?
Markus
Markus Neteler wrote:
> In any case, we should arguably be getting rid of the null file and
> just storing null values in the cell file. ISTR that the use of a
> separate null bitmap was identified as a significant performance hit.This was proposed a long time ago, I can't remember why the change
wasn't done. Maybe the person lost track on the idea.
I suspect that nobody is confident that they fully understand the
implications of doing so.
> > A bit dirty, but otherwise the
> > cellhd file needed an addition (null: true or whatever).
>
> Or you just accept the cost of scanning for nulls, or you assume that
> all maps conceptually have null cells, even if, for some maps, the
> number of null cells happens to be zero.Scanning all maps first is practically a problem. Imagine all
QGIS users waiting such long for every GRASS raster map...
That's why we should find a better way.
What about the other option, i.e. just treating every map as if it
does have null cells?
> Regarding the approach of adding a null flag to the cellhd file, I
> suspect that it may be harder to get right than it might initially
> appear. I.e. you could spend the next few years occasionally bumping
> into cases where the flag ends up out of sync with reality.Embedding the NULL values seems to be the best solution.
There may be a misunderstanding here. I didn't propose embedding the
nulls as a solution for this issue; I'm not sure how it would help. I
mentioned it as something which might happen for other reasons; in
which case, the proposed technique of testing for the presence of a
null file wouldn't work.
Would it be complicated (in fact it might simplify the GRASS
raster lib)?
One unknown (for me) is whether XDR supports NaNs. The integer case
should be straightforward (AFAICT, we can use minus zero for null).
It would certainly simplify the library.
OTOH, there are a fair number of other things which could reasonably
be done to the raster library. IMHO, we should discuss all the issues
then re-write the raster I/O code from scratch.
Also, I do note that the files:
src/libes/ogsf/Gs3.c
src/raster/r.le/r.le.pixel/driver.c
src/raster/r.le/r.le.pixel/cellclip.c
use G_get_null_value_row(). I have no idea why; application code
should probably just read the data and use G_is_null_value() etc.
--
Glynn Clements <glynn.clements@virgin.net>