[GRASS5] floating AI ASCII GRID import

Hi

It's on grass57_exp_2004_11_13 but the problem was in all older Grass versions I used.

Problem: 'r.in.gdal' and 'r.in.arc type=FCELL' output a different raster than 'r.in.arc type=DCELL'. And the latter outputs a proper one.

I've got a following ArcInfo ASCIIGrid:

ncols 6101
nrows 3201
xllcorner 649.5
yllcorner 849.5
cellsize 1
NODATA_value -9999
280.2687 280.2776 280.2866 (...)

The value range should be:
min=176.945400
max=284.015600

And it is the case when 'r.in.arc type=DCELL' is used for importing.

However, if 'r.in.gdal' or 'r.in.arc type=FCELL' is used then:

min=176.945404
max=284.015594

I suppose there should be no difference. What is the reason?

BTW - I couldn't find a GUI entry for the 'r.in.arc' in the File->Import->Raster. Shouldn't it be there?

Maciek

Maciek Sieczka wrote:

The value range should be:
min=176.945400
max=284.015600

And it is the case when 'r.in.arc type=DCELL' is used for importing.

However, if 'r.in.gdal' or 'r.in.arc type=FCELL' is used then:

min=176.945404
max=284.015594

I suppose there should be no difference. What is the reason?

Maciek,

Well, the obvious conclusion is that a FCELL, which I presume reads as a
4byte float gives
slightly less precision than DCELL which is an 8byte float. The change
in the seventh digit would
seem to be what we would expect from floating point precision limitations.

Best regards,

--

---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent

From: "Frank Warmerdam" <fwarmerdam@gmail.com>

The value range should be:
min=176.945400
max=284.015600

And it is the case when 'r.in.arc type=DCELL' is used for importing.

However, if 'r.in.gdal' or 'r.in.arc type=FCELL' is used then:

min=176.945404
max=284.015594

I suppose there should be no difference. What is the reason?

Well, the obvious conclusion is that a FCELL, which I presume reads as a
4byte float gives
slightly less precision than DCELL which is an 8byte float. The change
in the seventh digit would seem to be what we would expect from floating point precision limitations.

Ok, so why doesn't r.in.gdal recognize what precission the input requires? Actually, all floating point AI ASCII GRIDs I met were DCELL, no matter where they came from. So maybe the default for the r.in.gdal in case of the AI ASCII GRID should be DCELL instead of FCELL?

GRASS DEVELOPERS - I couldn't find a GUI entry for the 'r.in.arc' in the File->Import->Raster. Shouldn't it be there?

Maciek

On 11/27/04 1:51 PM, "Maciek Sieczka" <werchowyna@pf.pl> wrote:

AI ASCII GRID should be DCELL instead of FCELL?

GRASS DEVELOPERS - I couldn't find a GUI entry for the 'r.in.arc' in the
File->Import->Raster. Shouldn't it be there?

Maciek

I need to know that it has been added to the available modules in order to
add it to the menus. Thanks for letting me know. I'll add it to the CVS
version soon.

On 11/27/04 2:03 PM, "Maciek Sieczka" <werchowyna@pf.pl> wrote:

grass57_exp_2004_11_13

When I choose the "v.surf.rst" from the GUI it doesn't pop up but prints in
the terminal:

ERROR: Required parameter <input> not set:
    (Name of the vector file with input data)

Anybody having the same? Give it a bug report?

Maciek Sieczka

I just tried this this weekend and noticed the same behavior. Seems like
I've changed the menu entry for this command a couple of times to deal with
odd behavior. Maybe I can now change it back to the original 'execute'
version.

Michael Barton

____________________
C. Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
PO Box 872402
Arizona State University
Tempe, AZ 85287-2402
USA

Phone: 480-965-6262
Fax: 480-965-7671
www: <www.public.asu.edu/~cmbarton>

On Sat, Nov 27, 2004 at 09:51:55PM +0100, Maciek Sieczka wrote:
...

GRASS DEVELOPERS - I couldn't find a GUI entry for the 'r.in.arc' in the
File->Import->Raster. Shouldn't it be there?

As r.in.gdal should do the job, there is no need to maintain
an extra command for that.

Markus

Maciek,

I just had a moment to check on r.in.arc and it is already in the GRASS 5.7
menus (25 Nov. release), under file>import>raster>ESRI Arc/Info ASCII grid

On 11/27/04 1:51 PM, "Maciek Sieczka" <werchowyna@pf.pl> wrote:

AI ASCII GRID should be DCELL instead of FCELL?

GRASS DEVELOPERS - I couldn't find a GUI entry for the 'r.in.arc' in the
File->Import->Raster. Shouldn't it be there?

Maciek

Michael Barton

____________________
C. Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
PO Box 872402
Arizona State University
Tempe, AZ 85287-2402
USA

Phone: 480-965-6262
Fax: 480-965-7671
www: <www.public.asu.edu/~cmbarton>

On Sun, Nov 28, 2004 at 12:11:52AM -0700, Michael Barton wrote:

On 11/27/04 1:51 PM, "Maciek Sieczka" <werchowyna@pf.pl> wrote:

> AI ASCII GRID should be DCELL instead of FCELL?
>
> GRASS DEVELOPERS - I couldn't find a GUI entry for the 'r.in.arc' in the
> File->Import->Raster. Shouldn't it be there?
>
> Maciek
>

I need to know that it has been added to the available modules in order to
add it to the menus. Thanks for letting me know. I'll add it to the CVS
version soon.

I wasn't aware that it is present in 5.7.

Still I vote for removal of 'r.in.arc' as 'r.in.gdal' should do the same
(right?).
Replicated functionality should be minimized IMHO.

Markus

I haven’t had a chance to test this to see if r.in.gdal is the same as r.in.arc. If it is, I agree completely that we need eliminate redundancy where possible. Perhaps Maciek can test it as he seems to have a set of AI ASCII grid files he is working with now. Let me know if it is redundant and I’ll take it off the input menu.

It would be nice on both GDAL and OGR to hit a button and get a list of readable formats. Since GDAL at least generates such a list, this should be doable.

Cheers
Michael

On 11/29/04 2:18 AM, “Markus Neteler” neteler@itc.it wrote:

On Sun, Nov 28, 2004 at 12:11:52AM -0700, Michael Barton wrote:

On 11/27/04 1:51 PM, “Maciek Sieczka” werchowyna@pf.pl wrote:

AI ASCII GRID should be DCELL instead of FCELL?

GRASS DEVELOPERS - I couldn’t find a GUI entry for the ‘r.in.arc’ in the
File->Import->Raster. Shouldn’t it be there?

Maciek

I need to know that it has been added to the available modules in order to
add it to the menus. Thanks for letting me know. I’ll add it to the CVS
version soon.

I wasn’t aware that it is present in 5.7.

Still I vote for removal of ‘r.in.arc’ as ‘r.in.gdal’ should do the same
(right?).
Replicated functionality should be minimized IMHO.

Markus


Michael Barton, Professor of Anthropology
School of Human, Evolution and Social Change
Arizona State University
Tempe, AZ 85287-2402
USA

voice: 480-965-6262; fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: "Michael Barton" <michael.barton@asu.edu>
Sent: Monday, November 29, 2004 5:56 PM
Subject: Re: [GRASS5] floating AI ASCII GRID import

I haven¹t had a chance to test this to see if r.in.gdal is the same as
r.in.arc. If it is, I agree completely that we need eliminate redundancy
where possible. Perhaps Maciek can test it as he seems to have a set of AI
ASCII grid files he is working with now. Let me know if it is redundant
and
I¹ll take it off the input menu.

The problem is that GDAL may round the floating AI ASCII GRID to integer.
Below is what Frank Warmerdam once wrote on the gdal-list. Frank, is it
still not fixed? BTW - what is the chance that GDAL could be thought to recognize if the AI AGRID is FCELL or DCELL?

From: "Frank Warmerdam" <warmerdam@pobox.com>
Sent: Tuesday, June 01, 2004 2:59 PM
Subject: Re: [Gdal-dev] floating point bug

This is already a known issue in bug:
http://bugzilla.remotesensing.org/show_bug.cgi?id=313
Basically I scan the first 1K or so of the file to look for floating point
values.
If I don't find any I assume the whole coverage is integer.
I have been hesitant to "fix" this as a proper fix would seem to require
scanning the whole file on open which is pretty expensive.

Maciek

Maciek Sieczka wrote:

From: "Michael Barton" <michael.barton@asu.edu>
Sent: Monday, November 29, 2004 5:56 PM
Subject: Re: [GRASS5] floating AI ASCII GRID import

I haven¹t had a chance to test this to see if r.in.gdal is the same as
r.in.arc. If it is, I agree completely that we need eliminate redundancy
where possible. Perhaps Maciek can test it as he seems to have a set of AI
ASCII grid files he is working with now. Let me know if it is redundant
and
I¹ll take it off the input menu.

The problem is that GDAL may round the floating AI ASCII GRID to integer.
Below is what Frank Warmerdam once wrote on the gdal-list. Frank, is it
still not fixed? BTW - what is the chance that GDAL could be thought to recognize if the AI AGRID is FCELL or DCELL?

From: "Frank Warmerdam" <warmerdam@pobox.com>
Sent: Tuesday, June 01, 2004 2:59 PM
Subject: Re: [Gdal-dev] floating point bug

This is already a known issue in bug:
http://bugzilla.remotesensing.org/show_bug.cgi?id=313
Basically I scan the first 1K or so of the file to look for floating point
values.
If I don't find any I assume the whole coverage is integer.
I have been hesitant to "fix" this as a proper fix would seem to require
scanning the whole file on open which is pretty expensive.

Maciek,

This is still an open issue.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent

Frank,

I happen to work with ascii grid files right now and I have a situation
where I have both integer and FP values, the integer being -9999 for no data.
I tried it with r.in.gdal and indeed it imported it as integer (which for DEMs
can be a serious problem). I would guess that this may be the most common
situation when users get it imported as integer rather than FP (besides manually
changing some of the data) - I think that this could be fixed relatively
easily if the integer value for nodata defined in the header is skipped.
If this cannot be fixed we should keep r.in.arc

Helena.

Frank Warmerdam wrote:

Maciek Sieczka wrote:

From: "Michael Barton" <michael.barton@asu.edu>
Sent: Monday, November 29, 2004 5:56 PM
Subject: Re: [GRASS5] floating AI ASCII GRID import

I haven¹t had a chance to test this to see if r.in.gdal is the same as
r.in.arc. If it is, I agree completely that we need eliminate redundancy
where possible. Perhaps Maciek can test it as he seems to have a set of AI
ASCII grid files he is working with now. Let me know if it is redundant
and
I¹ll take it off the input menu.

The problem is that GDAL may round the floating AI ASCII GRID to integer.
Below is what Frank Warmerdam once wrote on the gdal-list. Frank, is it
still not fixed? BTW - what is the chance that GDAL could be thought to recognize if the AI AGRID is FCELL or DCELL?

From: "Frank Warmerdam" <warmerdam@pobox.com>
Sent: Tuesday, June 01, 2004 2:59 PM
Subject: Re: [Gdal-dev] floating point bug

This is already a known issue in bug:
http://bugzilla.remotesensing.org/show_bug.cgi?id=313
Basically I scan the first 1K or so of the file to look for floating point
values.
If I don't find any I assume the whole coverage is integer.
I have been hesitant to "fix" this as a proper fix would seem to require
scanning the whole file on open which is pretty expensive.

Maciek,

This is still an open issue.

Best regards,

It would be nice on both GDAL and OGR to hit a button and get a list
of readable formats. Since GDAL at least generates such a list, this
should be doable.

A year ago I added support (locally) to r.in.gdal for a -l flag to list
available formats and exit. This was mostly just cut-and-paste C code
from 'gdalinfo --formats'. I never put it in CVS, can do if people want.
v.in.ogr should be a similar exercise. 'gdal-config --formats' is another
flavour.

v.in.ogr already has a -l flag for listing available layers.

Maybe -f for both? (for "list Formats")

comments?
Hamish

Hamish wrote:

It would be nice on both GDAL and OGR to hit a button and get a list
of readable formats. Since GDAL at least generates such a list, this
should be doable.

A year ago I added support (locally) to r.in.gdal for a -l flag to list
available formats and exit. This was mostly just cut-and-paste C code
from 'gdalinfo --formats'. I never put it in CVS, can do if people want.
v.in.ogr should be a similar exercise. 'gdal-config --formats' is another
flavour.

v.in.ogr already has a -l flag for listing available layers.

Maybe -f for both? (for "list Formats")

it would be very nice if you could add such a flag,I miss it too.
  -f looks good to me,

Helena

comments?
Hamish

_______________________________________________
grass5 mailing list
grass5@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass5

Hamish,

Personally, I think it is a very useful idea and would like to see it
implemented.

Michael

On 11/29/04 10:03 PM, "Hamish" <hamish_nospam@yahoo.com> wrote:

It would be nice on both GDAL and OGR to hit a button and get a list
of readable formats. Since GDAL at least generates such a list, this
should be doable.

A year ago I added support (locally) to r.in.gdal for a -l flag to list
available formats and exit. This was mostly just cut-and-paste C code
from 'gdalinfo --formats'. I never put it in CVS, can do if people want.
v.in.ogr should be a similar exercise. 'gdal-config --formats' is another
flavour.

v.in.ogr already has a -l flag for listing available layers.

Maybe -f for both? (for "list Formats")

comments?
Hamish

____________________
C. Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
PO Box 872402
Arizona State University
Tempe, AZ 85287-2402
USA

Phone: 480-965-6262
Fax: 480-965-7671
www: <www.public.asu.edu/~cmbarton>

>>It would be nice on both GDAL and OGR to hit a button and get a list
>>of readable formats. Since GDAL at least generates such a list, this
>>should be doable.
>
> A year ago I added support (locally) to r.in.gdal for a -l flag to
> list available formats and exit. This was mostly just cut-and-paste
> C code from 'gdalinfo --formats'. I never put it in CVS, can do if
> people want. v.in.ogr should be a similar exercise. 'gdal-config
> --formats' is another flavour.
>
> v.in.ogr already has a -l flag for listing available layers.
>
> Maybe -f for both? (for "list Formats")

it would be very nice if you could add such a flag,I miss it too.
  -f looks good to me,

ok, done for both r.in.gdal and v.in.ogr.

r.in.gdal: code follows GDAL 1.2.5 gcore/gdal_misc.cpp
  Frank is the author of all this code, but this may mix licenses:
  -> I assume this does not constitute a "substantial portion" of
     gdal_misc.cpp and therefore it is ok? Frank?

v.in.ogr: not sure how to do the read/write check.
  [as these are both reading functions, reporting 'write' support
   doesn't really matter, but is interesting..]

Both modules have non-optional options (eg output=). You have to provide
dummy values to get past the parser. Is there a better way?

Hamish

On Thu, 2 Dec 2004 16:55:56 +1300, Hamish <hamish_nospam@yahoo.com> wrote:

ok, done for both r.in.gdal and v.in.ogr.

r.in.gdal: code follows GDAL 1.2.5 gcore/gdal_misc.cpp
  Frank is the author of all this code, but this may mix licenses:
  -> I assume this does not constitute a "substantial portion" of
     gdal_misc.cpp and therefore it is ok? Frank?

Hamish,

Cool, good work! I am happy to allow portions of gdal_misc.cpp
to be relicensed under the GPL when copied into GRASS. In
fact, I am happy to allow any amount of code from GDAL that I
hold the copyright on to be relicensed under the GPL if it would
be useful in GRASS, as long as the original copy in GDAL remains
under it's original license.

v.in.ogr: not sure how to do the read/write check.
  [as these are both reading functions, reporting 'write' support
   doesn't really matter, but is interesting..]

In C++ you would do a test like this to see if a driver supports
creating new datasources. I thnk the C equivelent is to call
OGR_Dr_TestCapability().

        if( !poDriver->TestCapability( ODrCCreateDataSource ) )
        {
            printf( "%s driver does not support data source creation.\n",
                    pszFormat );
            exit( 1 );
        }

Both modules have non-optional options (eg output=). You have to provide
dummy values to get past the parser. Is there a better way?

Ugg, this really sucks doesn't it?

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent

> v.in.ogr: not sure how to do the read/write check.
> [as these are both reading functions, reporting 'write' support
> doesn't really matter, but is interesting..]

In C++ you would do a test like this to see if a driver supports
creating new datasources. I thnk the C equivelent is to call
OGR_Dr_TestCapability().

        if( !poDriver->TestCapability( ODrCCreateDataSource ) )
        {
            printf( "%s driver does not support data source
            creation.\n",
                    pszFormat );
            exit( 1 );
        }

This is what I have so far:

  for(iDriver = 0; iDriver < OGRGetDriverCount(); iDriver++) {
      OGRSFDriverH *poDriver = OGRGetDriver(iDriver);
      fprintf(stdout, " %s\n", OGR_Dr_GetName(poDriver));

  /*** TODO: read/write check:
      if( OGR_Dr_TestCapability(poDriver, "??") )
    fprintf(stdout, " %s (read/write)\n",
        OGR_Dr_GetName(poDriver) );
      else
    fprintf(stdout, " %s (read only)\n",
        OGR_Dr_GetName(poDriver) );
  ***/

  }

What I don't understand is what *pszCap string to give
int CPL_DLL OGR_Dr_TestCapability( OGRSFDriverH, const char * );
as a test. (if that works at all)

Hamish