[GRASS-user] Problem importing ESRI grid file (.adf)

rshepard · June 8, 2018, 7:48pm

I have several LiDAR files from a state agency I want to import into
grass75. For one file I've used this command:

r.in.gdal -o --overwrite input=<path/to/directory/>Bare_Earth/bh45122c3/w001001.adf output=bare_earth

and get the output in the attached .png.

The Bare_Earth/ directory, as a compressed tarball, is 11.3M so I've put a
copy in the public directory of my dropbox account. If I've done this
correctly you can download a copy using this URL:
<https://www.dropbox.com/preview/Public/bare_earth.tgz?role=personal>

I want to learn what I'm doing incorrectly so I can get a LiDAR point
cloud for analyses.

Rich

(attachments)

Blumentrath_Stefan1 · June 8, 2018, 9:06pm

Hi Rich,

There should be a "dblbnd.adf" file in the ESRI grid directory. Please try that to import the data. So:

r.in.gdal -o --overwrite
input=<path/to/directory/>Bare_Earth/bh45122c3/dblbnd.adf output=bare_earth

Hope that helps.

Cheers
Stefan

-----Original Message-----
From: grass-user <grass-user-bounces@lists.osgeo.org> On Behalf Of Rich Shepard
Sent: fredag 8. juni 2018 21:49
To: grass-user@lists.osgeo.org
Subject: [GRASS-user] Problem importing ESRI grid file (.adf)

I have several LiDAR files from a state agency I want to import into grass75. For one file I've used this command:

r.in.gdal -o --overwrite
input=<path/to/directory/>Bare_Earth/bh45122c3/w001001.adf output=bare_earth

and get the output in the attached .png.

The Bare_Earth/ directory, as a compressed tarball, is 11.3M so I've put a copy in the public directory of my dropbox account. If I've done this correctly you can download a copy using this URL:
<https://www.dropbox.com/preview/Public/bare_earth.tgz?role=personal>

I want to learn what I'm doing incorrectly so I can get a LiDAR point cloud for analyses.

Rich

rshepard · June 8, 2018, 9:31pm

On Fri, 8 Jun 2018, Stefan Blumentrath wrote:

There should be a "dblbnd.adf" file in the ESRI grid directory. Please try
that to import the data. So:

Stefan,

I found an old web page that used the w001001.adf as it is supposed to
contain the topographic data. Regardless,

r.in.gdal -o --overwrite
input=<path/to/directory/>Bare_Earth/bh45122c3/dblbnd.adf output=bare_earth

This makes no difference in the output. Same display as on the .png
attached to my original message. Sigh.

Thanks,

Rich

hellik · June 9, 2018, 5:47am

Rich Shepard wrote

On Fri, 8 Jun 2018, Stefan Blumentrath wrote:

There should be a "dblbnd.adf" file in the ESRI grid directory. Please
try
that to import the data. So:

Stefan,

I found an old web page that used the w001001.adf as it is supposed to
contain the topographic data. Regardless,

r.in.gdal -o --overwrite
input=<path/to/directory/>Bare_Earth/bh45122c3/dblbnd.adf
output=bare_earth

This makes no difference in the output. Same display as on the .png
attached to my original message. Sigh.

Thanks,

Rich

_______________________________________________
grass-user mailing list

grass-user@.osgeo

https://lists.osgeo.org/mailman/listinfo/grass-user

Have you looked into GDAL's "AIG -- Arc/Info Binary Grid" format
description?

-----
best regards
Helmut
--
Sent from: http://osgeo-org.1560.x6.nabble.com/Grass-Users-f3884509.html

hellik · June 9, 2018, 6:02am

I want to learn what I'm doing incorrectly so I can get a LiDAR >point
cloud for analyses.

re-reading your original report.

ESRI grid file (.adf) isn't a format for point clouds, but, as the format
name tells, a processed grid of points.

you have to ask your data provider which dataset holds your raw point cloud.

-----
best regards
Helmut
--
Sent from: http://osgeo-org.1560.x6.nabble.com/Grass-Users-f3884509.html

rshepard · June 9, 2018, 2:19pm

On Fri, 8 Jun 2018, Helmut Kudrnovsky wrote:

re-reading your original report.

ESRI grid file (.adf) isn't a format for point clouds, but, as the format
name tells, a processed grid of points.

you have to ask your data provider which dataset holds your raw point cloud.

Helmut,

Yes, they'll sell me those data. But, it was point cloud data in the .gdb
format that they provided (for no charge) the LiDAR data for a different
county. That's why I expected it to be the same for these data.

Point data need to be converted to raster and vector contours for analyses
so they don't do me any good by themselves.

Best regards,

Rich

rshepard · June 9, 2018, 2:38pm

On Fri, 8 Jun 2018, Helmut Kudrnovsky wrote:

Have you looked into GDAL's "AIG -- Arc/Info Binary Grid" format
description?

Helmut,

I now have. Not much information (the format spec page didn't want to
load), but it appears to be what I need for the DEM: a raster with small
size cells, each having a geographic coordinate and elevation.

   The format is installed here:
$ gdalinfo --formats | sort
   AAIGrid -raster- (rwv): Arc/Info ASCII Grid
   ACE2 -raster- (rov): ACE2
   ADRG -raster- (rw+vs): ARC Digitized Raster Graphics
   AIG -raster- (rov): Arc/Info Binary Grid

The output is not what I expect; see attached .png. I'm open to all
suggstions and recommendations because I must be doing something incorrectly
here and do not see what that is.

The command line used is:

r.in.gdal -o --overwrite
input=/home/rshepard/.../data/topography/LDQ-45122C3/2014_OLC_Metro/Bare_Earth/bh45122c3/hdrbnd.adf output=bare_earth
(wrapped by alpine; it's entered as a single line).

That directory with all files should be downloadable from my dropbox
public directory: <https://www.dropbox.com/preview/Public/bare_earth.tgz?role=personal>

Regards,

Rich

hellik · June 9, 2018, 4:28pm

Rich Shepard wrote

On Fri, 8 Jun 2018, Helmut Kudrnovsky wrote:

Have you looked into GDAL's "AIG -- Arc/Info Binary Grid" format
description?

Helmut,

   I now have. Not much information (the format spec page didn't want to
load), but it appears to be what I need for the DEM: a raster with small
size cells, each having a geographic coordinate and elevation.

   The format is installed here:
$ gdalinfo --formats | sort
   AAIGrid -raster- (rwv): Arc/Info ASCII Grid
   ACE2 -raster- (rov): ACE2
   ADRG -raster- (rw+vs): ARC Digitized Raster Graphics
   AIG -raster- (rov): Arc/Info Binary Grid

   The output is not what I expect; see attached .png. I'm open to all
suggstions and recommendations because I must be doing something
incorrectly
here and do not see what that is.

   The command line used is:

r.in.gdal -o --overwrite
input=/home/rshepard/.../data/topography/LDQ-45122C3/2014_OLC_Metro/Bare_Earth/bh45122c3/hdrbnd.adf
output=bare_earth
(wrapped by alpine; it's entered as a single line).

   That directory with all files should be downloadable from my dropbox
public directory:
<https://www.dropbox.com/preview/Public/bare_earth.tgz?role=personal>

Regards,

Rich
_______________________________________________
grass-user mailing list

grass-user@.osgeo

https://lists.osgeo.org/mailman/listinfo/grass-user

As always recommended, look at the output of gdalinfo, compare it with the
computional region of your interest and then decide if the data expects your
need.

-----
best regards
Helmut
--
Sent from: http://osgeo-org.1560.x6.nabble.com/Grass-Users-f3884509.html

rshepard · June 9, 2018, 5:42pm

On Sat, 9 Jun 2018, Helmut Kudrnovsky wrote:

As always recommended, look at the output of gdalinfo, compare it with the
computional region of your interest and then decide if the data expects
your need.

Helmut,

I don't understand what you're suggesting. The coverage of the LiDAR data
is for a specific 1:24000 topographic quad map which contains the project
area of interest. When I import the data (or try to, anyway) I use the '-o'
option to set the computational region to that of the data being imported.

The LiDAR coordinator for the agency wrote that, "The shapefile is
project[sic] in EPSG 2995 - NAD83(2011) Oregon Lambert Conformal Conic."

When I run 'gdalinfo --stats' on the Bare_Earth subdirectory I see this:

Driver: AIG/Arc/Info Binary Grid
Files: bh45122c3
        bh45122c3.ovr
        bh45122c3.aux.xml
        bh45122c3/hdr.adf
        bh45122c3/dblbnd.adf
        bh45122c3/sta.adf
        bh45122c3/w001001.adf
        bh45122c3/metadata.xml
        bh45122c3/w001001x.adf
        bh45122c3/prj.adf
Size is 11052, 15426
Coordinate System is:
PROJCS["unnamed",
     GEOGCS["Unknown datum based upon the GRS 1980 ellipsoid",
         DATUM["Not_specified_based_on_GRS_1980_ellipsoid",
             SPHEROID["GRS 1980",6378137,298.257222101,
                 AUTHORITY["EPSG","7019"]],
             AUTHORITY["EPSG","6019"]],
         PRIMEM["Greenwich",0,
             AUTHORITY["EPSG","8901"]],
         UNIT["degree",0.0174532925199433,
             AUTHORITY["EPSG","9122"]],
         AUTHORITY["EPSG","4019"]],
     PROJECTION["Lambert_Conformal_Conic_2SP"],
     PARAMETER["standard_parallel_1",43],
     PARAMETER["standard_parallel_2",45.5],
     PARAMETER["latitude_of_origin",41.75],
     PARAMETER["central_meridian",-120.5],
     PARAMETER["false_easting",1312335.958005249],
     PARAMETER["false_northing",0],
     UNIT["user-defined",0.3048]]
Origin = (829504.500000000000000,1326862.500000000000000)
Pixel Size = (3.000000000000000,-3.000000000000000)
Metadata:
   DataType=Generic
Corner Coordinates:
Upper Left ( 829504.500, 1326862.500) (122d22'44.53"W, 45d22'29.77"N)
Lower Left ( 829504.500, 1280584.500) (122d22'29.72"W, 45d14'52.95"N)
Upper Right ( 862660.500, 1326862.500) (122d15' 0.15"W, 45d22'37.00"N)
Lower Right ( 862660.500, 1280584.500) (122d14'46.37"W, 45d15' 0.16"N)
Center ( 846082.500, 1303723.500) (122d18'45.19"W, 45d18'45.04"N)
Band 1 Block=256x16 Type=Float32, ColorInterp=Undefined
   Min=206.380 Max=1157.470
   Minimum=206.380, Maximum=1157.470, Mean=526.722, StdDev=308.445
   NoData Value=-3.4028234663852886e+38
   Overviews: 5526x7713, 2763x3857, 1382x1929, 691x965, 346x483, 173x242
   Metadata:
     STATISTICS_MAXIMUM=1157.4699707031
     STATISTICS_MEAN=526.72212274996
     STATISTICS_MINIMUM=206.38000488281
     STATISTICS_STDDEV=308.44451906832

This looks OK to me and does not help me understand why my r.in.gdal
command line does not produce a rectangular map with the elevations colored
by the default scheme.

Regards,

Rich

neteler · June 9, 2018, 10:01pm

On Sat, Jun 9, 2018 at 4:38 PM, Rich Shepard <rshepard@appl-ecosys.com> wrote:
...

That directory with all files should be downloadable from my dropbox
public directory:
<https://www.dropbox.com/preview/Public/bare_earth.tgz?role=personal>

That link lacks your personal ID (or whatever personal part or hash) -
nothing found. Can you please post the right link?

Markus

rshepard · June 10, 2018, 1:44am

On Sun, 10 Jun 2018, Markus Neteler wrote:

That link lacks your personal ID (or whatever personal part or hash) -
nothing found. Can you please post the right link?

Markus,

I give up on dropbox. I use it very infrequently and each time it rejects
my password when I sign in. Yeterday I 'reset' the password to what it had
been before and today dropbox again tells me that's not a correct password.
Feh!

Anyway, this should be a link to download the file:
<http://www.filedropper.com/bareearth>\.

Rich

neteler · June 10, 2018, 12:18pm

Rich,

On Sun, Jun 10, 2018 at 3:44 AM, Rich Shepard <rshepard@appl-ecosys.com> wrote:
....

Anyway, this should be a link to download the file:
<http://www.filedropper.com/bareearth>\.

The file size of the uploaded bare_earth.tgz is 13,927,771 bytes.
Unfortunately the file is incomplete:

tar xvf ~/Downloads/bare_earth.tgz
Bare_Earth/
Bare_Earth/bh45122c3.ovr
Bare_Earth/bare_earth.tgz
Bare_Earth/info/
Bare_Earth/info/arc0001.nit
Bare_Earth/info/arc0001.dat
Bare_Earth/info/arc.dir
Bare_Earth/info/arc0000.nit
Bare_Earth/info/arc0000.dat
Bare_Earth/bh45122c3.aux.xml
Bare_Earth/bh45122c3/
Bare_Earth/bh45122c3/hdr.adf
Bare_Earth/bh45122c3/dblbnd.adf
Bare_Earth/bh45122c3/sta.adf
Bare_Earth/bh45122c3/w001001.adf
tar: Unexpected EOF in archive
tar: rmtlseek not stopped at a record boundary
tar: Error is not recoverable: exiting now

Can you pls check?

Markus

rshepard · June 10, 2018, 1:21pm

On Sun, 10 Jun 2018, Markus Neteler wrote:

Rich,

The file size of the uploaded bare_earth.tgz is 13,927,771 bytes.
Unfortunately the file is incomplete:

Bare_Earth/bare_earth.tgz

Markus,

My fault: I included the tarball in the overall tarball. I've re-done it
more carefully: the size is 11356447 bytes and the URL is:
<http://www.filedropper.com/bareearth_1>\. (I don't see a way for me to
delete the original tarball; perhaps it's removed at the end of the month.)

Regards,

Rich

rshepard · June 10, 2018, 2:18pm

On Sun, 10 Jun 2018, Micha Silver wrote:

Hi Rich:

Still seems to be a problem (is the tar file zipped??): micha@RMS Downloads $ ll bareearth.tgz
-rw-r--r-- 1 micha micha 11356447 Jun 10 16:31 bareearth.tgz

Micha,

I wonder what happened in the transfer. Yes, the file is gzipped (per the
.tgz extension). What I have here is this:

$ tar tzvf bareearth.tgz drwxr-xr-x rshepard/users 0 2018-06-08 12:23 Bare_Earth/
-rw-r--r-- rshepard/users 3057617 2015-08-31 11:45 Bare_Earth/bh45122c3.ovr
drwxr-xr-x rshepard/users 0 2016-11-28 17:45 Bare_Earth/info/
-rw-r--r-- rshepard/users 576 2015-08-31 11:45 Bare_Earth/info/arc0001.nit
-rw-r--r-- rshepard/users 80 2015-08-31 11:45 Bare_Earth/info/arc0001.dat
-rw-r--r-- rshepard/users 760 2015-08-31 11:45 Bare_Earth/info/arc.dir
-rw-r--r-- rshepard/users 576 2015-08-31 11:45 Bare_Earth/info/arc0000.nit
-rw-r--r-- rshepard/users 80 2015-08-31 11:45 Bare_Earth/info/arc0000.dat
-rw-r--r-- rshepard/users 2577 2018-06-09 10:33 Bare_Earth/bh45122c3.aux.xml
drwxr-xr-x rshepard/users 0 2016-11-28 17:45 Bare_Earth/bh45122c3/
-rw-r--r-- rshepard/users 308 2015-08-31 11:45 Bare_Earth/bh45122c3/hdr.adf
-rw-r--r-- rshepard/users 32 2015-08-31 11:45 Bare_Earth/bh45122c3/dblbnd.adf
-rw-r--r-- rshepard/users 32 2015-08-31 11:45 Bare_Earth/bh45122c3/sta.adf
-rw-r--r-- rshepard/users 695749660 2015-08-31 11:45 Bare_Earth/bh45122c3/w001001.adf
-rw-r--r-- rshepard/users 15503 2015-08-31 11:45 Bare_Earth/bh45122c3/metadata.xml
-rw-r--r-- rshepard/users 339780 2015-08-31 11:45 Bare_Earth/bh45122c3/w001001x.adf
-rw-r--r-- rshepard/users 365 2015-12-29 16:20 Bare_Earth/bh45122c3/prj.adf

Let me try two things: 1) explicitly separating tar and gzip and 2) a
different 'cloud' provider.

Rich

Markus_Metz · June 14, 2018, 7:17pm

On Fri, Jun 8, 2018 at 9:48 PM, Rich Shepard <rshepard@appl-ecosys.com> wrote:

I have several LiDAR files from a state agency I want to import into
grass75. For one file I’ve used this command:

r.in.gdal -o --overwrite input=<path/to/directory/>Bare_Earth/bh45122c3/w001001.adf output=bare_earth

and get the output in the attached .png.

The Bare_Earth/ directory, as a compressed tarball, is 11.3M so I’ve put a
copy in the public directory of my dropbox account. If I’ve done this
correctly you can download a copy using this URL:
<https://www.dropbox.com/preview/Public/bare_earth.tgz?role=personal>

I want to learn what I’m doing incorrectly so I can get a LiDAR point

cloud for analyses.

The only thing that is not correct is the -o flag, otherwise everything is correct and also the result is correct. You need to import all bare earth data for 45122c3 from different years and mosaic them together.

Markus M

Rich

grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user