[GRASS-user] Unusual [Open|ESRI]FileGDB: how to import

rshepard · October 26, 2017, 8:52pm

I am trying to import/open data files from www.streamnet.org but their GDB
format is quite different from what I've seen before now and grass-7.3.svn
will not open them. As an aside, I was thrown for a while looking for
OpenFileGDB in the v.in.ogr supported format list until I noticed it's been
renamed to ESRIFileGDB.

The results of running 'ogrinfo -al so' on the directory,
BullTrout_201201_v931.gdb/, produces this result:

Had to open data source read-only.
INFO: Open of `BullTrout_201201_v931.gdb/'
using driver `OpenFileGDB' successful.

Layer name: FishD_BullTrout_January2012
Geometry: Measured Multi Line String
Feature Count: 4620
Extent: (1158336.501590, 288917.146174) - (4143404.913877, 3148258.748272)
Layer SRS WKT:
PROJCS["NAD_1983_Lambert_Conformal_Conic",
     GEOGCS["GCS_North_American_1983",
         DATUM["North_American_Datum_1983",
             SPHEROID["GRS_1980",6378137.0,298.257222101]],
         PRIMEM["Greenwich",0.0],
         UNIT["Degree",0.0174532925199433]],
     PROJECTION["Lambert_Conformal_Conic_2SP"],
     PARAMETER["False_Easting",2999999.999988],
     PARAMETER["False_Northing",0.0],
     PARAMETER["Central_Meridian",-117.0],
     PARAMETER["Standard_Parallel_1",42.33333333333334],
     PARAMETER["Standard_Parallel_2",48.66666666666666],
     PARAMETER["Latitude_Of_Origin",41.0],
     UNIT["Foot_US",0.3048006096012192]]
FID Column = OBJECTID
Geometry Column NOT NULL = Shape
RecordID: Integer (0.0)
LocationID: String (13.0)
BegFt: Integer (0.0)
EndFt: Integer (0.0)
StreamName: String (100.0)
StreamAndTribNames: String (160.0)
SpecieID: Integer(Int16) (0.0)
Species: String (50.0)
SciName: String (50.0)
RunID: Integer(Int16) (0.0)
Run: String (50.0)
SubRunID: Integer(Int16) (0.0)
SubRun: String (50.0)
LifeHistoryID: Integer(Int16) (0.0)
LifeHistoryType: String (80.0)
UseTypeID: Integer(Int16) (0.0)
UseType: String (50.0)
RefID: Integer (0.0)
Citation: String (3600.0)
URL: String (1000.0)
CompAgy: String (12.0)
BasisID: Integer(Int16) (0.0)
Basis: String (80.0)
UpdDate: DateTime (0.0)
EndExtentID: Integer(Int16) (0.0)
Shape_Length: Real (0.0)

There's no ESPG code there. So I wrote a proj4 file

+datum=NAD83 +ellps=GRS80 +lat_0=41.0 +lat_1=42.33333333333334
+lat_2=48.66666666666666 +proj=lcc +units=foot_us +x_0=2999999.999988
+y_0=0.0

Grass did not like it. There's no projection name in
../PERMANENT/PROJ_INFO:

name: unnamed
datum: nad83
ellps: grs80
proj: lcc
lat_1: 42.33333333333334
lat_2: 48.66666666666666
lat_0: 41
lon_0: 0
x_0: 2999999.999988
y_0: 0
no_defs: defined
towgs84: 0.000,0.000,0.000

and there are no data displayable.

Please suggest what projection values I should use on these data.

Rich

rouault · October 26, 2017, 9:24pm

On jeudi 26 octobre 2017 13:52:21 CEST Rich Shepard wrote:

I am trying to import/open data files from www.streamnet.org but their

GDB format is quite different from what I’ve seen before now and

grass-7.3.svn will not open them. As an aside, I was thrown for a while

looking for OpenFileGDB in the v.in.ogr supported format list until I

noticed it’s been renamed to ESRIFileGDB.

The results of running ‘ogrinfo -al so’ on the directory,

BullTrout_201201_v931.gdb/, produces this result:

Had to open data source read-only.

INFO: Open of `BullTrout_201201_v931.gdb/’

using driver `OpenFileGDB’ successful.

Layer name: FishD_BullTrout_January2012

Geometry: Measured Multi Line String

Feature Count: 4620

Extent: (1158336.501590, 288917.146174) - (4143404.913877, 3148258.748272)

Layer SRS WKT:

PROJCS[“NAD_1983_Lambert_Conformal_Conic”,

GEOGCS[“GCS_North_American_1983”,

DATUM[“North_American_Datum_1983”,

SPHEROID[“GRS_1980”,6378137.0,298.257222101]],

PRIMEM[“Greenwich”,0.0],

UNIT[“Degree”,0.0174532925199433]],

PROJECTION[“Lambert_Conformal_Conic_2SP”],

PARAMETER[“False_Easting”,2999999.999988],

PARAMETER[“False_Northing”,0.0],

PARAMETER[“Central_Meridian”,-117.0],

PARAMETER[“Standard_Parallel_1”,42.33333333333334],

PARAMETER[“Standard_Parallel_2”,48.66666666666666],

PARAMETER[“Latitude_Of_Origin”,41.0],

UNIT[“Foot_US”,0.3048006096012192]]

FID Column = OBJECTID

Geometry Column NOT NULL = Shape

RecordID: Integer (0.0)

LocationID: String (13.0)

BegFt: Integer (0.0)

EndFt: Integer (0.0)

StreamName: String (100.0)

StreamAndTribNames: String (160.0)

SpecieID: Integer(Int16) (0.0)

Species: String (50.0)

SciName: String (50.0)

RunID: Integer(Int16) (0.0)

Run: String (50.0)

SubRunID: Integer(Int16) (0.0)

SubRun: String (50.0)

LifeHistoryID: Integer(Int16) (0.0)

LifeHistoryType: String (80.0)

UseTypeID: Integer(Int16) (0.0)

UseType: String (50.0)

RefID: Integer (0.0)

Citation: String (3600.0)

URL: String (1000.0)

CompAgy: String (12.0)

BasisID: Integer(Int16) (0.0)

Basis: String (80.0)

UpdDate: DateTime (0.0)

EndExtentID: Integer(Int16) (0.0)

Shape_Length: Real (0.0)

There’s no ESPG code there. So I wrote a proj4 file

+datum=NAD83 +ellps=GRS80 +lat_0=41.0 +lat_1=42.33333333333334

+lat_2=48.66666666666666 +proj=lcc +units=foot_us +x_0=2999999.999988

+y_0=0.0

You forgot +lon_0, and a bit counter-intuitively , proj.4 requires x_0 to be expressed in meters (*)

So try instead

+proj=lcc +lat_1=42.33333333333334 +lat_2=48.66666666666666 +lat_0=41 +lon_0=-117 +x_0=914401.8287999999 +y_0=0 +datum=NAD83 +units=us-ft +no_defs

(hint: was given by “gdalsrsinfo the.gdb”)

Even

(*) See http://proj4.org/parameters.html#false-easting-northing

–

Spatialys - Geospatial professional services

http://www.spatialys.com

rshepard · October 26, 2017, 9:47pm

On Thu, 26 Oct 2017, Even Rouault wrote:

You forgot +lon_0, and a bit counter-intuitively , proj.4 requires x_0 to
be expressed in meters (*)

Even,

OK. Thanks.

So try instead

+proj=lcc +lat_1=42.33333333333334 +lat_2=48.66666666666666 +lat_0=41
+lon_0=-117 +x_0=914401.8287999999 +y_0=0 +datum=NAD83 +units=us-ft
+no_defs

(hint: was given by "gdalsrsinfo the.gdb")

This is a new one for me.

Is it common for *FileGDB formats to not have EPSG codes?

Regards,

Rich

Markus_Metz · October 27, 2017, 6:43am

On Thu, Oct 26, 2017 at 11:47 PM, Rich Shepard <rshepard@appl-ecosys.com> wrote:

On Thu, 26 Oct 2017, Even Rouault wrote:

You forgot +lon_0, and a bit counter-intuitively , proj.4 requires x_0 to
be expressed in meters (*)

Even,

OK. Thanks.

Instead of manually creating a proj.4 definition, you can use
v.in.ogr in=BullTrout_201201_v931.gdb location=bulltrout_location -i

to create a location from that gdb. If you don’t use the -i flag, the data will also be imported into the new location.

With GRASS 7.3, v.in.ogr will report back if there are any problems with the included SRS, or if no SRS is available for the selected layer.

Markus M

So try instead

+proj=lcc +lat_1=42.33333333333334 +lat_2=48.66666666666666 +lat_0=41
+lon_0=-117 +x_0=914401.8287999999 +y_0=0 +datum=NAD83 +units=us-ft
+no_defs

(hint: was given by “gdalsrsinfo the.gdb”)

This is a new one for me.

Is it common for *FileGDB formats to not have EPSG codes?

Regards,

Rich

grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

rshepard · October 27, 2017, 12:45pm

On Fri, 27 Oct 2017, Markus Metz wrote:

Instead of manually creating a proj.4 definition, you can use v.in.ogr
in=BullTrout_201201_v931.gdb location=bulltrout_location -i

to create a location from that gdb. If you don't use the -i flag, the data
will also be imported into the new location.

With GRASS 7.3, v.in.ogr will report back if there are any problems with
the included SRS, or if no SRS is available for the selected layer.

Markus,

This is a better approach and I appreciate learning it.

Thanks,

Rich

rshepard · October 30, 2017, 3:35pm

On Fri, 27 Oct 2017, Markus Metz wrote:

Instead of manually creating a proj.4 definition, you can use
v.in.ogr in=BullTrout_201201_v931.gdb location=bulltrout_location -i
to create a location from that gdb. If you don't use the -i flag, the data
will also be imported into the new location.

With GRASS 7.3, v.in.ogr will report back if there are any problems with
the included SRS, or if no SRS is available for the selected layer.

Markus,

I'm importing all species into their own locations; will combine later.
The above command worked (so far) with four species, but on one it threw a
seqfault:

GRASS 7.3.svn (Fish):~/data/grassdata > v.in.ogr in=~/data/species/fish/RedbandTrout_201201_v931.gdb/ location=redband_trout_loc
WARNING: Updating spatial reference with embedded proj4 definition
WARNING: Datum <unknown> not recognised by GRASS and no parameters found
Segmentation fault

I tried to find the proj4 defintion but have no idea in which file it is
found. I can provide the directory if desired; it's 18M so I could put it on
Dropbox. Or, is there something I can do here to clear the blockage?

Regards,

Rich

hellik · October 30, 2017, 7:48pm

Rich Shepard wrote

On Fri, 27 Oct 2017, Markus Metz wrote:

Instead of manually creating a proj.4 definition, you can use
v.in.ogr in=BullTrout_201201_v931.gdb location=bulltrout_location -i
to create a location from that gdb. If you don't use the -i flag, the
data
will also be imported into the new location.

With GRASS 7.3, v.in.ogr will report back if there are any problems with
the included SRS, or if no SRS is available for the selected layer.

Markus,

I'm importing all species into their own locations; will combine later.
The above command worked (so far) with four species, but on one it threw a
seqfault:

GRASS 7.3.svn (Fish):~/data/grassdata > v.in.ogr
in=~/data/species/fish/RedbandTrout_201201_v931.gdb/
location=redband_trout_loc
WARNING: Updating spatial reference with embedded proj4 definition
WARNING: Datum
<unknown>
not recognised by GRASS and no parameters found
Segmentation fault

I tried to find the proj4 defintion but have no idea in which file it
is
found. I can provide the directory if desired; it's 18M so I could put it
on
Dropbox. Or, is there something I can do here to clear the blockage?

Regards,

Rich

_______________________________________________
grass-user mailing list

grass-user@.osgeo

https://lists.osgeo.org/mailman/listinfo/grass-user

the best thing to get any information about srs of your data is using
ogrinfo:

ogrinfo yourGISdata

compare then the ogrinfo output with the srs of your location

-----
best regards
Helmut
--
Sent from: http://osgeo-org.1560.x6.nabble.com/Grass-Users-f3884509.html

Markus_Metz · October 30, 2017, 8:47pm

On Mon, Oct 30, 2017 at 4:35 PM, Rich Shepard <rshepard@appl-ecosys.com> wrote:

On Fri, 27 Oct 2017, Markus Metz wrote:

Instead of manually creating a proj.4 definition, you can use
v.in.ogr in=BullTrout_201201_v931.gdb location=bulltrout_location -i
to create a location from that gdb. If you don’t use the -i flag, the data
will also be imported into the new location.

With GRASS 7.3, v.in.ogr will report back if there are any problems with
the included SRS, or if no SRS is available for the selected layer.

Markus,

I’m importing all species into their own locations; will combine later.
The above command worked (so far) with four species, but on one it threw a
seqfault:

GRASS 7.3.svn (Fish):~/data/grassdata > v.in.ogr in=~/data/species/fish/RedbandTrout_201201_v931.gdb/ location=redband_trout_loc
WARNING: Updating spatial reference with embedded proj4 definition
WARNING: Datum not recognised by GRASS and no parameters found
Segmentation fault

Rich,

I have fixed the segmentation fault in trunk r71613. Please update your local copy and test.

Markus M

I tried to find the proj4 defintion but have no idea in which file it is
found. I can provide the directory if desired; it’s 18M so I could put it on
Dropbox. Or, is there something I can do here to clear the blockage?

Regards,

Rich

grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

rshepard · October 30, 2017, 8:50pm

On Mon, 30 Oct 2017, Helmut Kudrnovsky wrote:

the best thing to get any information about srs of your data is using
ogrinfo:

ogrinfo yourGISdata

compare then the ogrinfo output with the srs of your location

Helmut,

The projection information I see looks acceptable to grass:

Layer SRS WKT:
PROJCS["WGS 84 / Pseudo-Mercator",
     GEOGCS["WGS 84",
         DATUM["WGS_1984",
             SPHEROID["WGS 84",6378137,298.257223563,
                 AUTHORITY["EPSG","7030"]],
             AUTHORITY["EPSG","6326"]],
         PRIMEM["Greenwich",0,
             AUTHORITY["EPSG","8901"]],
         UNIT["degree",0.0174532925199433,
             AUTHORITY["EPSG","9122"]],
         AUTHORITY["EPSG","4326"]],
     PROJECTION["Mercator_1SP"],
     PARAMETER["central_meridian",0],
     PARAMETER["scale_factor",1],
     PARAMETER["false_easting",0],
     PARAMETER["false_northing",0],
     UNIT["metre",1,
         AUTHORITY["EPSG","9001"]],
     AXIS["X",EAST],
     AXIS["Y",NORTH],
     EXTENSION["PROJ4","+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs"],
     AUTHORITY["EPSG","3857"]]

I created a new location using EPSG 3857 but when I tried to import the
data using v.in.ogr from the GUI this results:

(Mon Oct 30 13:46:30 2017) v.in.ogr input=/home/rshepard/data/species/fish/RedbandTrout_201201_v931.gdb layer=FishD_RedbandTrout_September2012 output=redband_trout2012
WARNING: Updating spatial reference with embedded proj4 definition
WARNING: Datum <unknown> not recognised by GRASS and no parameters found
(Mon Oct 30 13:46:31 2017) Command finished (0 sec)

Why is WGS_1984 not recognized by grass7.3.svn? Have I used the incorrect
EPSG code? And why doesn't grass find the correct datum when I let it create
a new location?

Thanks,

Rich

rshepard · October 30, 2017, 9:12pm

On Mon, 30 Oct 2017, Markus Metz wrote:

I have fixed the segmentation fault in trunk r71613. Please update your
local copy and test.

Markus,

The fix works.

Thanks very much!

Rich

rshepard · October 30, 2017, 9:16pm

On Mon, 30 Oct 2017, Rich Shepard wrote:

The projection information I see looks acceptable to grass:

Why is WGS_1984 not recognized by grass7.3.svn? Have I used the incorrect
EPSG code? And why doesn't grass find the correct datum when I let it
create a new location?

Helmut,

Markus M's fix for the segfault works. Because I used the same command
in the shell as the GUI used the non-recognition of the datum and the
segfault were of the same origin.

My importing of FileGDB data has exposed a few bugs which is a good thing.
Markus' immediate fixes is an even better thing.

My thanks to both of you,

Rich

Markus_Metz · October 30, 2017, 9:28pm

On Mon, Oct 30, 2017 at 10:16 PM, Rich Shepard <rshepard@appl-ecosys.com> wrote:

On Mon, 30 Oct 2017, Rich Shepard wrote:

The projection information I see looks acceptable to grass:

Why is WGS_1984 not recognized by grass7.3.svn? Have I used the incorrect
EPSG code? And why doesn’t grass find the correct datum when I let it
create a new location?

Helmut,

Markus M’s fix for the segfault works. Because I used the same command
in the shell as the GUI used the non-recognition of the datum and the
segfault were of the same origin.

My importing of FileGDB data has exposed a few bugs which is a good thing.
Markus’ immediate fixes is an even better thing.

My thanks to both of you,

Thanks for testing GRASS 7.3!

The projection information is problematic because it is contradicting itself. WKT says datum and ellipsoid are WGS84, but the embedded proj.4 string says there is no datum and the ellipsoid is a sphere because of “+a=6378137 +b=6378137”.

You should check the geolocation accuracy after reprojection.

Markus M

Rich

grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

rshepard · October 30, 2017, 10:17pm

On Mon, 30 Oct 2017, Markus Metz wrote:

Thanks for testing GRASS 7.3!

Markus,

Glad to do it.

The projection information is problematic because it is contradicting
itself. WKT says datum and ellipsoid are WGS84, but the embedded proj.4
string says there is no datum and the ellipsoid is a sphere because of
"+a=6378137 +b=6378137".

I wondered about this but I'm not sufficiently focused on these details to
have such contradictions immediately visible.

You should check the geolocation accuracy after reprojection.

Will do this.

Best regards,

Rich

rshepard · October 30, 2017, 10:22pm

On Mon, 30 Oct 2017, Markus Metz wrote:

You should check the geolocation accuracy after reprojection.

Markus,

I created a new location for importing the redband trout data (which I
assume used the Mercator projection shown in the SRS). Then I re-projected
it to the location with the rest of the fish (they all came from the same
source so I'll let them know about the discrepancy). The projection
information for all these species is:

name: NAD_1983_Lambert_Conformal_Conic
datum: nad83
ellps: grs80
proj: lcc
lat_1: 42.33333333333334
lat_2: 48.66666666666666
lat_0: 41
lon_0: -117
x_0: 914401.8287999999
y_0: 0
no_defs: defined

Rich

Markus_Metz · October 30, 2017, 10:30pm

On Mon, Oct 30, 2017 at 11:22 PM, Rich Shepard <rshepard@appl-ecosys.com> wrote:

On Mon, 30 Oct 2017, Markus Metz wrote:

You should check the geolocation accuracy after reprojection.

Markus,

I created a new location for importing the redband trout data (which I
assume used the Mercator projection shown in the SRS). Then I re-projected
it to the location with the rest of the fish (they all came from the same
source so I’ll let them know about the discrepancy).

You need to visually check the geolocation accuracy, i.e. plot the data together with other reference data and search for any geolocation inaccuracies. The target projection information does not matter.

Markus M

The projection
information for all these species is:

name: NAD_1983_Lambert_Conformal_Conic
datum: nad83
ellps: grs80
proj: lcc
lat_1: 42.33333333333334
lat_2: 48.66666666666666
lat_0: 41
lon_0: -117
x_0: 914401.8287999999
y_0: 0
no_defs: defined

Rich

grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

rouault · October 30, 2017, 10:30pm

On lundi 30 octobre 2017 22:28:27 CET Markus Metz wrote:

On Mon, Oct 30, 2017 at 10:16 PM, Rich Shepard rshepard@appl-ecosys.com

wrote:

On Mon, 30 Oct 2017, Rich Shepard wrote:

The projection information I see looks acceptable to grass:

Why is WGS_1984 not recognized by grass7.3.svn? Have I used the

incorrect

EPSG code? And why doesn’t grass find the correct datum when I let it

create a new location?

Helmut,

Markus M’s fix for the segfault works. Because I used the same command

in the shell as the GUI used the non-recognition of the datum and the

segfault were of the same origin.

My importing of FileGDB data has exposed a few bugs which is a good

thing.

Markus’ immediate fixes is an even better thing.

My thanks to both of you,

Thanks for testing GRASS 7.3!

The projection information is problematic because it is contradicting

itself. WKT says datum and ellipsoid are WGS84, but the embedded proj.4

string says there is no datum and the ellipsoid is a sphere because of

“+a=6378137 +b=6378137”.

Yes, this is because WebMercator is fundamentally a hack from a cartography point of view. It uses the spherical formulation of Mercator projection for an ellipsoid. PROJ.4 has no better way of expressing this has +a=6378137 +b=6378137 +nadgrids=@null.

A cleaner way would to add a new projection method in proj.4, let’s say +proj=spherical_merc, and then you’d have:

+proj=spherical_merc +datum=WGS84

The pure WKT part of the WKT string reported by GDAL for EPSG:3857 is atually a lie. It shouldn’t use PROJECTION[“Mercator_1SP”] (ESRI uses a PROJECTION[“Mercator_Auxiliary_Sphere”] instead.)

The way GDAL solves the issue is to add the

EXTENSION[“PROJ4”,“+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs”] node so that the SRS is correctly exported to something that works with proj.4

–

Spatialys - Geospatial professional services

http://www.spatialys.com

rshepard · October 30, 2017, 10:45pm

On Mon, 30 Oct 2017, Markus Metz wrote:

You need to visually check the geolocation accuracy, i.e. plot the data
together with other reference data and search for any geolocation
inaccuracies. The target projection information does not matter.

Markus,

Will do.

Thanks,

Rich