[Geoserver-users] bulk shapefile import

I'm attempting to import a large collection of shape files and want
them all added to a specific map layer. I found the "import a
directory of shape files" feature in the create a new data store admin
page, but I'm not seeing an easy way to dump everything into a map
layer.

also, do we have support for esri geo databases?

On Thu, Dec 17, 2015 at 1:22 AM, Alex O'Ree <spyhunter99@anonymised.com> wrote:

I'm attempting to import a large collection of shape files and want
them all added to a specific map layer. I found the "import a
directory of shape files" feature in the create a new data store admin
page, but I'm not seeing an easy way to dump everything into a map
layer.

All in a single layer? Does that mean they all have the same set of
attributes?
Anyways, no, we don't have a way to support that.

also, do we have support for esri geo databases?

No supported way, but you could try using the geotools OGR data store
(there is no plugin packaging for it, you'll have to build from sources):

https://github.com/geotools/geotools/tree/master/modules/plugin/ogr

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

*Geosolutions' Winter Holidays from 24/12 to 6/1*

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

but I’m not seeing an easy way to dump everything into a map layer.

All in a single layer? Does that mean they all have the same set of attributes?

Anyways, no, we don’t have a way to support that.

One way could be to import them into postgis and set up a view that selects all from those tables.

Does it really have to be a single layer? Could you not use a layer group?

···

On Thu, Dec 17, 2015 at 1:22 AM, Alex O’Ree <spyhunter99@…84…> wrote:

I’m attempting to import a large collection of shape files and want
them all added to a specific map layer. I found the “import a
directory of shape files” feature in the create a new data store admin
page, but I’m not seeing an easy way to dump everything into a map
layer.

All in a single layer? Does that mean they all have the same set of attributes?

Anyways, no, we don’t have a way to support that.

also, do we have support for esri geo databases?

No supported way, but you could try using the geotools OGR data store

(there is no plugin packaging for it, you’ll have to build from sources):

https://github.com/geotools/geotools/tree/master/modules/plugin/ogr

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit

http://goo.gl/it488V for more information.

==

Geosolutions’ Winter Holidays from 24/12 to 6/1

Ing. Andrea Aime

@geowolf

Technical Lead

GeoSolutions S.A.S.

Via Poggio alle Viti 1187

55054 Massarosa (LU)

Italy

phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


If all Shape files have the same attributes and table structure, you could append the Shapes into one file before uploading to GeoServer using ogr2ogr:

ogr2ogr -update -append


From: VICTOR.KIRK@anonymised.com
To: geoserver-users@lists.sourceforge.net
Date: Thu, 17 Dec 2015 09:29:22 +0000
Subject: Re: [Geoserver-users] bulk shapefile import

but I’m not seeing an easy way to dump everything into a map layer.

All in a single layer? Does that mean they all have the same set of attributes?

Anyways, no, we don’t have a way to support that.

One way could be to import them into postgis and set up a view that selects all from those tables.

Does it really have to be a single layer? Could you not use a layer group?

From: Andrea Aime [mailto:andrea.aime@anonymised.com]
Sent: 17 December 2015 07:21
To: Alex O’Ree
Cc: GeoServer Mailing List List
Subject: Re: [Geoserver-users] bulk shapefile import

On Thu, Dec 17, 2015 at 1:22 AM, Alex O’Ree <spyhunter99@anonymised.com…> wrote:

I’m attempting to import a large collection of shape files and want
them all added to a specific map layer. I found the “import a
directory of shape files” feature in the create a new data store admin
page, but I’m not seeing an easy way to dump everything into a map
layer.

All in a single layer? Does that mean they all have the same set of attributes?

Anyways, no, we don’t have a way to support that.

also, do we have support for esri geo databases?

No supported way, but you could try using the geotools OGR data store

(there is no plugin packaging for it, you’ll have to build from sources):

https://github.com/geotools/geotools/tree/master/modules/plugin/ogr

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit

http://goo.gl/it488V for more information.

==

Geosolutions’ Winter Holidays from 24/12 to 6/1

Ing. Andrea Aime

@geowolf

Technical Lead

GeoSolutions S.A.S.

Via Poggio alle Viti 1187

55054 Massarosa (LU)

Italy

phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.



_______________________________________________ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users

Yet another way could be to use the REST API:
http://docs.geoserver.org/latest/en/user/rest/api/datastores.html#workspaces-ws-datastores-ds-file-url-external-extension

You should issue one PUT request for each shape file you want to upload; the first one will create a shape datastore in GeoServer, the following requests will result in the data being appended to it.

I’ve not tested it, but it should work :slight_smile:

Hope this helps.

···

On Thu, Dec 17, 2015 at 10:38 AM, Edward Mac Gillavry <emacgillavry@anonymised.com> wrote:

If all Shape files have the same attributes and table structure, you could append the Shapes into one file before uploading to GeoServer using ogr2ogr:

ogr2ogr -update -append


From: VICTOR.KIRK@anonymised.com
To: geoserver-users@lists.sourceforge.net
Date: Thu, 17 Dec 2015 09:29:22 +0000

Subject: Re: [Geoserver-users] bulk shapefile import

but I’m not seeing an easy way to dump everything into a map layer.

All in a single layer? Does that mean they all have the same set of attributes?

Anyways, no, we don’t have a way to support that.

One way could be to import them into postgis and set up a view that selects all from those tables.

Does it really have to be a single layer? Could you not use a layer group?

From: Andrea Aime [mailto:andrea.aime@anonymised.com]
Sent: 17 December 2015 07:21
To: Alex O’Ree
Cc: GeoServer Mailing List List
Subject: Re: [Geoserver-users] bulk shapefile import

On Thu, Dec 17, 2015 at 1:22 AM, Alex O’Ree <spyhunter99@anonymised.com> wrote:

I’m attempting to import a large collection of shape files and want
them all added to a specific map layer. I found the “import a
directory of shape files” feature in the create a new data store admin
page, but I’m not seeing an easy way to dump everything into a map
layer.

All in a single layer? Does that mean they all have the same set of attributes?

Anyways, no, we don’t have a way to support that.

also, do we have support for esri geo databases?

No supported way, but you could try using the geotools OGR data store

(there is no plugin packaging for it, you’ll have to build from sources):

https://github.com/geotools/geotools/tree/master/modules/plugin/ogr

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit

http://goo.gl/it488V for more information.

==

Geosolutions’ Winter Holidays from 24/12 to 6/1

Ing. Andrea Aime

@geowolf

Technical Lead

GeoSolutions S.A.S.

Via Poggio alle Viti 1187

55054 Massarosa (LU)

Italy

phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.



_______________________________________________ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users



Geoserver-users mailing list
Geoserver-users@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Best regards,
Stefano Costa

==
GeoServer Professional Services from the experts! Visit
[http://goo.gl/it488V](http://goo.gl/it488V) for more information.
==
Dott. Stefano Costa
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:     +39 0584 1660272

[http://www.geo-solutions.it](http://www.geo-solutions.it)
[http://twitter.com/geosolutions_it](http://twitter.com/geosolutions_it)

-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate.
Il loro utilizzo è consentito esclusivamente al destinatario del
messaggio, per le finalità indicate nel messaggio stesso. Qualora
riceviate questo messaggio senza esserne il destinatario, Vi preghiamo
cortesemente di darcene notizia via e-mail e di procedere alla
distruzione del messaggio stesso, cancellandolo dal Vostro sistema.
Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità
diverse, costituisce comportamento contrario ai principi dettati dal
D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely
for the attention and use of the named addressee(s) and may be
confidential or proprietary in nature or covered by the provisions of
privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New
Data Protection Code).Any use not in accord with its purpose, any
disclosure, reproduction, copying, distribution, or either
dissemination, either whole or partial, is strictly forbidden except
previous formal approval of the named addressee(s). If you are not the
intended recipient, please contact immediately the sender by
telephone, fax or e-mail and delete the information in this message
that has been received in error. The sender does not give any warranty
or accept liability as the content, accuracy or completeness of sent
messages and accepts no responsibility  for changes made after they
were sent or for other risks which arise as a result of e-mail
transmission, viruses, etc.

Yes, I want to import these....
http://earth-info.nga.mil/GandG/coordsys/grids/mgrs_100km_dloads.html
As well as the 1km sharefiles. We're talking about 2*60*26
shapefiles=3120. It's basically a grid overlay that you would put on
top of a base layer. Each shape file is non-overlapping. Also at a
given zoom level, say OSM's zoom < 10, I'd want the 100km files to be
shown, and beyond that, the 1km shape files. I have ogr running, but
what would be the recommended format to convert the files to, in order
bulk import them. I'm just looking for the easiest solution to do
this.

On Thu, Dec 17, 2015 at 2:21 AM, Andrea Aime
<andrea.aime@anonymised.com> wrote:

On Thu, Dec 17, 2015 at 1:22 AM, Alex O'Ree <spyhunter99@anonymised.com> wrote:

I'm attempting to import a large collection of shape files and want
them all added to a specific map layer. I found the "import a
directory of shape files" feature in the create a new data store admin
page, but I'm not seeing an easy way to dump everything into a map
layer.

All in a single layer? Does that mean they all have the same set of
attributes?
Anyways, no, we don't have a way to support that.

also, do we have support for esri geo databases?

No supported way, but you could try using the geotools OGR data store
(there is no plugin packaging for it, you'll have to build from sources):

https://github.com/geotools/geotools/tree/master/modules/plugin/ogr

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Geosolutions' Winter Holidays from 24/12 to 6/1

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i
file/s allegato/i sono da considerarsi strettamente riservate. Il loro
utilizzo è consentito esclusivamente al destinatario del messaggio, per le
finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio
senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia
via e-mail e di procedere alla distruzione del messaggio stesso,
cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo
anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per
finalità diverse, costituisce comportamento contrario ai principi dettati
dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender does
not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

There is an OGR VRT format (http://www.gdal.org/drv_vrt.html) which you have to hand create (last time I tried anyway) and then use ogr2ogr to convert it back to a single shapefile for import. I would probably save myself the effort and dump the whole lot into PostGIS using a simple loop and either ogr2ogr or shppsql, just remember to leave creating the indexes until the end.

Ian

···

On 17 December 2015 at 11:58, Alex O’Ree <spyhunter99@anonymised.com…84…> wrote:

Yes, I want to import these…
http://earth-info.nga.mil/GandG/coordsys/grids/mgrs_100km_dloads.html
As well as the 1km sharefiles. We’re talking about 26026
shapefiles=3120. It’s basically a grid overlay that you would put on
top of a base layer. Each shape file is non-overlapping. Also at a
given zoom level, say OSM’s zoom < 10, I’d want the 100km files to be
shown, and beyond that, the 1km shape files. I have ogr running, but
what would be the recommended format to convert the files to, in order
bulk import them. I’m just looking for the easiest solution to do
this.

On Thu, Dec 17, 2015 at 2:21 AM, Andrea Aime
<andrea.aime@anonymised.com> wrote:

On Thu, Dec 17, 2015 at 1:22 AM, Alex O’Ree <spyhunter99@anonymised.com> wrote:

I’m attempting to import a large collection of shape files and want
them all added to a specific map layer. I found the “import a
directory of shape files” feature in the create a new data store admin
page, but I’m not seeing an easy way to dump everything into a map
layer.

All in a single layer? Does that mean they all have the same set of
attributes?
Anyways, no, we don’t have a way to support that.

also, do we have support for esri geo databases?

No supported way, but you could try using the geotools OGR data store
(there is no plugin packaging for it, you’ll have to build from sources):

https://github.com/geotools/geotools/tree/master/modules/plugin/ogr

Cheers
Andrea

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Geosolutions’ Winter Holidays from 24/12 to 6/1

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i
file/s allegato/i sono da considerarsi strettamente riservate. Il loro
utilizzo è consentito esclusivamente al destinatario del messaggio, per le
finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio
senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia
via e-mail e di procedere alla distruzione del messaggio stesso,
cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo
anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per
finalità diverse, costituisce comportamento contrario ai principi dettati
dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender does
not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.




Geoserver-users mailing list
Geoserver-users@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Ian Turton