Hi,
I’ve just put toghether a GSIP to improve how the REST file uploads
are handled in GeoServer, in particular to add some control of where the
files are going to get placed (today a fixed strategy is being used).
The proposal follows the idea of URLMangler, that has been used with
success in a number of scenarios so far (personally I’ve been using
it in a number of GeoServer customizations, it has been a life saver
in more than one occasion)
The proposal also contains the idea of a new extensible panel for the
settings page, following along the lines of AdminPagePanel for
the service pages.
There may be some overlap with the work I am doing - was just in the REST code today making it avoid the use of the old vfny GeoserverDataDirectory class.
Hi,
I’ve just put toghether a GSIP to improve how the REST file uploads
are handled in GeoServer, in particular to add some control of where the
files are going to get placed (today a fixed strategy is being used).
The proposal follows the idea of URLMangler, that has been used with
success in a number of scenarios so far (personally I’ve been using
it in a number of GeoServer customizations, it has been a life saver
in more than one occasion)
The proposal also contains the idea of a new extensible panel for the
settings page, following along the lines of AdminPagePanel for
the service pages.
Learn Graph Databases - Download FREE O’Reilly Book
“Graph Databases” is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today! http://p.sf.net/sfu/NeoTech
On Thu, Apr 17, 2014 at 12:13 PM, Jody Garnett <jody.garnett@anonymised.com>wrote:
There may be some overlap with the work I am doing - was just in the REST
code today making it avoid the use of the old vfny GeoserverDataDirectory
class.
Hum.. uh?
The code that I was talking about already uses the loader:
There may be some overlap with the work I am doing - was just in the REST code today making it avoid the use of the old vfny GeoserverDataDirectory class.
Hum… uh?
The code that I was talking about already uses the loader:
final File dir = GeoserverDataDirectory.findCreateConfigDir("data");
The only really used method in here is GeoserverDataDirectory.findDataFile( url ). It does the magic of untangling a URL relative to the data directory. But surprisingly it does not work as consistently in test cases:
1) FileModelTest - accidentally functions with a null baseDirectory during tests - and behaves differently when used in the actual application
2) OvverideTransformationsTest - makes use of various geotools factories - with a null baseDirectory - while the catalog is setup, and then calls CRS test it all for real once the baseDirectory.
But I am off topic and can save it for the pull request.
<details class='elided'>
<summary title='Show trimmed content'>···</summary>
Jody Garnett
On Thu, Apr 17, 2014 at 8:30 PM, Andrea Aime <[andrea.aime@anonymised.com](mailto:andrea.aime@anonymised.com)> wrote:
> Hi,
> made a quick search, on trunk rest-config it seems that the old GeoServerDataDirectory is only used in a test class, CoveragesStoreTest
>
> Cheers
>
> Andrea
On Thu, Apr 17, 2014 at 12:16 PM, Andrea Aime <[andrea.aime@anonymised.com](mailto:andrea.aime@anonymised.com)> wrote:
>
--
==
Meet us at GEO Business 2014! in London! Visit [http://goo.gl/fES3aK](http://goo.gl/fES3aK)
for more information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
[http://www.geo-solutions.it](http://www.geo-solutions.it)
[http://twitter.com/geosolutions_it](http://twitter.com/geosolutions_it)
-------------------------------------------------------
On Thu, Apr 17, 2014 at 12:13 PM, Jody Garnett <[jody.garnett@anonymised.com..](mailto:jody.garnett@anonymised.com)> wrote:
> There may be some overlap with the work I am doing - was just in the REST code today making it avoid the use of the old vfny GeoserverDataDirectory class.
Hum.. uh?
The code that I was talking about already uses the loader:
directory = catalog.getResourceLoader()
.findOrCreateDirectory("data", workspaceName, storeName);
Cheers
Andrea
--
==
Meet us at GEO Business 2014! in London! Visit [http://goo.gl/fES3aK](http://goo.gl/fES3aK)
for more information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
[http://www.geo-solutions.it](http://www.geo-solutions.it)
[http://twitter.com/geosolutions_it](http://twitter.com/geosolutions_it)
-------------------------------------------------------
</details>
On Thu, Apr 17, 2014 at 1:07 PM, Jody Garnett <jody.garnett@anonymised.com>wrote:
I can find RESTUtils using the old class.
final File dir = GeoserverDataDirectory.findCreateConfigDir("data");
The only really used method in here is GeoserverDataDirectory.findDataFile( url ). It does the magic of untangling a URL relative to the data directory. But surprisingly it does not work as consistently in test cases:
1) FileModelTest - accidentally functions with a null baseDirectory during tests - and behaves differently when used in the actual application
2) OvverideTransformationsTest - makes use of various geotools factories - with a null baseDirectory - while the catalog is setup, and then calls CRS test it all for real once the baseDirectory.
I see.. funny...
But I am off topic and can save it for the pull request.
Yep, back on topic... any feedback to the proposal contents?
Cheers
Andrea
--
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.
Ing. Andrea Aime @geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
On Thu, Apr 17, 2014 at 3:33 PM, Jody Garnett <jody.garnett@anonymised.com>wrote:
But I am off topic and can save it for the pull request.
Yep, back on topic... any feedback to the proposal contents?
Yes .. that.
While I understand the proposal I need to tag in Kevin for this one....
Kevin and I were talking today about different sensible ways of allowing
admin access to the resource store stuff:
1) sync against directory and throw file events around
2) create a user interface (yes I said that was out of scope earlier)
3) Put up a REST API
You can see how GSIP 114 overlaps with (3) which was also the option Kevin
strongly preferred.
Hmmm no, I don't see the interaction.
We are not talking about configuration items here, we are talking about
data.
The PathMapper will only manage data that you upload via REST (to create a
new store, or to add files in a mosaic),
and where it ends on the server file system.
It is not meant to be used for configuration items of any kind.
Cheers
Andrea
--
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.
Ing. Andrea Aime @geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
Is the interface going to be used outside of rest uploads? If not perhaps a name that is more specific to it’s use, like RESTUploadPathMapper or something.
Hi,
I’ve just put toghether a GSIP to improve how the REST file uploads
are handled in GeoServer, in particular to add some control of where the
files are going to get placed (today a fixed strategy is being used).
The proposal follows the idea of URLMangler, that has been used with
success in a number of scenarios so far (personally I’ve been using
it in a number of GeoServer customizations, it has been a life saver
in more than one occasion)
The proposal also contains the idea of a new extensible panel for the
settings page, following along the lines of AdminPagePanel for
the service pages.
Learn Graph Databases - Download FREE O’Reilly Book
“Graph Databases” is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today! http://p.sf.net/sfu/NeoTech
On Thu, Apr 17, 2014 at 4:02 PM, Justin Deoliveira <
jdeolive@anonymised.com> wrote:
Is the interface going to be used outside of rest uploads? If not perhaps
a name that is more specific to it's use, like RESTUploadPathMapper or
something.
Good question. The current plan is to use it only for data uploaded via
REST config.
So yes, RESTUploadPathMapper is a better name.
Wondering if we should also stress that we are mapping paths for data, not
dealing with any configuration item.
DataUploadPathMapper maybe?
Cheers
Andrea
--
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.
Ing. Andrea Aime @geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
On Thu, Apr 17, 2014 at 8:05 AM, Andrea Aime
<andrea.aime@anonymised.com>wrote:
On Thu, Apr 17, 2014 at 4:02 PM, Justin Deoliveira <
jdeolive@anonymised.com> wrote:
Is the interface going to be used outside of rest uploads? If not perhaps
a name that is more specific to it's use, like RESTUploadPathMapper or
something.
Good question. The current plan is to use it only for data uploaded via
REST config.
So yes, RESTUploadPathMapper is a better name.
Wondering if we should also stress that we are mapping paths for data, not
dealing with any configuration item.
DataUploadPathMapper maybe?
Right. Makes sense. Although I think anything that marks it specific to
rest is probably fine. I don't see this interface getting to much usage
amongst most developers... I imagine once the regex implementation is in
place that will handle most needs.
Cheers
Andrea
--
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.
Ing. Andrea Aime @geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
On Thu, Apr 17, 2014 at 4:24 PM, Justin Deoliveira <
jdeolive@anonymised.com> wrote:
On Thu, Apr 17, 2014 at 8:05 AM, Andrea Aime <andrea.aime@anonymised.com
> wrote:
On Thu, Apr 17, 2014 at 4:02 PM, Justin Deoliveira <
jdeolive@anonymised.com> wrote:
Is the interface going to be used outside of rest uploads? If not
perhaps a name that is more specific to it's use, like RESTUploadPathMapper
or something.
Good question. The current plan is to use it only for data uploaded via
REST config.
So yes, RESTUploadPathMapper is a better name.
Wondering if we should also stress that we are mapping paths for data,
not dealing with any configuration item.
DataUploadPathMapper maybe?
Right. Makes sense. Although I think anything that marks it specific to
rest is probably fine. I don't see this interface getting to much usage
amongst most developers... I imagine once the regex implementation is in
place that will handle most needs.
Sure, let's go for RESTUploadPathMapper then.
About being the only implementation eh... not everyone is a fan of regexp
(you know the saying, "you had a problem, decided to solve it with
a regular expression, now you have two problems").
When scripting languages become an official extension I have no trouble
imagining someone interested in creating
a custom mapper in jython/groovy to satisfy their needs without having to
bend over backwards and write
a complex regex with group matching :-p
Cheers
Andrea
--
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.
Ing. Andrea Aime @geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
On Thu, Apr 17, 2014 at 4:28 PM, Andrea Aime
<andrea.aime@anonymised.com>wrote:
On Thu, Apr 17, 2014 at 4:24 PM, Justin Deoliveira <
jdeolive@anonymised.com> wrote:
On Thu, Apr 17, 2014 at 8:05 AM, Andrea Aime <
andrea.aime@anonymised.com> wrote:
On Thu, Apr 17, 2014 at 4:02 PM, Justin Deoliveira <
jdeolive@anonymised.com> wrote:
Is the interface going to be used outside of rest uploads? If not
perhaps a name that is more specific to it's use, like RESTUploadPathMapper
or something.
Good question. The current plan is to use it only for data uploaded via
REST config.
So yes, RESTUploadPathMapper is a better name.
Wondering if we should also stress that we are mapping paths for data,
not dealing with any configuration item.
DataUploadPathMapper maybe?
Right. Makes sense. Although I think anything that marks it specific to
rest is probably fine. I don't see this interface getting to much usage
amongst most developers... I imagine once the regex implementation is in
place that will handle most needs.
Sure, let's go for RESTUploadPathMapper then.
Proposal updated to use RESTUploadPathMapper
Cheers
Andrea
--
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.
Ing. Andrea Aime @geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
Learn Graph Databases - Download FREE O’Reilly Book
“Graph Databases” is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today! http://p.sf.net/sfu/NeoTech
Right. Makes sense. Although I think anything that marks it specific to rest is probably fine. I don’t see this interface getting to much usage amongst most developers… I imagine once the regex implementation is in place that will handle most needs.
Is the interface going to be used outside of rest uploads? If not perhaps a name that is more specific to it’s use, like RESTUploadPathMapper or something.
Good question. The current plan is to use it only for data uploaded via REST config.
So yes, RESTUploadPathMapper is a better name.
Wondering if we should also stress that we are mapping paths for data, not dealing with any configuration item.
DataUploadPathMapper maybe?