[Geoserver-devel] ResourcePool always get a File from the CoverageStore URL String. How to deal with custom URLs?

GeoServer GeoServer Developer
1

Hi List,

I’m working on simplifying a bit the steps needed to configure an ImageMosaicJDBC based on PGRaster.
http://jira.codehaus.org/browse/GEOT-4507

Right now, I’m playing with Wicket in order to allow specifying configuration parameters from the GUI (PG server, port, pguser, databaset, …) so that it can be passed down to the AbstractGridFormatReader as a special URL.

something like: pgraster://user:pass@anonymised.com:port:database.schema.table…

I have voted for this approach since while looking for examples, I have seen that the ArcSDE store panel does something similar.
sde://user:pass@anonymised.com:port/instance

When doing some tests right now, I have noticed that the ResourcePool always converts the coverageStore url String to a File (by using the GeoserverDataDirectory.findDataFile utility method).

I’m not sure about how this allows the sde://user:pass@anonymised.com:port/instance String url to be interpreted as a File (I didin’t check the arcSde store code).

I was thinking if instead of changing the ImageMosaicJDBC code to try to parse Files having similar strange path (pgraster://something), we could relax somehow this “force to be a File” behaviour on ResourcePool…
We could probably modify the findDataFile to return a File only in case it really deals with a file:

(file:/ + file: + /some/path/to/something + c:/ + …) and skip “peculiar” cases like:

myformat://customSyntax.

We can probably think about returning null (as a File, to continue passing down a string) in case we see a “something://” prefix longer than a certain amount of chars (to make sure that C:/ still work)… (Do we risk by this way to improperly parse some peculiar network mapping / protocol I’m unaware of?)

What are your thoughts on this topic?

Please, let me know.
Best Regards,
Daniele

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

2

Hi guys,
I didn’t receive any feedback on this.

I have encountered again the issue on windows which converts my “pgraster://” string into a “pgraster:\ file” which let fail my URL parsing code.

I would like to do something like this on that utility method.

The last else in GeoserverDataDirectory.findDataFileFile simply returns new File(path).

I would do instead something like this:

File file = new File(path);
if (file.exists()) {

return file;
}

return null;

Afterwards in the ResourcePool code (near to line 1244):
final File obj = GeoserverDataDirectory.findDataFile(info.getURL());
reader = gridFormat.getReader(obj, new Hints(hints));

if obj is null, I will use the original info.getURL() String instead of obj as input of the getReader.

What do you think about it?

Cheers,
Daniele

···

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

On Thu, Jul 18, 2013 at 3:59 PM, Daniele Romagnoli <daniele.romagnoli@anonymised.com> wrote:

Hi List,

I’m working on simplifying a bit the steps needed to configure an ImageMosaicJDBC based on PGRaster.
http://jira.codehaus.org/browse/GEOT-4507

Right now, I’m playing with Wicket in order to allow specifying configuration parameters from the GUI (PG server, port, pguser, databaset, …) so that it can be passed down to the AbstractGridFormatReader as a special URL.

something like: pgraster://user:pass@anonymised.com:port:database.schema.table…

I have voted for this approach since while looking for examples, I have seen that the ArcSDE store panel does something similar.
sde://user:pass@anonymised.com:port/instance

When doing some tests right now, I have noticed that the ResourcePool always converts the coverageStore url String to a File (by using the GeoserverDataDirectory.findDataFile utility method).

I’m not sure about how this allows the sde://user:pass@anonymised.com:port/instance String url to be interpreted as a File (I didin’t check the arcSde store code).

I was thinking if instead of changing the ImageMosaicJDBC code to try to parse Files having similar strange path (pgraster://something), we could relax somehow this “force to be a File” behaviour on ResourcePool…
We could probably modify the findDataFile to return a File only in case it really deals with a file:

(file:/ + file: + /some/path/to/something + c:/ + …) and skip “peculiar” cases like:

myformat://customSyntax.

We can probably think about returning null (as a File, to continue passing down a string) in case we see a “something://” prefix longer than a certain amount of chars (to make sure that C:/ still work)… (Do we risk by this way to improperly parse some peculiar network mapping / protocol I’m unaware of?)

What are your thoughts on this topic?

Please, let me know.
Best Regards,
Daniele

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

3

On Wed, Jul 24, 2013 at 6:52 PM, Daniele Romagnoli <
daniele.romagnoli@anonymised.com> wrote:

Hi guys,
I didn't receive any feedback on this.
I have encountered again the issue on windows which converts my
"pgraster://" string into a "pgraster:\ file" which let fail my URL parsing
code.
I would like to do something like this on that utility method.
The last else in GeoserverDataDirectory.findDataFileFile simply returns
new File(path).

I would do instead something like this:
File file = new File(path);
if (file.exists()) {
   return file;
}
return null;

Where do you see that? Afaik GeoServerDataDirectory ends up delegating to
this method,
which seems to be checking for existance on all resources:

https://github.com/geoserver/geoserver/blob/master/src/platform/src/main/java/org/geoserver/platform/GeoServerResourceLoader.java#L191

If that is true, then checking if the returned obj is not null and use the
original string should suffice.

Btw, this should also fix the SDE raster source

Cheers
Andrea

--

Our support, Your Success! Visit http://opensdi.geo-solutions.it for more
information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

4

Hi Andrea,
please, read below…

···

On Wed, Jul 24, 2013 at 7:03 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On ResourcePool class, the getGridCoverageReader method tries to get a File in case the reader is null, by invoking GeoserverDataDirectory.findDataFile();
https://github.com/geoserver/geoserver/blob/master/src/main/src/main/java/org/geoserver/catalog/ResourcePool.java#L1244

findDataFile (https://github.com/geoserver/geoserver/blob/master/src/main/src/main/java/org/vfny/geoserver/global/GeoserverDataDirectory.java#L210)
return a newFile as last “else” statement:
https://github.com/geoserver/geoserver/blob/master/src/main/src/main/java/org/vfny/geoserver/global/GeoserverDataDirectory.java#L242

Cheers,
Daniele

On Wed, Jul 24, 2013 at 6:52 PM, Daniele Romagnoli <daniele.romagnoli@anonymised.com8…> wrote:

Hi guys,
I didn’t receive any feedback on this.

I have encountered again the issue on windows which converts my “pgraster://” string into a “pgraster:\ file” which let fail my URL parsing code.

I would like to do something like this on that utility method.

The last else in GeoserverDataDirectory.findDataFileFile simply returns new File(path).

I would do instead something like this:

File file = new File(path);
if (file.exists()) {

return file;
}

return null;

Where do you see that?

Afaik GeoServerDataDirectory ends up delegating to this method,
which seems to be checking for existance on all resources:

https://github.com/geoserver/geoserver/blob/master/src/platform/src/main/java/org/geoserver/platform/GeoServerResourceLoader.java#L191

If that is true, then checking if the returned obj is not null and use the original string should suffice.

Btw, this should also fix the SDE raster source

Cheers
Andrea

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

5

On Thu, Jul 25, 2013 at 9:19 AM, Daniele Romagnoli <
daniele.romagnoli@anonymised.com> wrote:

Hi Andrea,
please, read below...

On Wed, Jul 24, 2013 at 7:03 PM, Andrea Aime <andrea.aime@anonymised.com
> wrote:

On Wed, Jul 24, 2013 at 6:52 PM, Daniele Romagnoli <
daniele.romagnoli@anonymised.com> wrote:

Hi guys,
I didn't receive any feedback on this.
I have encountered again the issue on windows which converts my
"pgraster://" string into a "pgraster:\ file" which let fail my URL parsing
code.
I would like to do something like this on that utility method.
The last else in GeoserverDataDirectory.findDataFileFile simply returns
new File(path).

I would do instead something like this:
File file = new File(path);
if (file.exists()) {
   return file;
}
return null;

Where do you see that?

On ResourcePool class, the getGridCoverageReader method tries to get a
File in case the reader is null, by invoking
GeoserverDataDirectory.findDataFile();

https://github.com/geoserver/geoserver/blob/master/src/main/src/main/java/org/geoserver/catalog/ResourcePool.java#L1244

findDataFile (
https://github.com/geoserver/geoserver/blob/master/src/main/src/main/java/org/vfny/geoserver/global/GeoserverDataDirectory.java#L210
)
return a newFile as last "else" statement:

https://github.com/geoserver/geoserver/blob/master/src/main/src/main/java/org/vfny/geoserver/global/GeoserverDataDirectory.java#L242

Doh, I went down the wrong path.
Yeah, I believe what you're suggesting makes sense. The code should not
really be tied only to file system paths,
grid coverages can come from remote services too. (postgis, oracle, remote
wcs and so on)

Cheers
Andrea

--

Our support, Your Success! Visit http://opensdi.geo-solutions.it for more
information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------