So it looks like there's a couple things needed to get the directory datastore up to real production quality.
1) Proper reporting of suffix list.
If you make a new directory datastore, make a suffix list, and then go back to edit it again your suffix list shows up as: '[Ljava.lang.String;@ab4b89' The interesting things is that it still seems to work fine?
2) Relative paths
Everything else that one defines on a file system works with relative paths, so this should as well. Hopefully the fix is just needed in one place, maybe too. Though I know we make a few assumptions about 'url' being the one to try to make relative and all. But hopefully since this is also a file most of the other code will also work.
So it looks like there's a couple things needed to get the directory datastore up to real production quality.
1) Proper reporting of suffix list.
If you make a new directory datastore, make a suffix list, and then go back to edit it again your suffix list shows up as: '[Ljava.lang.String;@ab4b89' The interesting things is that it still seems to work fine?
We could probably do a reflective check for an array or collection and unwrap it if true. I cant think of any other data store parameters that are array or collection based.
2) Relative paths
Everything else that one defines on a file system works with relative paths, so this should as well. Hopefully the fix is just needed in one place, maybe too. Though I know we make a few assumptions about 'url' being the one to try to make relative and all. But hopefully since this is also a file most of the other code will also work.
Chris
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
So it looks like there's a couple things needed to get the directory datastore up to real production quality.
1) Proper reporting of suffix list.
If you make a new directory datastore, make a suffix list, and then go back to edit it again your suffix list shows up as: '[Ljava.lang.String;@ab4b89' The interesting things is that it still seems to work fine?
We could probably do a reflective check for an array or collection and unwrap it if true. I cant think of any other data store parameters that are array or collection based.
Makes sense to me. You want to take this one. I can (shudder) take the relative paths one.
2) Relative paths
Everything else that one defines on a file system works with relative paths, so this should as well. Hopefully the fix is just needed in one place, maybe too. Though I know we make a few assumptions about 'url' being the one to try to make relative and all. But hopefully since this is also a file most of the other code will also work.
Chris
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
So it looks like there's a couple things needed to get the directory datastore up to real production quality.
1) Proper reporting of suffix list.
If you make a new directory datastore, make a suffix list, and then go back to edit it again your suffix list shows up as: '[Ljava.lang.String;@ab4b89' The interesting things is that it still seems to work fine?
We could probably do a reflective check for an array or collection and unwrap it if true. I cant think of any other data store parameters that are array or collection based.
Makes sense to me. You want to take this one. I can (shudder) take the relative paths one.
Can do, I will get to it after i finish the estimated_extent for postgis.
2) Relative paths
Everything else that one defines on a file system works with relative paths, so this should as well. Hopefully the fix is just needed in one place, maybe too. Though I know we make a few assumptions about 'url' being the one to try to make relative and all. But hopefully since this is also a file most of the other code will also work.
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
So it looks like there's a couple things needed to get the directory datastore up to real production quality.
1) Proper reporting of suffix list.
If you make a new directory datastore, make a suffix list, and then go back to edit it again your suffix list shows up as: '[Ljava.lang.String;@ab4b89' The interesting things is that it still seems to work fine?
We could probably do a reflective check for an array or collection and unwrap it if true. I cant think of any other data store parameters that are array or collection based.
Makes sense to me. You want to take this one. I can (shudder) take the relative paths one.
Oh, relative paths... this is because we do the path transformation
only if the parameter is named url.... I guess there are a few places
to change (hopefully just one, but not sure about it).
Cheers
Andrea
So it looks like there's a couple things needed to get the directory datastore up to real production quality.
1) Proper reporting of suffix list.
If you make a new directory datastore, make a suffix list, and then go back to edit it again your suffix list shows up as: '[Ljava.lang.String;@ab4b89' The interesting things is that it still seems to work fine?
We could probably do a reflective check for an array or collection and unwrap it if true. I cant think of any other data store parameters that are array or collection based.
Makes sense to me. You want to take this one. I can (shudder) take the relative paths one.
Oh, relative paths... this is because we do the path transformation
only if the parameter is named url.... I guess there are a few places
to change (hopefully just one, but not sure about it).
Yeah, things look to be fairly encapsulated, indeed I saw some places where it looked to be redundant, I suppose this will be the test.
I was going to do file:data/relative_dir style, but I'm wondering if I should have it be dir:data/relative_dir ? Having it be 'file' I don't think actually gets me a ton of re-use.
Ughhh. Yes the fix I made turns it into an absolute path so when you go back to the editor that is what you will get. However everything works ok on my end. Are you sure you have an up to date geotools jar?
So looking in deeper, it appears that relative paths like this only will work in *some* datastores. Ones for which the following conditions are true:
1. the DataStoreFactory#canProcess(Map) does not do any checks for file existsance on the url or file
2. the name of the url parameter actually ends in "url"
DirectoryDataStoreFactory actually checks the url to make sure it exists and that it is a directory. And the DIRECTORY parameter does not end in url.
Removing the directory check and renaming the parameter makes it all work. I know that removing the check is a bit of a hack but since directory data store is pretty unsupported i guess it wont affect much. And also GeoServer still reports back an error if the url is not a directory so there is no loss from a usability standpoint. The only alternative I see would be to make the url processing code in GeoServer more robust, but at this point that will probably lead to more trouble then what its worth.
-Justin
Chris Holmes wrote:
Hmmmm... It's still not working for me. If I put in a relative link then it will turn it in to an absolute link in the form. But it won't go, it gets a not so helpful 'connection parameters are invalid' error. If I put in the full URL straight away I get the same result.
Logs reveal nothing...
It's definitely working on yours? If so I can restart and try to clean out some repositories.
Justin Deoliveira wrote:
Chris Holmes wrote:
Justin Deoliveira wrote:
Chris Holmes wrote:
So it looks like there's a couple things needed to get the directory datastore up to real production quality.
1) Proper reporting of suffix list.
If you make a new directory datastore, make a suffix list, and then go back to edit it again your suffix list shows up as: '[Ljava.lang.String;@ab4b89' The interesting things is that it still seems to work fine?
We could probably do a reflective check for an array or collection and unwrap it if true. I cant think of any other data store parameters that are array or collection based.
Makes sense to me. You want to take this one. I can (shudder) take the relative paths one.
Can do, I will get to it after i finish the estimated_extent for postgis.
2) Relative paths
Everything else that one defines on a file system works with relative paths, so this should as well. Hopefully the fix is just needed in one place, maybe too. Though I know we make a few assumptions about 'url' being the one to try to make relative and all. But hopefully since this is also a file most of the other code will also work.
Ughhh. Yes the fix I made turns it into an absolute path so when you go back to the editor that is what you will get. However everything works ok on my end. Are you sure you have an up to date geotools jar?
So looking in deeper, it appears that relative paths like this only will work in *some* datastores. Ones for which the following conditions are true:
1. the DataStoreFactory#canProcess(Map) does not do any checks for file existsance on the url or file
2. the name of the url parameter actually ends in "url"
DirectoryDataStoreFactory actually checks the url to make sure it exists and that it is a directory. And the DIRECTORY parameter does not end in url.
Removing the directory check and renaming the parameter makes it all work. I know that removing the check is a bit of a hack but since directory data store is pretty unsupported i guess it wont affect much.
Well, Jody's the mantainer of the directory data store, so that's his call. I guess we have a few possible paths in front of us:
* Jody allows for the changes to be made
* we ditch the directory data store and live happily without it
* we fork directory data store into ddsGsEdition and hack it without
mercy :-p
it seems we're in no condition to have Geoserver workaround its different behavior
Ughhh. Yes the fix I made turns it into an absolute path so when you go back to the editor that is what you will get. However everything works ok on my end. Are you sure you have an up to date geotools jar?
So looking in deeper, it appears that relative paths like this only will work in *some* datastores. Ones for which the following conditions are true:
1. the DataStoreFactory#canProcess(Map) does not do any checks for file existsance on the url or file
2. the name of the url parameter actually ends in "url"
DirectoryDataStoreFactory actually checks the url to make sure it exists and that it is a directory. And the DIRECTORY parameter does not end in url.
Removing the directory check and renaming the parameter makes it all work. I know that removing the check is a bit of a hack but since directory data store is pretty unsupported i guess it wont affect much.
Well, Jody's the mantainer of the directory data store, so that's his call. I guess we have a few possible paths in front of us:
* Jody allows for the changes to be made
Sure. Two things I would ask for (I know time is tight)
a) provide an example of correct use at the end of the day
b) Reduce conflicts between directory datastore and property datastore
(As is usual I have an idea of how we could solve this right across the board)
Can we make a "PathResolver" extension point / hint thing .. the default GeoTools implementation will just return the current directory; GeoServer can inject one based on the data directory etc...
* we ditch the directory data store and live happily without it
If you do I may just remove it; what it does would be better handled as a catalog. You can also find a Registry implementation that processes a directory and serves up the shapefiles used in the validation tests.
* we fork directory data store into ddsGsEdition and hack it without
Don't fork; hack.
mercy :-p
it seems we're in no condition to have Geoserver workaround its different behavior
Ughhh. Yes the fix I made turns it into an absolute path so when you go back to the editor that is what you will get. However everything works ok on my end. Are you sure you have an up to date geotools jar?
So looking in deeper, it appears that relative paths like this only will work in *some* datastores. Ones for which the following conditions are true:
1. the DataStoreFactory#canProcess(Map) does not do any checks for file existsance on the url or file
2. the name of the url parameter actually ends in "url"
DirectoryDataStoreFactory actually checks the url to make sure it exists and that it is a directory. And the DIRECTORY parameter does not end in url.
Removing the directory check and renaming the parameter makes it all work. I know that removing the check is a bit of a hack but since directory data store is pretty unsupported i guess it wont affect much.
Well, Jody's the mantainer of the directory data store, so that's his call. I guess we have a few possible paths in front of us:
* Jody allows for the changes to be made
Sure. Two things I would ask for (I know time is tight)
a) provide an example of correct use at the end of the day
b) Reduce conflicts between directory datastore and property datastore
(As is usual I have an idea of how we could solve this right across the board)
Can we make a "PathResolver" extension point / hint thing .. the default GeoTools implementation will just return the current directory; GeoServer can inject one based on the data directory etc...
* we ditch the directory data store and live happily without it
If you do I may just remove it; what it does would be better handled as a catalog. You can also find a Registry implementation that processes a directory and serves up the shapefiles used in the validation tests.
* we fork directory data store into ddsGsEdition and hack it without
Don't fork; hack.
mercy :-p
it seems we're in no condition to have Geoserver workaround its different behavior
Can my "PathResolver" idea work Andrea?
It may, yeah, but we should start treating directory data store
differently from the others. We don't have the infrastructure for
custom treatment (besides ifs in the code base, that would trigger
direct dependencies on that data store).
Justin hacks fixed the thing for the moment, no?
Cheers
Andrea
It may, yeah, but we should start treating directory data store
differently from the others. We don't have the infrastructure for
custom treatment (besides ifs in the code base, that would trigger
direct dependencies on that data store).
I think Jody meant more this would be a DataStore implementation thing. Like inject a file based datastore with a "PathResolver" so it could handle these types of paths directly and geoserver would not have to hack around it like we do now. I like the idea but it is way too much work to be useful to as this point in release cycle.
Justin hacks fixed the thing for the moment, no?
Yup, although it would be nice if someone else tested as well.
