On 23-07-15 17:53, Jody Garnett wrote:
Have a look at the examples I linked to. We need a second QA pass through
the codebase to remove references to files (and deprecate methods like
getOrCreateDirectory). For the first pass I just implemented them all in
terms of Resource - but any code that asks for a file will get one
(unpacking the blob from the resource store if needed).
I'm currently trying to implement that changes to the cached files are
noticed by a file-watcher and then written to the database, it's
technically possible but that seems quite dodgy and error prone to me. It
is fine as a fall-back mechanism for backwards compatibility and it allows
us to gradually change all the modules rather than all at once, but I would
strongly discourage it and deprecate those methods in the API.
I implemented replacement watchers as part of the migration? Are they not
working for you ...
Right now we actually have lots of code that just asks one time for one
directory in the datadir and then uses the file system from there
onwards....
And that code should still work, the directory will get unpacked - but it
is not any kind of fun.
Either way, I still find it strange in the API that we can have a resource
that represent a directory, but no way to create such a thing, unless by
creating a file inside it. This was my concern, and I haven't been 100%
convinced it makes sense.
Let me know how you go, you can still list the contents and so on ...
You can do:
Resource fileRes = store.get("/some/path/to/a/file");
fileRes.out() //creates the file and the directories "some", "path",
"to", and "a" if necessary
Okay, that is convenient. But what about this:
Resource dirRes = fileRes.getParent();
Type type = dirRes.getType(); //returns dirRes.getDirectory()
Cool. But imagine now doing this:
Resource dirRes = store.get("/some/path/to/a/dir");
//there is no way to simply create this path as a directory in the
store!
//note the dirRes.dir() always creates a directory on the _file system_
(caching it if necessary), a method we wish to deprecate!
You statement is a bit circular. Yes you can use dir() to create the folder
on the file system if needed (just like you can use file() to create a
file). Is there anything else you need that is not provided?
Like who cares if the directory is created or not, as long as it behaves
the same... It is up to your store implementation if you want an internal
representation of a directory or not. I think you can create it in a lazy
fashion as needed while respecting the api ...
Is directory creation or not just an internal detail?
Yes, of course, technically we could modify all the code to create
configuration files directly by their full path, and ignore directories
altogether. But then why is there such a thing as a Resource of type
directory?
To list contents, rename, etc...
For the sake of convenience you could want to work with a directory
resource to query and modify its children. For example :
class MyClass {
Resource myDir;
void myMethod() {
... //I need to do something depending on all the files inside our
directory
List<Resource> children = myDir.list() ...
...
}
MyClass() {
//initialize our config dir
myDir = store.get("/my_config_dir");
if (myDir.getType() == Type.UNDEFINED) {
//what now???
//there is no way to create this directory unless we
create a file inside it. but what if we don't need to yet at this stage?
//create a dummy file == ugly
}
}
}
Okay I see your concern. If you list() an undefined resource you get an
empty list .. can you use a UNDEFINED resource and trust it to behave as a
directory for your code?
Resource myDir;
void myMethod() {
... //I need to do something depending on all the files inside our
directory
List<Resource> children = myDir.list() ...
...
}
init() {
// initialize our config dir
myDir = store.get("/my_config_dir");
if (myDir.getType() == Type.UNDEFINED) {
// empty configuration, directory will be created during
first save()
}
else {
// configuration exists, work with myDir.list() to access
contents...
}
}
}
This would help if you can hunt down an example from the codebase 
Especially if you want to replace FileWatch ...
See how it can be inconvenient that there is no explicit directory creating
mechanism?
We can wait to hear from kevin, but I am not sure he is too concerned
about how JDBCResourceStore takes shape. We went over a couple different
designs (including placeholder entries for directories, so that resources
could link to their "parent").
That seems to be the design indeed.
What is you idea with Paths? Use a like filter to shortlist directory
contents for Resource.list() ?
The design of the database does not need to be changed necessarily, but
definitely the design of the code. The resource object needs to hold the
path, not the ID. Otherwise, it will not be possible to hold use a resource
object for longer than a single method, and even then it is not concurrency
safe.
However, the efficiency of the database design might indeed be questioned
if we change the design of the code. In this design each method would need
find the record by recursively looping through the path to find the
resource. Is that so different from how files query the file system though?
I'm not sure. Advise regarding this question is welcome.
I am not close enough to the code to know specifically what you are talking
about.
Okay, understood. Fair enough, as long as they are not used to write data, and preferably not when not necessary but you agree with that. Yes I know, but again, I was talking about creating a directory in the store (which may or may not be a FS or a DB) and not on the FS. Anyway, let’s finish this on chat. Regards Niels