[Geoserver-devel] FileWatcher, avoid unnecessary reloads

This is more a design question:

FileWatcher checks for external modifications of a file. But if tgeoserver code is writing the content of this file itself, how can I tell the FileWatcher NOT to indicate that the file is modified forcing an additional reload.

Proposals

1) add a method

setKnownLastModified(long lastModified)

2) add a method

write(T object)

which deserializes the object and adjusts the internal last modified timestamp.

Proposal 2 has another advantage. It is possible to detect conflicts if the file was changed externally and internally, throwing a proper IOException.

What do you think ?.

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

On Thu, Jun 9, 2011 at 6:37 AM, <christian.mueller@anonymised.com> wrote:

This is more a design question:

FileWatcher checks for external modifications of a file. But if
tgeoserver code is writing the content of this file itself, how can I
tell the FileWatcher NOT to indicate that the file is modified forcing
an additional reload.

Proposals

1) add a method

setKnownLastModified(long lastModified)

2) add a method

write(T object)

which deserializes the object and adjusts the internal last modified
timestamp.

Proposal 2 has another advantage. It is possible to detect conflicts
if the file was changed externally and internally, throwing a proper
IOException.

What do you think ?.

The first is probably more generic, does not require a case by case subclass,
but I'm wondering if this is an early optimization case.
Are the files in question so expensive to reload, or the expected
change rate of the file very high?

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

Please read below

Zitat von Andrea Aime <andrea.aime@anonymised.com>:

On Thu, Jun 9, 2011 at 6:37 AM, <christian.mueller@anonymised.com> wrote:

This is more a design question:

FileWatcher checks for external modifications of a file. But if
tgeoserver code is writing the content of this file itself, how can I
tell the FileWatcher NOT to indicate that the file is modified forcing
an additional reload.

Proposals

1) add a method

setKnownLastModified(long lastModified)

2) add a method

write(T object)

which deserializes the object and adjusts the internal last modified
timestamp.

Proposal 2 has another advantage. It is possible to detect conflicts
if the file was changed externally and internally, throwing a proper
IOException.

What do you think ?.

The first is probably more generic, does not require a case by case subclass,
but I'm wondering if this is an early optimization case.
Are the files in question so expensive to reload, or the expected
change rate of the file very high?

Cheers
Andrea

In my case it is reading an XML file of unknown size, parsing, building a DOM and validate the DOM against an XML Schema.

Another Idea which does not hurt:

class WirteableFileWatcher<T> extends FileWatcher<T> {
public write(T object) trhows IOException() {
      // code
     }
}

Cheers
Christian

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

On Thu, Jun 9, 2011 at 10:47 AM, <christian.mueller@anonymised.com> wrote:

Please read below

Zitat von Andrea Aime <andrea.aime@anonymised.com>:

On Thu, Jun 9, 2011 at 6:37 AM, <christian.mueller@anonymised.com> wrote:

This is more a design question:

FileWatcher checks for external modifications of a file. But if
tgeoserver code is writing the content of this file itself, how can I
tell the FileWatcher NOT to indicate that the file is modified forcing
an additional reload.

Proposals

1) add a method

setKnownLastModified(long lastModified)

2) add a method

write(T object)

which deserializes the object and adjusts the internal last modified
timestamp.

Proposal 2 has another advantage. It is possible to detect conflicts
if the file was changed externally and internally, throwing a proper
IOException.

What do you think ?.

The first is probably more generic, does not require a case by case
subclass,
but I'm wondering if this is an early optimization case.
Are the files in question so expensive to reload, or the expected
change rate of the file very high?

Cheers
Andrea

In my case it is reading an XML file of unknown size, parsing, building a
DOM and validate the DOM against an XML Schema.

Oh, I thought you were using xstream, which does not build a DOM,
uses a pull parser which is faster.

Another Idea which does not hurt:

class WirteableFileWatcher<T> extends FileWatcher<T> {
public write(T object) trhows IOException() {
// code
}
}

Still prefer the file watcher to be notified. Otherwise it's no
more a file watcher but an almost full persistence subsystem
(if you write, why not read too while you're at it?).
The persistence subsystem could be built using the file watcher
services as a delegate.

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

Zitat von Andrea Aime <andrea.aime@anonymised.com>:

On Thu, Jun 9, 2011 at 10:47 AM, <christian.mueller@anonymised.com> wrote:

Please read below

Zitat von Andrea Aime <andrea.aime@anonymised.com>:

On Thu, Jun 9, 2011 at 6:37 AM, <christian.mueller@anonymised.com> wrote:

This is more a design question:

FileWatcher checks for external modifications of a file. But if
tgeoserver code is writing the content of this file itself, how can I
tell the FileWatcher NOT to indicate that the file is modified forcing
an additional reload.

Proposals

1) add a method

setKnownLastModified(long lastModified)

2) add a method

write(T object)

which deserializes the object and adjusts the internal last modified
timestamp.

Proposal 2 has another advantage. It is possible to detect conflicts
if the file was changed externally and internally, throwing a proper
IOException.

What do you think ?.

The first is probably more generic, does not require a case by case
subclass,
but I'm wondering if this is an early optimization case.
Are the files in question so expensive to reload, or the expected
change rate of the file very high?

Cheers
Andrea

In my case it is reading an XML file of unknown size, parsing, building a
DOM and validate the DOM against an XML Schema.

Oh, I thought you were using xstream, which does not build a DOM,
uses a pull parser which is faster.

You have the choice,
XStream xstream = new XStream(new DomDriver());

Another Idea which does not hurt:

class WirteableFileWatcher<T> extends FileWatcher<T> {
public write(T object) trhows IOException() {
// code
}
}

Still prefer the file watcher to be notified. Otherwise it's no
more a file watcher but an almost full persistence subsystem
(if you write, why not read too while you're at it?).
The persistence subsystem could be built using the file watcher
services as a delegate.

Bought , should I prepare a patch for

setKnownLastModified(long lastModified)

Cheers

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

On Thu, Jun 9, 2011 at 11:25 AM, <christian.mueller@anonymised.com> wrote:

Oh, I thought you were using xstream, which does not build a DOM,
uses a pull parser which is faster.

You have the choice,
XStream xstream = new XStream(new DomDriver());

Yep, thought it's a discouraged approach (because it's slow)

Another Idea which does not hurt:

class WirteableFileWatcher<T> extends FileWatcher<T> {
public write(T object) trhows IOException() {
// code
}
}

Still prefer the file watcher to be notified. Otherwise it's no
more a file watcher but an almost full persistence subsystem
(if you write, why not read too while you're at it?).
The persistence subsystem could be built using the file watcher
services as a delegate.

Bought , should I prepare a patch for

setKnownLastModified(long lastModified)

Yep, go ahead

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

Last message here concerning building a DOM.
On reading I will use the pull parser and only validate if I get an exception. The xsd files
are in the GEOSERVER_DATA_DIR/security/..., users can user their own tool for validating
if the modify the xml files manually, I will point out this fact in the documentation.

On writing (GUI or hopefully REST in the future), I will always validate.

Cheers
Christian

Zitat von Andrea Aime <andrea.aime@anonymised.com>:

On Thu, Jun 9, 2011 at 11:25 AM, <christian.mueller@anonymised.com> wrote:

Oh, I thought you were using xstream, which does not build a DOM,
uses a pull parser which is faster.

You have the choice,
XStream xstream = new XStream(new DomDriver());

Yep, thought it's a discouraged approach (because it's slow)

Another Idea which does not hurt:

class WirteableFileWatcher<T> extends FileWatcher<T> {
public write(T object) trhows IOException() {
// code
}
}

Still prefer the file watcher to be notified. Otherwise it's no
more a file watcher but an almost full persistence subsystem
(if you write, why not read too while you're at it?).
The persistence subsystem could be built using the file watcher
services as a delegate.

Bought , should I prepare a patch for

setKnownLastModified(long lastModified)

Yep, go ahead

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.