[Geoserver-devel] Live backup and restore and concurrency issues

Andrea_Aime4 · November 9, 2011, 10:12am

Hi,
me and Alessio are looking for a way to perform a live backup and restore
of the GeoServer data directory without having to stop the instance.

Now, there is a number of issues to be faced but let me start with the
major one: concurrent catalog modifications.
During a backup nothign should be able to write, during a restore nothing
should be able to even read.

Actually the same problem occurs in GeoServer today in other occasions
and I have the impression it's also the cause of those null pointer exceptions
that are getting reported on the user list:
- concurrent admin operations, either via GUI or REST (hello Justin and his
multi-admin proposal)
- concurrent reload/reset operations (eventually with other stuff going on)

Now, if every single configuration bit was in a db and we had a
transaction api for
config changes this would not be a problem, but we're not there yet,
and that would require a very significant investment that we don't have
the funding for, so we are going to propose a simpler solution instead.

What about a global configuration read/write lock? GS 1.7 used to have it.

At the very least GUI and REST config would take a read lock when accessing
configuration data, and a write lock every time they are about to write on
the catalog, to protect the whole sequence of oprations and the associated
side effects happening via catalog/configuration listeners.
Reload would get a write lock, reset a read one.
The backup operation (probably implemented via rest) a read one, the restore
one a write one.

OGC operations might also get a read lock, that would prevent at least part
of the reasons why the stores die on their back while they are running
(the other would be the lru cache pushing out the stores, that's another story).
However I'm concerned that a long WFS download or a long WPS request
might make the configuration untouchable for a very long time.
So we might want to let these fail in face of configuration changes.

Btw, backup wise we'd like to backup the entire data directory (data, fonts,
styling icons and whatnot included), and restore
would wipe it out, replace it with the new contentes, and force a reload.

GWC is actually putting a wrench in this plan since it keeps a database open,
wondering if there is any way to have it give up on the db connections
for some time?

As an alternative we could have the backup and restore tools work on
selected directories
and files, configurable in the request (which could be a post) and
with some defaults
that only include the gwc xml config file, I guess GWC would
just reconfigure itself after the reload and wipe out the caches, right?

Anyways... opinions? Other things that might go pear shaped that we
did not account for?

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

Gabriel_Roldan2 · November 9, 2011, 2:29pm

Hey Andrea,

interesting issue.

Just as a heads up, so that some of the comments bellow make more
sense, I'm starting to investigate on what it would take for a
scalable catalog/configuration backend, possibly with HA and on a
cluster environment.
The only I know for sure now is that I want to leverage the
infrastructure already in place in terms of catalog objects
marshaling/unmarshaling.
That is, the back end, either a rdbms or a key/value db would just
need to be able of storing the serialized catalog objects in the same
way we store them now, in some sort of clob, and provide indexes for
the common queries (id and name mostly, given the current queries the
catalog performs).
That is because the hibernate catalog strikes me as overkill
complexity for a rather simple issue, and there's just too much
knowledge and effort put on the xstream persistence that I think it
would be nice to leverage it.

On Wed, Nov 9, 2011 at 7:12 AM, Andrea Aime
<andrea.aime@anonymised.com> wrote:

Hi,
me and Alessio are looking for a way to perform a live backup and restore
of the GeoServer data directory without having to stop the instance.

Now, there is a number of issues to be faced but let me start with the
major one: concurrent catalog modifications.
During a backup nothign should be able to write, during a restore nothing
should be able to even read.

So I think the easiest way of preventing that is to serialize write
access to the catalog, which is basically what you're proposing.
An alternate approach could be based on commands instead of the client
code being responsible of explicitly acquiring and releasing the
locks.
There's the usual case, for example, of adding a new layer from
scratch, where perhaps a StoreInfo needs to be added, as well as a
ResourceInfo, a LayerInfo, and a StyleInfo. Or the cascade delete of a
StoreInfo, for instance.

So my idea would be to encapsulate all the locking logic inside the
catalog, so that client code doesn't need to worry about acquiring a
read lock, but any operation that does write lock would inherently
lock read operations until done. Internally it could have a queue of
commands, and the client code that needs to do multiple operations
like in the examples above, would need to issue a command on the
catalog. The call would be synchronous, but the catalog would put the
command on the queue and wait for it to finish before returning.

I think this model would also make it easier to implement any message
passing needed when on a clustered environment, like in acquiring a
cluster wide write lock, or notifying other nodes of a configuration
change so that they release stuff from their own resource pool, etc.

Actually the same problem occurs in GeoServer today in other occasions
and I have the impression it's also the cause of those null pointer exceptions
that are getting reported on the user list:
- concurrent admin operations, either via GUI or REST (hello Justin and his
multi-admin proposal)
- concurrent reload/reset operations (eventually with other stuff going on)

Now, if every single configuration bit was in a db and we had a
transaction api for
config changes this would not be a problem, but we're not there yet,
and that would require a very significant investment that we don't have
the funding for, so we are going to propose a simpler solution instead.

What about a global configuration read/write lock? GS 1.7 used to have it.

At the very least GUI and REST config would take a read lock when accessing
configuration data, and a write lock every time they are about to write on
the catalog, to protect the whole sequence of oprations and the associated
side effects happening via catalog/configuration listeners.
Reload would get a write lock, reset a read one.
The backup operation (probably implemented via rest) a read one, the restore
one a write one.

OGC operations might also get a read lock, that would prevent at least part
of the reasons why the stores die on their back while they are running
(the other would be the lru cache pushing out the stores, that's another story).
However I'm concerned that a long WFS download or a long WPS request
might make the configuration untouchable for a very long time.
So we might want to let these fail in face of configuration changes.

agreed. The lru expiration being a separate issue, I think it's
generally fair to let an ongoing operation fail in face of a
configuration change that affects the resource(s) it's using (like in
a wfs request serving a feature type that's removed meanwhile).
Otherwise we would be making too much effort for a use case that we
don't even know is the _correct_ one. Perhaps the admin shuts down
that resource exactly due to client abuse and wants to stop any usage
of it immediately, but we would be preventing that. Anyway, just
thinking out loud on this regard

Btw, backup wise we'd like to backup the entire data directory (data, fonts,
styling icons and whatnot included), and restore
would wipe it out, replace it with the new contentes, and force a reload.

GWC is actually putting a wrench in this plan since it keeps a database open,
wondering if there is any way to have it give up on the db connections
for some time?

There could be, just lets sync up on what's needed, what the workflow
would be, and lets make it possible.
Would you be backing up the tile caches too?

As an alternative we could have the backup and restore tools work on
selected directories
and files, configurable in the request (which could be a post) and
with some defaults
that only include the gwc xml config file, I guess GWC would
just reconfigure itself after the reload and wipe out the caches, right?

oh I see, makes sense. Yeah, it should reconfigure itself.

Anyways... opinions? Other things that might go pear shaped that we
did not account for?

I think it's an issue worth taking the time to get it right.
Fortunately we do have the funding to do so now, starting firmly two
weeks from now as I'll be on vacation for the next two weeks, so I'd
be glad to run the whole process, exploratory prototyping included, by
the community.

Do you have a rush to get something working asap? or would be ok in
going for a larger but steady devel cycle?

Best regards,
Gabriel

Andrea_Aime4 · November 9, 2011, 2:59pm

On Wed, Nov 9, 2011 at 3:29 PM, Gabriel Roldan <groldan@anonymised.com> wrote:

Hey Andrea,

interesting issue.

Just as a heads up, so that some of the comments bellow make more
sense, I'm starting to investigate on what it would take for a
scalable catalog/configuration backend, possibly with HA and on a
cluster environment.

Interesting, and very much welcomed.

This work of ours is unfortunately on a short deadline, and needs
to land on the stable series.
This is why I was trying to suggest something on the low complexity
and low risk.
The deadline is something I cannot move so the other option is to
keep an internal fork of GeoServer for this project and/or enough time to
let the GS api evolve enough to allow this use case to be handled
by a nosql solution (we have no requirement of contributing back the
work in this project).

I have the impression your work is going to be longer term and trunk
only, but I may be wrong.

The only I know for sure now is that I want to leverage the
infrastructure already in place in terms of catalog objects
marshaling/unmarshaling.
That is, the back end, either a rdbms or a key/value db would just
need to be able of storing the serialized catalog objects in the same
way we store them now, in some sort of clob, and provide indexes for
the common queries (id and name mostly, given the current queries the
catalog performs).
That is because the hibernate catalog strikes me as overkill
complexity for a rather simple issue, and there's just too much
knowledge and effort put on the xstream persistence that I think it
would be nice to leverage it.

I can relate to that, indeed a relational dbms seems overkill for the
task at hand, a XML db or nosql db could work as well I guess.
However in some places a relational dbms (thinking Oracle here)
is the only thing allowed to store data, by
policy, so we should also leave the door open to that option to
be developed too, in the future of course.

So my idea would be to encapsulate all the locking logic inside the
catalog, so that client code doesn't need to worry about acquiring a
read lock, but any operation that does write lock would inherently
lock read operations until done. Internally it could have a queue of
commands, and the client code that needs to do multiple operations
like in the examples above, would need to issue a command on the
catalog. The call would be synchronous, but the catalog would put the
command on the queue and wait for it to finish before returning.

I think this model would also make it easier to implement any message
passing needed when on a clustered environment, like in acquiring a
cluster wide write lock, or notifying other nodes of a configuration
change so that they release stuff from their own resource pool, etc.

Sounds like a reasonable approach. So the command would be a transaction,
and the code calling it would have to build one for each of its specific needs,
and send it down.
It seems it would make for a 180 shift in how the catalog api works though,
with calling code have to be redone to use commands any time a transaction
is needed.
However... a typical command pattern would allow for just a write lock, allowing
reads to flow uncontrolled.
For example, during a reload I'm not sure we want the GUI or rest
config to access
the catalog, even read only, as they would get incosistent information.
I guess we could attach a lock type to the command?

agreed. The lru expiration being a separate issue, I think it's
generally fair to let an ongoing operation fail in face of a
configuration change that affects the resource(s) it's using (like in
a wfs request serving a feature type that's removed meanwhile).
Otherwise we would be making too much effort for a use case that we
don't even know is the _correct_ one. Perhaps the admin shuts down
that resource exactly due to client abuse and wants to stop any usage
of it immediately, but we would be preventing that. Anyway, just
thinking out loud on this regard

Yep, I agree.

Btw, backup wise we'd like to backup the entire data directory (data, fonts,
styling icons and whatnot included), and restore
would wipe it out, replace it with the new contentes, and force a reload.

GWC is actually putting a wrench in this plan since it keeps a database open,
wondering if there is any way to have it give up on the db connections
for some time?

There could be, just lets sync up on what's needed, what the workflow
would be, and lets make it possible.

I guess I would just need a way to tell GWC

Would you be backing up the tile caches too?

I don't think so, they are often just too big and can be rebuilt.
However that idea of having a configurable set of subdirs to be backed
up would allow complete or partial tile cache backup (don't know, maybe
the admin is aware that a particular layer is extremely expensive to re-create)

As an alternative we could have the backup and restore tools work on
selected directories
and files, configurable in the request (which could be a post) and
with some defaults
that only include the gwc xml config file, I guess GWC would
just reconfigure itself after the reload and wipe out the caches, right?

oh I see, makes sense. Yeah, it should reconfigure itself.

Anyways... opinions? Other things that might go pear shaped that we
did not account for?

I think it's an issue worth taking the time to get it right.

I would agree in a perfect world, unfortunately we work in one where
time

Fortunately we do have the funding to do so now, starting firmly two
weeks from now as I'll be on vacation for the next two weeks, so I'd
be glad to run the whole process, exploratory prototyping included, by
the community.

Do you have a rush to get something working asap? or would be ok in
going for a larger but steady devel cycle?

Unfortunately not, I have a week at most. Maybe the locking could be
handled at a very high level, so that it's not obstrusive and can be easily
removed later.
For example, at the request filter level, if a request is GUI and the page is
reserved to admins just do a write lock, if it's REST go and lock read if
the request is a get, write otherwise.
Actually this way I could avoid a full blown fork, but just use standard +
custom plugins of sorts.
The downside is that the same problem is afflicting the user base, especially
heavy users of rest config or simply people having multiple people using
the admin GUI, and the solution would not be provided to anyone
else but our customer...

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

Andrea_Aime4 · November 9, 2011, 3:22pm

It seems I left a couple of sentences unfishied. Amending.

There could be, just lets sync up on what's needed, what the workflow
would be, and lets make it possible.

I guess I would just need a way to tell GWC

that I need it to let go of everything, basically a shut down of sorts,
and restart fresh with the new fs contents.

I would agree in a perfect world, unfortunately we work in one where
time

... is tirant.

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

Justin_Deoliveira · November 9, 2011, 3:27pm

Interesting stuff.

How “constrained” can we keep the global lock approach? Will locking code and logic be scattered throughout the various systems? I am worried about the strategy not being able to be easily disabled in cases where we want to adopt an alterate strategy, or support other catalog/configuration implementations that natively support transactions, etc…

On Wed, Nov 9, 2011 at 8:22 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

It seems I left a couple of sentences unfishied. Amending.

There could be, just lets sync up on what’s needed, what the workflow
would be, and lets make it possible.

I guess I would just need a way to tell GWC

that I need it to let go of everything, basically a shut down of sorts,
and restart fresh with the new fs contents.

I would agree in a perfect world, unfortunately we work in one where
time

… is tirant.

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Justin_Deoliveira · November 9, 2011, 3:42pm

On Wed, Nov 9, 2011 at 7:59 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Wed, Nov 9, 2011 at 3:29 PM, Gabriel Roldan <groldan@anonymised.com> wrote:

Hey Andrea,

interesting issue.

Just as a heads up, so that some of the comments bellow make more
sense, I’m starting to investigate on what it would take for a
scalable catalog/configuration backend, possibly with HA and on a
cluster environment.

Interesting, and very much welcomed.

This work of ours is unfortunately on a short deadline, and needs
to land on the stable series.
This is why I was trying to suggest something on the low complexity
and low risk.
The deadline is something I cannot move so the other option is to
keep an internal fork of GeoServer for this project and/or enough time to
let the GS api evolve enough to allow this use case to be handled
by a nosql solution (we have no requirement of contributing back the
work in this project).

I have the impression your work is going to be longer term and trunk
only, but I may be wrong.

The only I know for sure now is that I want to leverage the
infrastructure already in place in terms of catalog objects
marshaling/unmarshaling.
That is, the back end, either a rdbms or a key/value db would just
need to be able of storing the serialized catalog objects in the same
way we store them now, in some sort of clob, and provide indexes for
the common queries (id and name mostly, given the current queries the
catalog performs).
That is because the hibernate catalog strikes me as overkill
complexity for a rather simple issue, and there’s just too much
knowledge and effort put on the xstream persistence that I think it
would be nice to leverage it.

I can relate to that, indeed a relational dbms seems overkill for the
task at hand, a XML db or nosql db could work as well I guess.
However in some places a relational dbms (thinking Oracle here)
is the only thing allowed to store data, by
policy, so we should also leave the door open to that option to
be developed too, in the future of course.

So my idea would be to encapsulate all the locking logic inside the
catalog, so that client code doesn’t need to worry about acquiring a
read lock, but any operation that does write lock would inherently
lock read operations until done. Internally it could have a queue of
commands, and the client code that needs to do multiple operations
like in the examples above, would need to issue a command on the
catalog. The call would be synchronous, but the catalog would put the
command on the queue and wait for it to finish before returning.

I think this model would also make it easier to implement any message
passing needed when on a clustered environment, like in acquiring a
cluster wide write lock, or notifying other nodes of a configuration
change so that they release stuff from their own resource pool, etc.

Sounds like a reasonable approach. So the command would be a transaction,
and the code calling it would have to build one for each of its specific needs,
and send it down.
It seems it would make for a 180 shift in how the catalog api works though,
with calling code have to be redone to use commands any time a transaction
is needed.
However… a typical command pattern would allow for just a write lock, allowing
reads to flow uncontrolled.
For example, during a reload I’m not sure we want the GUI or rest
config to access
the catalog, even read only, as they would get incosistent information.
I guess we could attach a lock type to the command?

agreed. The lru expiration being a separate issue, I think it’s
generally fair to let an ongoing operation fail in face of a
configuration change that affects the resource(s) it’s using (like in
a wfs request serving a feature type that’s removed meanwhile).
Otherwise we would be making too much effort for a use case that we
don’t even know is the correct one. Perhaps the admin shuts down
that resource exactly due to client abuse and wants to stop any usage
of it immediately, but we would be preventing that. Anyway, just
thinking out loud on this regard

Yep, I agree.

Btw, backup wise we’d like to backup the entire data directory (data, fonts,
styling icons and whatnot included), and restore
would wipe it out, replace it with the new contentes, and force a reload.

GWC is actually putting a wrench in this plan since it keeps a database open,
wondering if there is any way to have it give up on the db connections
for some time?
There could be, just lets sync up on what’s needed, what the workflow
would be, and lets make it possible.

I guess I would just need a way to tell GWC

Would you be backing up the tile caches too?

I don’t think so, they are often just too big and can be rebuilt.
However that idea of having a configurable set of subdirs to be backed
up would allow complete or partial tile cache backup (don’t know, maybe
the admin is aware that a particular layer is extremely expensive to re-create)

As an alternative we could have the backup and restore tools work on
selected directories
and files, configurable in the request (which could be a post) and
with some defaults
that only include the gwc xml config file, I guess GWC would
just reconfigure itself after the reload and wipe out the caches, right?
oh I see, makes sense. Yeah, it should reconfigure itself.

Anyways… opinions? Other things that might go pear shaped that we
did not account for?

I think it’s an issue worth taking the time to get it right.

I would agree in a perfect world, unfortunately we work in one where
time

Fortunately we do have the funding to do so now, starting firmly two
weeks from now as I’ll be on vacation for the next two weeks, so I’d
be glad to run the whole process, exploratory prototyping included, by
the community.

Do you have a rush to get something working asap? or would be ok in
going for a larger but steady devel cycle?

Unfortunately not, I have a week at most. Maybe the locking could be
handled at a very high level, so that it’s not obstrusive and can be easily
removed later.
For example, at the request filter level, if a request is GUI and the page is
reserved to admins just do a write lock, if it’s REST go and lock read if
the request is a get, write otherwise.
Actually this way I could avoid a full blown fork, but just use standard +
custom plugins of sorts.
The downside is that the same problem is afflicting the user base, especially
heavy users of rest config or simply people having multiple people using
the admin GUI, and the solution would not be provided to anyone
else but our customer…

Missed this the first time around While not ideal I think this will be the best approach for now (keep it constrained) until funding/mandate allows us to move toward a more comprehensive solution at the catalog level.

$0.02

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Andrea_Aime4 · November 9, 2011, 4:29pm

Missed this the first time around While not ideal I think this will be
the best approach for now (keep it constrained) until funding/mandate allows
us to move toward a more comprehensive solution at the catalog level.
$0.02

Yep yep, I fully agree, especially since it seems we might get one for the
2.2.0 series.

So I looked around a bit and it might be that we have it working with
coarse locks
based solely on pluggable callbacks:
* Rest wise I guess I can just plugin a rest callback that checks the current
  request method and impose a lock consequently, put/post/delete would raise
  a global write lock, all other methods a read one.
* OGC wise I could have a dispatche callback that looks for the WPS
  ImportProcess, which is the only thing I know of modifying the configuration.
  Maybe have that be configurable somehow, with a list of processes that need
  write locking
* GUI wise we need a new extension point, some sort of WicketCallback.

I noticed that I can change a small bit of GeoServerAppliaction code,
in particular
the RequestCycle class, to spot an admin page:

static class RequestCycle extends WebRequestCycle {
...

        @Override
        protected void onRequestTargetSet(IRequestTarget target) {
            super.onRequestTargetSet(target);
            if(target instanceof IPageRequestTarget) {
                IPageRequestTarget pt = (IPageRequestTarget) target;
                Page page = pt.getPage();
                if(page instanceof GeoServerSecuredPage) {
                    System.out.println("Hey, this is a secured page");
                }
            } else if(target instanceof IBookmarkablePageRequestTarget) {
                IBookmarkablePageRequestTarget bt =
(IBookmarkablePageRequestTarget) target;

if(GeoServerSecuredPage.class.isAssignableFrom(bt.getPageClass())) {
System.out.println("Hey, this is a secured page");
}

}
}

...
}

If we want this to be pluggable we could use the onXYZ methods in the
wicket request cycle to build a sort of WicketCallback, that the
RequestCallback would call to.

This would make the locking code easy to remove by commenting out from
Spring. And/or we could make it possible to turn off in a deplyment by
using a system variable.

If in the future all config backends support transaction we'd just wipe it out.

Would this be acceptable, even for the stable series?

Also, would this have to be part of its own extension plugin, be part of core
and spread out in the relevant modules, be part of core but be its own
plugin module that only web-app would depend to?

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

Gabriel_Roldan2 · November 9, 2011, 7:05pm

Hi Andrea,

as long as it works and it's clear how to disable (at least for
testing without it) I'm ok with your proposal getting into both 2.1.x
and 2.2.x _now_, taking into account for 2.2.x we'll hopefully come up
with a solution more general/less invasive, or whatever QA requirement
we assess we need to push for to enable scalability and HA.

Cheers!
Gabriel

On Wed, Nov 9, 2011 at 1:29 PM, Andrea Aime
<andrea.aime@anonymised.com> wrote:

Missed this the first time around While not ideal I think this will be
the best approach for now (keep it constrained) until funding/mandate allows
us to move toward a more comprehensive solution at the catalog level.
$0.02

Yep yep, I fully agree, especially since it seems we might get one for the
2.2.0 series.

So I looked around a bit and it might be that we have it working with
coarse locks
based solely on pluggable callbacks:
* Rest wise I guess I can just plugin a rest callback that checks the current
request method and impose a lock consequently, put/post/delete would raise
a global write lock, all other methods a read one.
* OGC wise I could have a dispatche callback that looks for the WPS
ImportProcess, which is the only thing I know of modifying the configuration.
Maybe have that be configurable somehow, with a list of processes that need
write locking
* GUI wise we need a new extension point, some sort of WicketCallback.

I noticed that I can change a small bit of GeoServerAppliaction code,
in particular
the RequestCycle class, to spot an admin page:

static class RequestCycle extends WebRequestCycle {
...
   @anonymised\.com
   protected void onRequestTargetSet$IRequestTarget target$ \{
       super\.onRequestTargetSet$target$;
       if$target instanceof IPageRequestTarget$ \{
           IPageRequestTarget pt = $IPageRequestTarget$ target;
           Page page = pt\.getPage;
           if$page instanceof GeoServerSecuredPage$ \{
               System\.out\.println$&quot;Hey, this is a secured page&quot;$;
           \}
       \} else if$target instanceof IBookmarkablePageRequestTarget$ \{
           IBookmarkablePageRequestTarget bt =
(IBookmarkablePageRequestTarget) target;

if(GeoServerSecuredPage.class.isAssignableFrom(bt.getPageClass())) {
System.out.println("Hey, this is a secured page");
}
       \}
   \}

   \.\.\.
}

If we want this to be pluggable we could use the onXYZ methods in the
wicket request cycle to build a sort of WicketCallback, that the
RequestCallback would call to.

This would make the locking code easy to remove by commenting out from
Spring. And/or we could make it possible to turn off in a deplyment by
using a system variable.

If in the future all config backends support transaction we'd just wipe it out.

Would this be acceptable, even for the stable series?

Also, would this have to be part of its own extension plugin, be part of core
and spread out in the relevant modules, be part of core but be its own
plugin module that only web-app would depend to?

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.

Justin_Deoliveira · November 10, 2011, 3:20pm

On Wed, Nov 9, 2011 at 9:29 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Missed this the first time around While not ideal I think this will be
the best approach for now (keep it constrained) until funding/mandate allows
us to move toward a more comprehensive solution at the catalog level.
$0.02

Yep yep, I fully agree, especially since it seems we might get one for the
2.2.0 series.

So I looked around a bit and it might be that we have it working with
coarse locks
based solely on pluggable callbacks:

Rest wise I guess I can just plugin a rest callback that checks the current
request method and impose a lock consequently, put/post/delete would raise
a global write lock, all other methods a read one.

OGC wise I could have a dispatche callback that looks for the WPS
ImportProcess, which is the only thing I know of modifying the configuration.
Maybe have that be configurable somehow, with a list of processes that need
write locking

GUI wise we need a new extension point, some sort of WicketCallback.

I noticed that I can change a small bit of GeoServerAppliaction code,
in particular
the RequestCycle class, to spot an admin page:

static class RequestCycle extends WebRequestCycle {
…

@anonymised.com
protected void onRequestTargetSet(IRequestTarget target) {
super.onRequestTargetSet(target);
if(target instanceof IPageRequestTarget) {
IPageRequestTarget pt = (IPageRequestTarget) target;
Page page = pt.getPage();
if(page instanceof GeoServerSecuredPage) {
System.out.println(“Hey, this is a secured page”);
}
} else if(target instanceof IBookmarkablePageRequestTarget) {
IBookmarkablePageRequestTarget bt =
(IBookmarkablePageRequestTarget) target;

if(GeoServerSecuredPage.class.isAssignableFrom(bt.getPageClass())) {
System.out.println(“Hey, this is a secured page”);
}

}
}

…
}

If we want this to be pluggable we could use the onXYZ methods in the
wicket request cycle to build a sort of WicketCallback, that the
RequestCallback would call to.

This would make the locking code easy to remove by commenting out from
Spring. And/or we could make it possible to turn off in a deplyment by
using a system variable.

If in the future all config backends support transaction we’d just wipe it out.

Would this be acceptable, even for the stable series?

Works for me.

Also, would this have to be part of its own extension plugin, be part of core
and spread out in the relevant modules, be part of core but be its own
plugin module that only web-app would depend to?

I would prefer an extension, since those are trivial to disable… but if its turns out to be too much work to do it that way then core is fine, as long as its relatively simple to disable by commenting out a few lines of spring, etc…

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Andrea_Aime4 · November 10, 2011, 3:40pm

On Thu, Nov 10, 2011 at 4:20 PM, Justin Deoliveira <jdeolive@anonymised.com.1501…> wrote:

Would this be acceptable, even for the stable series?

Works for me.

Also, would this have to be part of its own extension plugin, be part of core
and spread out in the relevant modules, be part of core but be its own
plugin module that only web-app would depend to?

I would prefer an extension, since those are trivial to disable… but if its turns out to be too much work to do it that way then core is fine, as long as its relatively simple to disable by commenting out a few lines of spring, etc…

It would be a “make GS work in face of concurrent changes” extension (the locking code I mean)
Or you were talking about the backup/restore code?

What about the locking code in core with a system variable to disable
it and backup/restore as an extension or community module?

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

Justin_Deoliveira · November 10, 2011, 8:08pm

On Thu, Nov 10, 2011 at 8:40 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Thu, Nov 10, 2011 at 4:20 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Would this be acceptable, even for the stable series?

Works for me.

Also, would this have to be part of its own extension plugin, be part of core
and spread out in the relevant modules, be part of core but be its own
plugin module that only web-app would depend to?

I would prefer an extension, since those are trivial to disable… but if its turns out to be too much work to do it that way then core is fine, as long as its relatively simple to disable by commenting out a few lines of spring, etc…

It would be a “make GS work in face of concurrent changes” extension (the locking code I mean)
Or you were talking about the backup/restore code?

What about the locking code in core with a system variable to disable
it and backup/restore as an extension or community module?

Sounds good to me.

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.