RE: [Geoserver-devel] Ingestion Engine

This "ingestion-engine" sounds interesting.

From what I understand, it appears that you want to be able to manage
the Geoserver outside of the web-admin tool. Specifically, you want to
be able to setup a bunch of directories and have the configuration
auto-magically derived from this.

So, what I see from this is a system ("the ingestion engine" that would
notice that the directory structure has changed, then look at it and
make a new geoserver configuration (ie. geoserver .xml file(s)). It
would then tell geoserver to re-load the configuration.

From this perspective, you can see the geoserver configuration (ie. its
XML files) and their management are independent. More specifically, you
can view the struts web-admin and the "ingestion engine" as two
different ways of making configuration files -- geoserver will not be
able to see the difference.

Take a look at the picture attached. If we abstract the configuration
into either a set of services or XML files (or both), we can have
multiple ways of viewing and updating the information.

I really like this, because I want to see the configuration system
redone - at least the struts web-app, but I'm sure with the 2.0
geoserver-web-application-platform will need some fundamental
configuration (ie. XML) changes. There are several JIRA tasks for
improvements.

I'd really like to see the current Struts-based codebase be improved -
one way to do this could be to keep the old one and slowly introduce a
new one (perhaps not even built with struts). That way, if there's any
stability (or whatever) issues with it, people could just use the old
one. This provides an extremely low-risk upgrade path that we can
treat more like the cathedral-and-bazaar model (since we do not have to
build the entire cathedral before we release it).

For people using the "ingestion Engine", they would use the web-app only
to look at configuration instead of make any modifications (the
modifications will be done by the ingestion engine).

So, what I'm trying to say is this:

The configuration stuff is going to be changing over time, and we want
to be able to support multiple configuration systems (ie. web apps and
auto-generators).

I think this means two things:

a) we need to make sure the base configuration (ie. its XML files and
perhaps a few services) is "good enough" for where we're going in the
future.
b) the systems that will be making or managing the configurations have
to designed in such a way that they can handle the base configuration
XML files changing a bit over time (so we can improve #1).

I think the minimum amount of work we can do for this is to have a
geoserver "service" (ie. a request you can send to geoserver) that says
"please reload your configuration" or "please load your configuration
from here". For starters, we can make this service the same as
pressing the "load" button on the current web application.

I also want to warn you (actually, invite you to help) that the
underlying configuration (XML/services) will be changing over time as
we make geoserver more into an application framework with the ability
to have many more services (like the ones you've been working on).

So, the part of the "ingestion engine" that actually writes out a
configuration (XML) needs to be written in such a way that its easy to
change.

Does this make sense? I'm not 100% sure exactly what you're trying to
do - I'm trying to make your job easier (for the long run) plus make
future people's jobs easier.

I'm trying to figure out ways of moving geoserver into the 2.0
web-application framework so that the maximum number of people can
contribute (like you folks are right now) while still remaining stable
and not having long periods of downtime for users who want to use
geoserver in production.

The configuration system is one of the "scariest" (and largest) things
inside geoserver and I want to make it as easy to deal with as possible
- and I really like the fact that you folks are doing configuration
stuff and thinking about it. I also want to ensure that there's
maximum isolation between geoserver and the various configuration
systems so each portion is easier to build, maintain, and understand.

What do you think? Am I understanding you correctly?

Dave

--------------

I just read Gabriel's reply, and my first thought on this would be to
have the OWC (OGC Web Context) as the underlying configuration (if
everyone's happy with it). We would have the Ingestion Engine (running
independently) write this out. The various other
configuration-managers (ie webapps) could also write these out.

----------------------------------------------------------
This mail sent through IMP: https://webmail.limegroup.com/

(attachments)

config.gif

It seems that me and Alessio were not the only who were feeling like
we needed an automatic ingestion engine!
:slight_smile:

Simone.

On 7/15/05, dblasby@anonymised.com <dblasby@anonymised.com> wrote:

This "ingestion-engine" sounds interesting.

From what I understand, it appears that you want to be able to manage
the Geoserver outside of the web-admin tool. Specifically, you want to
be able to setup a bunch of directories and have the configuration
auto-magically derived from this.

So, what I see from this is a system ("the ingestion engine" that would
notice that the directory structure has changed, then look at it and
make a new geoserver configuration (ie. geoserver .xml file(s)). It
would then tell geoserver to re-load the configuration.

From this perspective, you can see the geoserver configuration (ie. its
XML files) and their management are independent. More specifically, you
can view the struts web-admin and the "ingestion engine" as two
different ways of making configuration files -- geoserver will not be
able to see the difference.

Take a look at the picture attached. If we abstract the configuration
into either a set of services or XML files (or both), we can have
multiple ways of viewing and updating the information.

I really like this, because I want to see the configuration system
redone - at least the struts web-app, but I'm sure with the 2.0
geoserver-web-application-platform will need some fundamental
configuration (ie. XML) changes. There are several JIRA tasks for
improvements.

I'd really like to see the current Struts-based codebase be improved -
one way to do this could be to keep the old one and slowly introduce a
new one (perhaps not even built with struts). That way, if there's any
stability (or whatever) issues with it, people could just use the old
one. This provides an extremely low-risk upgrade path that we can
treat more like the cathedral-and-bazaar model (since we do not have to
build the entire cathedral before we release it).

For people using the "ingestion Engine", they would use the web-app only
to look at configuration instead of make any modifications (the
modifications will be done by the ingestion engine).

So, what I'm trying to say is this:

The configuration stuff is going to be changing over time, and we want
to be able to support multiple configuration systems (ie. web apps and
auto-generators).

I think this means two things:

a) we need to make sure the base configuration (ie. its XML files and
perhaps a few services) is "good enough" for where we're going in the
future.
b) the systems that will be making or managing the configurations have
to designed in such a way that they can handle the base configuration
XML files changing a bit over time (so we can improve #1).

I think the minimum amount of work we can do for this is to have a
geoserver "service" (ie. a request you can send to geoserver) that says
"please reload your configuration" or "please load your configuration
from here". For starters, we can make this service the same as
pressing the "load" button on the current web application.

I also want to warn you (actually, invite you to help) that the
underlying configuration (XML/services) will be changing over time as
we make geoserver more into an application framework with the ability
to have many more services (like the ones you've been working on).

So, the part of the "ingestion engine" that actually writes out a
configuration (XML) needs to be written in such a way that its easy to
change.

Does this make sense? I'm not 100% sure exactly what you're trying to
do - I'm trying to make your job easier (for the long run) plus make
future people's jobs easier.

I'm trying to figure out ways of moving geoserver into the 2.0
web-application framework so that the maximum number of people can
contribute (like you folks are right now) while still remaining stable
and not having long periods of downtime for users who want to use
geoserver in production.

The configuration system is one of the "scariest" (and largest) things
inside geoserver and I want to make it as easy to deal with as possible
- and I really like the fact that you folks are doing configuration
stuff and thinking about it. I also want to ensure that there's
maximum isolation between geoserver and the various configuration
systems so each portion is easier to build, maintain, and understand.

What do you think? Am I understanding you correctly?

Dave

--------------

I just read Gabriel's reply, and my first thought on this would be to
have the OWC (OGC Web Context) as the underlying configuration (if
everyone's happy with it). We would have the Ingestion Engine (running
independently) write this out. The various other
configuration-managers (ie webapps) could also write these out.

----------------------------------------------------------
This mail sent through IMP: https://webmail.limegroup.com/