On 11/18/05, Jody Garnett <jgarnett@anonymised.com> wrote:
We do have the event model in place for this stuff! Indeed we have been
looking hard at these
kinds of questions for uDig. I am forwarding this onto Jesse, and have
asked him to join
the geoserver devel list for this chat.We can set up uDig to respect those HTTP headers, but w/ respect to
Chris Holmes ideas
many are already inflight (and why we keep submitting bug reports
against the DataStores).
Interesting. I'd seen those bug reports flying by, but hadn't thought
deeply about it. We were thinking of doing things at the GeoServer level,
in the transactionResponse, thinking about putting in a plug-in structure
to handle a full sort of workflow. This is where the versioning would
take place, inserting into a meta-table whenever a change goes through,
but also do things like email notifications on areas of interest, wait for
a peer review approval, perform validation, and similar things. Could be
good to do it at the geotools level - where are you doing the validation
processing right now? Like I'm wondering about putting it somewhere in
the geotools Transaction, just like adding a state you could maybe add a
'action' - a validation or an email sending or waiting for a peer review?
If we did this at the gt level you could have set up something in uDig
that would like make a manager come over and review and insert his
password to do an update, for newbie employees. This is all a very new
thought, but it could work out design-wise to put much of the
geocollaborator stuff down at the geotools level.
Also, Saul explained where my logic was wrong, you have to decide
beforehand how long to set the cache, in a header you send out. To do
more complex stuff you'd have to have the client listen for the cache
resets, which shouldn't be all that hard with geocollab stuff.
Chris
Jody
> Chris,
>
> Glad to hear the effort is interesting to some other folks!
>
> As for handling cache-dumping/spatial-feature-caching, etc...that's a
> very very interesting question.
>
> First off, let me draw a line between what I've done so far, and what
I
> understand you to be talking about in terms of more advanced caching.
>
> What I've done so far is borderline trivial. Basically it's a GUI to
> control the addition/calculation of a single extra header on the HTTP
> response. If that header is present, it instructs other software
> (squid, particularly--and I'm a squid super-newbie) to cache getmaps
> fetched via KVP/GET (not KVP/POST and not POST/XML). Configuring the
> other software to properly cache stuff is actually pretty hard, and
the
> world of internet caching is extremely advanced. Basically, I've set
up
> squid, and made a tiny change to geoserver to make squid do it's
> squiddy-magic with getmaps.
>
>
> A much much larger and potentially more interesting question is
> something along the lines of "how can we do ka-map style maps
> super-efficiently, using all the knowledge that we have about GIS and
> WMS to optimize our solution?"
>
> To be honest, I think this is a separate application. Something like
a
> specialized implementation of squid which is a client to the main
> geoserver WMS, and could even be the place where the javascript for
the
> ka-map port lives.
>
> By the way, your intuition about ka-map was correct. Ka-map is
usually
> configured by a php script that uses PHP-mapscript to generate map
> tiles. But I just changed the format of the request and the php
script
> to refer to a WMS instead....minimal changes, really. Doing the same
> thing with a J2EE based backend (servlet + kamap-configuration-gui +
> ka-map source + caching logic) would be a cool port of ka-map to
> geoserver. Hmm...I'll start thinking about that...
>
> Maybe the ka-map j2ee port (ja-map?) could be a "listener" for WFS
> events involving data updates, and could dump the right portions of
its
> cache at the right times, as you mentioned.
>
> --saul
>
>
>
>
>
>
>
> On Thu, 2005-11-17 at 09:43 -0500, Chris Holmes wrote:
>
>> Thinking about things a bit more, have you given any thought to
tracking
>> the transactions against each layer as the sign to kill the cache,
instead
>> of using a user defined seconds thing? Right now it would have to be
sort
>> of hacked in, but it wouldn't be too bad a hack. In the future for a
>> versioning WFS we're thinking of making a pluggable processing chain,
so
>> you can define validations, peer reviews, email updates, versioning
>> tables, ect., and ideally this could slot right in - notify cache.
But I
>> think you could just add a bit of code in the current Transaction
Response
>> (which may be the current candidate for most in need of a
refactoring), to
>> reset the cache if a transaction on that layer came in. Eventually
we
>> could maybe even do complex checking of bounds so as to only update
the
>> cache on areas that were affected by the transaction, but that's
probably
>> a ways away.
>>
>> Also, have you given any thought of how mapbuilder could support
this? Or
>> have you tried with ka-map? It might be nice to see if we could have
an
>> embedded demo that shows this off in the initial start-up, but I
don't
>> think we really want to start including squid. I was also under the
>> impression that ka-map relied on mapserver specific ways of doing
things?
>> I would love to be able to see ka-map as a geoserver client.
>>
>> This also starts to bring up the issue of the featureType editor page
>> getting too complicated. This alone won't do it, but there are a
number
>> of things that could be quite nice at the featureType level, that we
just
>> set globally right now, citeConformanceHacks, numDecimals,
maxFeatures,
>> ect. And some, like this, are sort of WMS specific, and some, like
>> numDecimals, are sort of WFS specific. Do we have separate pages for
>> each? Or perhaps an 'advanced' page?
>>
>> I'm thinking about this in the context of GeoServer 2.0 thoughts,
where we
>> want to allow plug-ins, such as WCS, be able to set controls, and
they'll
>> want to do some of that at the featureType level. An alternative
thought
>> on doing this could be to flip it around, and have a page for the
config
>> option, and check boxes, controls for each featureType. So in like
the
>> WFS config, you'd have 'content', 'description', and 'featureTypes',
and
>> the 'featureTypes' would list _all_ your featureTypes, and each would
have
>> the option to check it. Also thinking about this in the context of
>> security and user level permissions, at the featureType level.
>>
>> You're really pushing the bleeding edge of GeoServer Saul, it's great
to
>> see! We'll do our best to catch up quickly and support your efforts.
>>
>> Chris
>>
>> On Wed, 16 Nov 2005, Saul Farber wrote:
>>
>>
>>> Efren,
>>>
>>> This is what this patch does (more or less).
>>>
>>> The WMSCache portion of your diagram is played by software like
>>> "squid" (www.squid-cache.org) or other caching proxy/http
accelerator.
>>>
>>> This patch does not have the WMS tier do any actual caching...it's
>>> really a very basic header-manipulator. Any request sent DIRECTLY
to
>>> the tomcat port will be fully rendered exactly as before. It's only
for
>>> those requesting content via a proxy (generally an intention of the
>>> setup architecture) that this would actually do any caching.
>>>
>>> --saul
>>>
>>>
>>> On Wed, 2005-11-16 at 11:22 -0800, Efren Serra wrote:
>>>
>>>> Folks,
>>>>
>>>> I was wondering about caching on the client side instead like is
touched on
>>>> the following URL:
http://www.codeproject.com/csharp/WMSOverviewAll.asp
>>>>
>>>> I think caching at the WMS tier would tend to convolute the WMS
source code
>>>> plus complicate the testsuite, etc. Also, if we really wanted to
"cache,"
>>>> then we should follow the model that CPU's follow (e.g., CPU <-->
Cache <-->
>>>> Main Memory <--> Disks, etc.) but in our domain:
>>>>
>>>> Client <--> WMSCache <--> WMS <--> RDBMS, where all WMSCache does
is index
>>>> GetMap requests to Map's; if there is a cache hit, return the map;
>>>> otherwise, forward the request to WMS, cache it, and return it to
the
>>>> client.
>>>>
>>>> This would be easily implemented in the Tomcat container by just
making sure
>>>> that all GetMap requests get sent to the WMSCache Servlet, for
instance.
>>>>
>>>> v/r,
>>>> Efren
>>>>
>>>> -----Original Message-----
>>>> From: geoserver-devel-admin@lists.sourceforge.net
>>>> [mailto:geoserver-devel-admin@lists.sourceforge.net]On Behalf Of
Saul
>>>> Farber (JIRA)
>>>> Sent: Tuesday, November 15, 2005 1:46 PM
>>>> To: geoserver-devel@lists.sourceforge.net
>>>> Subject: [Geoserver-devel] [jira] Created: (GEOS-454) Cache-Control
>>>> based WMS map caching
>>>>
>>>>
>>>> Cache-Control based WMS map caching
>>>> -----------------------------------
>>>>
--