There are some significant problems with harvesting which Doug Nebert
has uncovered in testing. In particular, there seems to be a
concurrency bug with the database interface, using both McKoi and MySQL.
This is an important issue because, in a scenario with a large number
of sources to harvest, it cannot be guaranteed that only one at a time
will run. We propose to investigate this and work with Simon and Jose,
who are already looking into the problem, to fix it.
At the same time, we propose to add the ability to harvest from both
Z39.50 and WAF (web-accessible folders) and to modify/enhance the
harvest administrative interface. WAF is quite simple - it's like
WebDAV only easier. A simple wget-like function would suffice.
Experience in GEOSS indicates that not all desirable sources respond
correctly to blank queries, so we propose that both Z39.50 and CSW ought
to include the capability to specify a query as part of the harvest
process.
--
Archie
-- Archie Warnock warnock@anonymised.com
-- A/WWW Enterprises www.awcubed.com
-- As a matter of fact, I _do_ speak for my employer.