[GeoNetwork-devel] Catalog Services

GeoNetwork GeoNetwork Developer
1

hi!

I was looking for all over the internet technical information about Catalog Services, How it works?

So I’m asking for help if some one can tell me where I can find this information.

Tanks.

–
Saludos…

2

Hi!

The catalog service, or WCS (Web Catalog Service), is a web access standard developed or
supported by the Open Geospatial Consortium, OGC.
So, you should find some documentation on http://www.opengeospatial.org/.

In short, the idea is that many web servers around the world support standardized access methods,
like getCapabilities. Any WCS client can then contact all those servers in a similar way and in turn those WCS servers respond in a standard way as well, which makes the sharing of geospatial information across the world a lot easier.

This access is mostly based on HTTP GET (or PUT) requests, similar to those issued by your browser when you are surfing the web.

To make your GeoNetwork software look for data on other WCS servers besides your own, you should look into the Harvesting options. But I haven't worked with those yet, so I can't help you much with setting that up.

Regards,

Tim

From: Khrizart <khrizart@anonymised.com>
To: geonetwork-devel@lists.sourceforge.net
Subject: [GeoNetwork-devel] Catalog Services
Date: Mon, 18 Dec 2006 10:55:22 -0600

hi!

I was looking for all over the internet technical information about *Catalog
Services*, How it works?

So I'm asking for help if some one can tell me where I can find this
information.

Tanks.
--
Saludos....

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

_________________________________________________________________
Who is the sweetheart of the Japanese and always holds something in his hands? Live Search knows! How about you? http://search.live.com/images/results.aspx?q=Manneken%20pis&FORM=QBIR

3

Hi Tim and Khrizart,

Just for clarification, the harvesting part of CSW has not (yet, anyone!?) been implemented in GeoNetwork, while the rest of CSW is just undergoing final testing and corrections within the OGC OWS4-CITE context to serve as OGC Reference Implementation.

Harvesting between GeoNetwork nodes and from file systems is available as well as distributed searches through CSW 1.0 (Z39.50).

Ciao,
Jeroen

On Dec 19, 2006, at 8:43 AM, Tim Jacobs wrote:

Hi!

The catalog service, or WCS (Web Catalog Service), is a web access standard
developed or
supported by the Open Geospatial Consortium, OGC.
So, you should find some documentation on http://www.opengeospatial.org/.

In short, the idea is that many web servers around the world support
standardized access methods,
like getCapabilities. Any WCS client can then contact all those servers in a
similar way and in turn those WCS servers respond in a standard way as well,
which makes the sharing of geospatial information across the world a lot
easier.

This access is mostly based on HTTP GET (or PUT) requests, similar to those
issued by your browser when you are surfing the web.

To make your GeoNetwork software look for data on other WCS servers besides
your own, you should look into the Harvesting options. But I haven't worked
with those yet, so I can't help you much with setting that up.

Regards,

Tim

From: Khrizart <khrizart@anonymised.com>
To: geonetwork-devel@lists.sourceforge.net
Subject: [GeoNetwork-devel] Catalog Services
Date: Mon, 18 Dec 2006 10:55:22 -0600

hi!

I was looking for all over the internet technical information about
*Catalog
Services*, How it works?

So I'm asking for help if some one can tell me where I can find this
information.

Tanks.
--
Saludos....

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share
your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

_________________________________________________________________
Who is the sweetheart of the Japanese and always holds something in his
hands? Live Search knows! How about you?
Manneken pis - Search Images

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

4

dear Jeroen, all,

On Tue, Dec 19, 2006 at 11:59:39AM +0100, Jeroen Ticheler wrote:

Just for clarification, the harvesting part of CSW has not (yet,
anyone!?) been implemented in GeoNetwork, while the rest of CSW is
just undergoing final testing and corrections within the OGC OWS4-
CITE context to serve as OGC Reference Implementation.
Harvesting between GeoNetwork nodes and from file systems is
available as well as distributed searches through CSW 1.0 (Z39.50).

Thanks for the correction, Jeroen :wink: CSW-2 support has been in the
version in subversion awhile, though? Hey, I am surprised that OGC
don't feel the need for 'harvesting' and Transactions in the reference
implementation at the start.

I would also like to ask about the status of an OAI-PMH interface
to GeoNetwork - both for publishing and for querying other
repositories - and to braindump a bit at you - sorry.

I have been digging out documents:
http://wiki.osgeo.org/index.php/Simple_Catalog_Interface
http://geometa.info/rappiinfo/wiki/index.php/OAI-PMH which has a
extended comparison of OAI-PMH/WFS/CAT+CSW. It does not treat of CSW-2

I am slowly trying to shed some of my immense burden of ignorance
on the subject of CSW-2. Now goodness knows OAI-PMH is not a gem of
modern enlightened design either, but it *is* simple to implement and
is well-understood in a wider world of "digital libraries" and
knowledge repository work. There is simple commonality between the two

CSW OAI
=== ===
GetRecords ListRecords
GetRecordById GetRecord
DescribeRecord ListMetadataFormats (kind of)

As an aside it is worth looking at what was happening to
DescribeFeatureType in WFS-Simple the last time i checked, they were
considering folding it all into the GetCapabilities response/request.

But is this a *right* approach just because it's in a lot of places?

Now as i understand it, what Harvest is doing in CSW-2 is just "Here is
a URL go get its contents add them to your model". Just like an RSS
aggregator behaves. Imagine this as like bloglines for geodata. A
system like bloglines becomes a repository through usage, and saves
load on the original resources. This is like what running a Tile Map
Service node would be if the TMS node was also distributing metadata
about what it is collecting. There really isn't much overlap with CSW
here that i see.

What i am stumbling towards saying, is there are simple behaviours in
an aggregator / harvester not accounted for in CSW-2 that are going to
be needed anyway - particularly wanting to make queries based
on changes over time relative to interactions with the repository.

Meanwhile the client use of a repository can be viewed as a different
problem. More queries that are spatial and temporal selections, and
more ways of using feedback about associations between datasets -
passed around as WMC documents for example - especially more ways to
correct and expand descriptions of data...

For a while now a group of OSGeo-related people have been thinking over
the lightweight information model / desire for lightweight catalog
service interface problem. Stefan Keller's ideas on geospatial
extensions to Dublin Core... Tom Kralidis' OWSCat is interesting but is
focused on providing collections of catalog services - a "service
discovery service"? and not of data as files or abstract contents.
I want to see something that is "as WFS-Simple to WFS, as GeoRSS to
GML" for geospatial metadata, and would wind up either extending
OAI-PMH or abusing/flattening CSW-2 so much that it would end up being
'something new'. http://wiki.osgeo.org/index.php/DCLite4G contains
development notes along these lines - "mass market metadata".

cheers,

jo

5

Hi Jo and all,

it is time to provide some light on what we are doing.

=========================================
The CS/W 2.0.1 reference implementation is complete. A test server
with 12 metadata is located at:

http://www.crisalis-tech.com:8081/geonetwork

To get the capabilities use the following url:

http://www.crisalis-tech.com:8081/geonetwork/srv/en/csw?request=GetCapabilities&service=CSW

The CITE test scripts for the RI are almost passed (there are a few glitches
to be fixed on the scripts, not on our side).

=========================================
The harvesting is not included into the RI for 2 reasons:

- the CS/W interface is a discovery interface. Clients are not interested
  in knowning how to feed data into the catalog. There is an exception
  here when the client needs to publish a data, but in this case he can
  use the native interface of the catalogue (i.e. geonetwork's one).

- it must be specified into the CS/W ISO19115 profile which, at this time,
  is not finalized yet.

=========================================
The OAI-PMH is planned and we have the funds for that. The schedule
is to deliver by February but this depends on another issue.

We have been asked to implement the CS/W interface for the Earth
Observation metadata profile (maybe with ebRIM). This project will
start at January so I don't know if we can manage to work on boths.

=========================================
The harvesting from a geonetwork node is finished and now I'm working
on moving some installer parameters to the web interface. I think we
could deliver a new alpha by the end of the year. This would allow some
play with the harvesting.

Cheers,
Andrea

> Just for clarification, the harvesting part of CSW has not (yet,
> anyone!?) been implemented in GeoNetwork, while the rest of CSW is
> just undergoing final testing and corrections within the OGC OWS4-
> CITE context to serve as OGC Reference Implementation.
> Harvesting between GeoNetwork nodes and from file systems is
> available as well as distributed searches through CSW 1.0 (Z39.50).

Thanks for the correction, Jeroen :wink: CSW-2 support has been in the
version in subversion awhile, though? Hey, I am surprised that OGC
don't feel the need for 'harvesting' and Transactions in the reference
implementation at the start.

I would also like to ask about the status of an OAI-PMH interface
to GeoNetwork - both for publishing and for querying other
repositories - and to braindump a bit at you - sorry.

I have been digging out documents:
http://wiki.osgeo.org/index.php/Simple_Catalog_Interface
http://geometa.info/rappiinfo/wiki/index.php/OAI-PMH which has a
extended comparison of OAI-PMH/WFS/CAT+CSW. It does not treat of CSW-2

I am slowly trying to shed some of my immense burden of ignorance
on the subject of CSW-2. Now goodness knows OAI-PMH is not a gem of
modern enlightened design either, but it *is* simple to implement and
is well-understood in a wider world of "digital libraries" and
knowledge repository work. There is simple commonality between the two

CSW OAI
=== ===
GetRecords ListRecords
GetRecordById GetRecord
DescribeRecord ListMetadataFormats (kind of)

As an aside it is worth looking at what was happening to
DescribeFeatureType in WFS-Simple the last time i checked, they were
considering folding it all into the GetCapabilities response/request.

But is this a *right* approach just because it's in a lot of places?

Now as i understand it, what Harvest is doing in CSW-2 is just "Here is
a URL go get its contents add them to your model". Just like an RSS
aggregator behaves. Imagine this as like bloglines for geodata. A
system like bloglines becomes a repository through usage, and saves
load on the original resources. This is like what running a Tile Map
Service node would be if the TMS node was also distributing metadata
about what it is collecting. There really isn't much overlap with CSW
here that i see.

What i am stumbling towards saying, is there are simple behaviours in
an aggregator / harvester not accounted for in CSW-2 that are going to
be needed anyway - particularly wanting to make queries based
on changes over time relative to interactions with the repository.

Meanwhile the client use of a repository can be viewed as a different
problem. More queries that are spatial and temporal selections, and
more ways of using feedback about associations between datasets -
passed around as WMC documents for example - especially more ways to
correct and expand descriptions of data...

For a while now a group of OSGeo-related people have been thinking over
the lightweight information model / desire for lightweight catalog
service interface problem. Stefan Keller's ideas on geospatial
extensions to Dublin Core... Tom Kralidis' OWSCat is interesting but is
focused on providing collections of catalog services - a "service
discovery service"? and not of data as files or abstract contents.
I want to see something that is "as WFS-Simple to WFS, as GeoRSS to
GML" for geospatial metadata, and would wind up either extending
OAI-PMH or abusing/flattening CSW-2 so much that it would end up being
'something new'. http://wiki.osgeo.org/index.php/DCLite4G contains
development notes along these lines - "mass market metadata".

cheers,

jo

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

6

Hi all,

On Dec 19, 2006, at 5:54 PM, Andrea Carboni wrote:

=========================================
The harvesting is not included into the RI for 2 reasons:

- the CS/W interface is a discovery interface. Clients are not interested
  in knowning how to feed data into the catalog. There is an exception
  here when the client needs to publish a data, but in this case he can
  use the native interface of the catalogue (i.e. geonetwork's one).

As Andrea said CSW harvesting is triggered in the 'opposite' manner of what you think at first, it is the client that performs the operation on the catalog (and gets the catalog to harvest something), not the catalog itself.
For the harvesting of the catalog itself, GeoNetwork has a custom interface/ process. This process will go out to some XML service (provided by another GeoNetwork node at this stage) and will request a very brief result set from the other catalog that contains file identifier (UUID), time stamp and catalog ID. By comparing those with its internally cached records it will decide to request new or updated records and it will remove those not found anymore.

It could be an option to add this functionality to do the same thing on CSW (1 & 2!?) catalogs as well.

The same is true for archives publishing through OAI. The catalog has to be sure it can handle (and will request) the format it supports and it uses some OAI properties to perform the same checks as described above (UUID, time stamp, catalog ID).

Considering the extensibility of the GeoNetwork supported metadata standards, adding a DCLite4G schema and presentation can be done by those familiar with XSL and XSD.

Anyone would like to start testing this on a development branch!?

- it must be specified into the CS/W ISO19115 profile which, at this time,
  is not finalized yet.

Thanks for the correction, Jeroen :wink: CSW-2 support has been in the
version in subversion awhile, though? Hey, I am surprised that OGC
don't feel the need for 'harvesting' and Transactions in the reference
implementation at the start.

Harvesting is explained above. 'Our' harvesting would be achieved through queries and than compare results with those already cached. I think there are some issues here since the CSW does not provide the required details to do this efficiently. From what I know, some prototypes doing this will harvest the full catalog again of every update... ;(

Ciao,
Jeroen

7

Hi all,

one important thing to say is related to the harvesting concept:

- In geonetwork, the harvesting is used as a form of metadata replication
  to reduce bandwidth usage in those countries where it is a premium.
  Thus, here the harvesting is intended as a sharing mechanism.

- In CS/W, the harvesting is used as a publishing mechanism. Once a user
  has created a metadata, he can feed it into the catalogue through 2 ways:
    - As a push mechanism : he uses the transaction operation to push
      the metadata into the catalogue
    - As a pull mechanism : he asks the catalogue to get the metadata from
      a provided url. The client can ask the catalogue to perform this process
      periodically but the specs are a bit vague on how the client's metadata
      should be kept in sync with the catalogue's one.

Cheers,
Andrea

Hi all,

On Dec 19, 2006, at 5:54 PM, Andrea Carboni wrote:

> =========================================
> The harvesting is not included into the RI for 2 reasons:
>
> - the CS/W interface is a discovery interface. Clients are not
> interested
> in knowning how to feed data into the catalog. There is an exception
> here when the client needs to publish a data, but in this case he
> can
> use the native interface of the catalogue (i.e. geonetwork's one).

As Andrea said CSW harvesting is triggered in the 'opposite' manner
of what you think at first, it is the client that performs the
operation on the catalog (and gets the catalog to harvest something),
not the catalog itself.
For the harvesting of the catalog itself, GeoNetwork has a custom
interface/ process. This process will go out to some XML service
(provided by another GeoNetwork node at this stage) and will request
a very brief result set from the other catalog that contains file
identifier (UUID), time stamp and catalog ID. By comparing those
with its internally cached records it will decide to request new or
updated records and it will remove those not found anymore.

It could be an option to add this functionality to do the same thing
on CSW (1 & 2!?) catalogs as well.

The same is true for archives publishing through OAI. The catalog has
to be sure it can handle (and will request) the format it supports
and it uses some OAI properties to perform the same checks as
described above (UUID, time stamp, catalog ID).

Considering the extensibility of the GeoNetwork supported metadata
standards, adding a DCLite4G schema and presentation can be done by
those familiar with XSL and XSD.

Anyone would like to start testing this on a development branch!?

>
> - it must be specified into the CS/W ISO19115 profile which, at
> this time,
> is not finalized yet.
>
>>
>> Thanks for the correction, Jeroen :wink: CSW-2 support has been in the
>> version in subversion awhile, though? Hey, I am surprised that OGC
>> don't feel the need for 'harvesting' and Transactions in the
>> reference
>> implementation at the start.
Harvesting is explained above. 'Our' harvesting would be achieved
through queries and than compare results with those already cached. I
think there are some issues here since the CSW does not provide the
required details to do this efficiently. From what I know, some
prototypes doing this will harvest the full catalog again of every
update... ;(

Ciao,
Jeroen

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

8

dear Jeroen, Andrea, thanks.

On Wed, Dec 20, 2006 at 02:40:12PM +0100, Andrea Carboni wrote:

- In geonetwork, the harvesting is used as a form of metadata replication
  to reduce bandwidth usage in those countries where it is a premium.
  Thus, here the harvesting is intended as a sharing mechanism.

Right; you have had to create your own intra-GeoNetwork exchange
protocol where you are synchronising information about your knowledge
and use of the ISO19115-level information. Is there documentation
on this protocol, could one help make some?

http://wiki.osgeo.org/index.php/Talk:DCLite4G#Interfaces talks about
splitting out 'catalog services' interfaces into different subsets
oriented at metadata for services, packages, and publishing.

The news of Google closing their SOAP search API and replacing it with
an AJAX one not allowing data access, concerns me more about "open"
APIs for data, and directs me more to sharing data as well as metadata
between repositories...
http://radar.oreilly.com/archives/2006/12/google_depreciates_SOAP_API.html

cheers,

jo