[Geoserver-devel] app-schema tutorial

Ben_Caradoc-Davies · June 8, 2009, 7:43am

Andrea,

I have committed the app-schema tutorial to trunk. For your convenience, I have put the generated HTML on one of our servers:

https://twiki.auscope.org/projects/geoserver/geoserver-trunk-doc/user/tutorials/app-schema/app-schema-tutorial.html

Please let me know if there is anything you would like clarified, simplified, or otherwise improved.

There is a a Hudson job to build the docs for 1.7.x, but not for trunk as far as I know. Do these Hudson jobs push the HTML directly to the web server?

Kind regards,

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

Michael_Melvin · June 8, 2009, 2:41pm

I have just created a Hudson job to build the trunk docs, and have also linked the HTML output to docs.geoserver.org:

http://docs.geoserver.org/trunk/
(is the same as)
http://gridlock.openplans.org/geoserver/trunk/doc/

Ben, FYI, I don't see your app-schema tutorial in the build on trunk, but I may have set things up incorrectly. Justin, I'm CCing you since I know you know Hudson well. Can you check my work?

Thanks,
Mike Pumphrey
OpenGeo - http://opengeo.org

Ben Caradoc-Davies wrote:

Andrea,

I have committed the app-schema tutorial to trunk. For your convenience, I have put the generated HTML on one of our servers:

https://twiki.auscope.org/projects/geoserver/geoserver-trunk-doc/user/tutorials/app-schema/app-schema-tutorial.html

Please let me know if there is anything you would like clarified, simplified, or otherwise improved.

There is a a Hudson job to build the docs for 1.7.x, but not for trunk as far as I know. Do these Hudson jobs push the HTML directly to the web server?

Kind regards,

Andrea_Aime3 · June 8, 2009, 3:13pm

Ben Caradoc-Davies ha scritto:

Andrea,

I have committed the app-schema tutorial to trunk. For your convenience, I have put the generated HTML on one of our servers:

https://twiki.auscope.org/projects/geoserver/geoserver-trunk-doc/user/tutorials/app-schema/app-schema-tutorial.html

Please let me know if there is anything you would like clarified, simplified, or otherwise improved.

Looked a bit into it. Some general feedback:
- the tutorial assumes familarity with SVN, it would be nicer to provide a link to a zipped file with everything needed to setup the tutorial
- the tutorial uses property files. Developers are comfortable with them, but I think no normal user has ever seen that data source in action. Some explanation about it is probably required
- the structure made me go up and down quite a few times because the
   title structure is flat, some titles explain parts of a file, some
   others explain a standalone file. For example
   "gsml:MappedFeature datastore" talks about a single file, but
   "Namespaces" talks about the content of a single mapping file.

As one that never played with complex features I miss a bit of context:
- why are we declaring namespaces in the mapping file (ok, this one I
   understand, but put yourself in the place of one that never saw
   complex features)
- why do we refer an external catalog... and what is a OASIS catalog
   to start with?
- the target types section refer to
   http://www.geosciml.org/geosciml/2.0/xsd/geosciml.xsd
   This leaves me baffled... no reference to a type or element name?
   A XSD file can define plenty, afaik there is not concept of main,
   root type. So what is the meaning of <FeatureType> there? It's not
   like the schema contains the definition of a specific one no?
   I understand you need to refer the schema that contains the mapped
   types, but the names are misleading and there is no explanation.
- in gml:id mapping, why OCQL? I've heard of ECQL (extended) but never
   about OCQL

Finally, I'm a little concerned about the many referencing to your
wiki... as in, how long is that wiki going to be around? Plus, say you
work on new features, how do you ensure the GeoServer 2.0 documentation
points to the state the wiki had for 2.0, and not the current one? (in
other terms, how to ensure the documentation is versioned?)

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

Ben_Caradoc-Davies · June 9, 2009, 1:47am

Mike Pumphrey wrote:

I have just created a Hudson job to build the trunk docs, and have also linked the HTML output to docs.geoserver.org
http://docs.geoserver.org/trunk/
(is the same as)
http://gridlock.openplans.org/geoserver/trunk/doc/

Thanks, Mike, that is excellent.

Ben, FYI, I don't see your app-schema tutorial in the build on trunk, but I may have set things up incorrectly. Justin, I'm CCing you since I know you know Hudson well. Can you check my work?

It is not yet in the index, so you need to know the path:
http://gridlock.openplans.org/geoserver/trunk/doc/user/tutorials/app-schema/app-schema-tutorial.html
http://docs.geoserver.org/trunk/user/tutorials/app-schema/app-schema-tutorial.html

I will add an index.

Kind regards,

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

Ben_Caradoc-Davies · June 9, 2009, 4:33am

Andrea Aime wrote:
[in an example app-schema mapping file]

- in gml:id mapping, why OCQL? I've heard of ECQL (extended) but never
about OCQL

This was implemented before my time on the project.

Gabriel, Rob A, or Jody. What is the origin of the term OCQL, as used in the community-schemas/app-schema mapping file?

Kind regards,

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

Ben_Caradoc-Davies · June 9, 2009, 4:45am

Thanks a lot, Andrea for your feedback. I will make the changes you suggest, as they are good. I have commented on a few below:

Andrea Aime wrote:

Looked a bit into it. Some general feedback:
- the tutorial assumes familarity with SVN, it would be nicer to provide a link to a zipped file with everything needed to setup the tutorial

The config includes a build.xml to create the zip (this is the usual way I distribute my configs), but would the zipped file need to go in SVN to be included in the docs? I need to understand :download:.

- the tutorial uses property files. Developers are comfortable with them, but I think no normal user has ever seen that data source in action. Some explanation about it is probably required

Is there a property file tutorial or documentation for PropertyDataSource in the user or developer manual?

- in gml:id mapping, why OCQL? I've heard of ECQL (extended) but never
about OCQL

Neither have I. This term was used before I joined the project. I have emailed some likely suspects for an explanation.

Finally, I'm a little concerned about the many referencing to your
wiki... as in, how long is that wiki going to be around? Plus, say you
work on new features, how do you ensure the GeoServer 2.0 documentation
points to the state the wiki had for 2.0, and not the current one? (in
other terms, how to ensure the documentation is versioned?)

Your concern is well-justified. All the configuration documentation on the seegrid wiki has to be migrated to the GeoServer user manual. There is no other sensible way to manage it. Now that we have a Hudson for trunk docs, there is no reason not to move it as soon as we have time. In my view, this piece of work is not complete until we have done so.

Thanks again for your extensive and useful feedback.

Kind regards,

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

Andrea_Aime3 · June 9, 2009, 6:42am

Ben Caradoc-Davies ha scritto:

Thanks a lot, Andrea for your feedback. I will make the changes you suggest, as they are good. I have commented on a few below:

Andrea Aime wrote:

Looked a bit into it. Some general feedback:
- the tutorial assumes familarity with SVN, it would be nicer to provide a link to a zipped file with everything needed to setup the tutorial

The config includes a build.xml to create the zip (this is the usual way I distribute my configs), but would the zipped file need to go in SVN to be included in the docs? I need to understand :download:.

Hmm... not much of an help here. I think David posted a mail on the
subject not long ago. We should add his wisdom to the sphix guide.

- the tutorial uses property files. Developers are comfortable with them, but I think no normal user has ever seen that data source in action. Some explanation about it is probably required

Is there a property file tutorial or documentation for PropertyDataSource in the user or developer manual?

Hmmm... I don't think so. Actually we have a Jira somewhere requesting
to remove the property datastore from the stores that ship along
with GeoServer because it caused confusion.
The only "documentation" that I remember is the reason why the property
data store was created in the first place, that is, as a tutorial
for making new datastores:
http://docs.codehaus.org/display/GEOTDOC/Old+Creating+a+DataStore+Tutorial
Then it was picked up for testing and became an officially supported
datastore for GeoTools, but afaik there is no much extra documentation
for it. Jody (cc'ed) might know more.

I think we already removed it for 1.7.x, thought it still shows up in trunk. However, removing it would break your demo... hmmm....

- in gml:id mapping, why OCQL? I've heard of ECQL (extended) but never
about OCQL

Neither have I. This term was used before I joined the project. I have emailed some likely suspects for an explanation.

Finally, I'm a little concerned about the many referencing to your
wiki... as in, how long is that wiki going to be around? Plus, say you
work on new features, how do you ensure the GeoServer 2.0 documentation
points to the state the wiki had for 2.0, and not the current one? (in
other terms, how to ensure the documentation is versioned?)

Your concern is well-justified. All the configuration documentation on the seegrid wiki has to be migrated to the GeoServer user manual. There is no other sensible way to manage it. Now that we have a Hudson for trunk docs, there is no reason not to move it as soon as we have time. In my view, this piece of work is not complete until we have done so.

Excellent

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

Justin_Deoliveira · June 9, 2009, 1:34pm

<snip>

The config includes a build.xml to create the zip (this is the usual way I distribute my configs), but would the zipped file need to go in SVN to be included in the docs? I need to understand :download:.

Hmm... not much of an help here. I think David posted a mail on the
subject not long ago. We should add his wisdom to the sphix guide.

http://docs.geoserver.org/1.7.x/docguide/sphinx.html#external-files

I think it can be used for local content (stored in svn) or an external link.

- the tutorial uses property files. Developers are comfortable with them, but I think no normal user has ever seen that data source in action. Some explanation about it is probably required

Is there a property file tutorial or documentation for PropertyDataSource in the user or developer manual?

Hmmm... I don't think so. Actually we have a Jira somewhere requesting
to remove the property datastore from the stores that ship along
with GeoServer because it caused confusion.
The only "documentation" that I remember is the reason why the property
data store was created in the first place, that is, as a tutorial
for making new datastores:
http://docs.codehaus.org/display/GEOTDOC/Old+Creating+a+DataStore+Tutorial
Then it was picked up for testing and became an officially supported
datastore for GeoTools, but afaik there is no much extra documentation
for it. Jody (cc'ed) might know more.

I think we already removed it for 1.7.x, thought it still shows up in trunk. However, removing it would break your demo... hmmm....

- in gml:id mapping, why OCQL? I've heard of ECQL (extended) but never
about OCQL

Neither have I. This term was used before I joined the project. I have emailed some likely suspects for an explanation.

Finally, I'm a little concerned about the many referencing to your
wiki... as in, how long is that wiki going to be around? Plus, say you
work on new features, how do you ensure the GeoServer 2.0 documentation
points to the state the wiki had for 2.0, and not the current one? (in
other terms, how to ensure the documentation is versioned?)

Your concern is well-justified. All the configuration documentation on the seegrid wiki has to be migrated to the GeoServer user manual. There is no other sensible way to manage it. Now that we have a Hudson for trunk docs, there is no reason not to move it as soon as we have time. In my view, this piece of work is not complete until we have done so.

Excellent

Cheers
Andrea

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Gabriel_Roldan2 · June 9, 2009, 4:15pm

This was implemented before the existence of the CQL module (hence used to use ExpressionBuilder), and choosed the OCQL acronym to mean OGC Commom Query Language, as the original name of the language is.

I think we should change to ECQL now that's there's a reference to the extended dialect.

Cheers,
Gabriel
Ben Caradoc-Davies wrote:

Andrea Aime wrote:
[in an example app-schema mapping file]

- in gml:id mapping, why OCQL? I've heard of ECQL (extended) but never
about OCQL

This was implemented before my time on the project.

Gabriel, Rob A, or Jody. What is the origin of the term OCQL, as used in the community-schemas/app-schema mapping file?

Kind regards,

--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.

Ben_Caradoc-Davies · June 10, 2009, 2:27am

Justin Deoliveira wrote:

<snip>

The config includes a build.xml to create the zip (this is the usual way I distribute my configs), but would the zipped file need to go in SVN to be included in the docs? I need to understand :download:.

Hmm... not much of an help here. I think David posted a mail on the
subject not long ago. We should add his wisdom to the sphix guide.

http://docs.geoserver.org/1.7.x/docguide/sphinx.html#external-files

I think it can be used for local content (stored in svn) or an external link.

The Sphinx documentation for :download: only mentions local files. An examination of the source indicates that shutil.copyfile() is used to copy downloadable files, so it looks to me that URL downloads are not supported.

I am reluctant to include a zip in svn.

Which is less bad: a link to zip on a CSIRO download server, or a 604 KB zip committed to svn?

If committing the zip to svn is preferred, I can reduce the massively bloated OASIS catalog and cut it down to the minimum required for the tutorial. I should be able to get it under 100 KB without too much effort.

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

David_Winslow · June 10, 2009, 2:45am

On Wed, 2009-06-10 at 10:27 +0800, Ben Caradoc-Davies wrote:

Justin Deoliveira wrote:
> <snip>
>>> The config includes a build.xml to create the zip (this is the usual way
>>> I distribute my configs), but would the zipped file need to go in SVN to
>>> be included in the docs? I need to understand :download:.
>> Hmm... not much of an help here. I think David posted a mail on the
>> subject not long ago. We should add his wisdom to the sphix guide.
>
> http://docs.geoserver.org/1.7.x/docguide/sphinx.html#external-files
>
> I think it can be used for local content (stored in svn) or an external
> link.

The Sphinx documentation for :download: only mentions local files. An
examination of the source indicates that shutil.copyfile() is used to
copy downloadable files, so it looks to me that URL downloads are not
supported.

I am reluctant to include a zip in svn.

Which is less bad: a link to zip on a CSIRO download server, or a 604 KB
zip committed to svn?

If committing the zip to svn is preferred, I can reduce the massively
bloated OASIS catalog and cut it down to the minimum required for the
tutorial. I should be able to get it under 100 KB without too much effort.

The :download: role is specifically for links to data included in the
source tree. For external links, the normal external link syntax works
just fine for downloads as well as documents. For example:

`Link Text <http://path/to/link/>\`\_

That said, a 600kb zip file in SVN will be just fine, especially if we
don't plan on modifying it frequently. In fact, I think this is
encouraged for resources needed in tutorials:

[dwins@anonymised.com geoserver]$ find . -name '*.zip' | xargs du -h
8.0K ./data/citewfs-1.1-h2/cite.db.zip
8.0K ./data/citewfs-1.1-h2-xlink/cite.db.zip
36K ./doc/user/source/tutorials/feature-pregeneralized/streams.zip
1.8M ./doc/user/source/getting-started/shapefile-quickstart/roads.zip
108K ./doc/user/source/getting-started/shapefile-quickstart/nyc_roads.zip
576K ./doc/user/source/getting-started/postgis-quickstart/nyc_buildings.zip
576K ./doc/user/build/html/_downloads/nyc_buildings.zip
36K ./doc/user/build/html/_downloads/streams.zip
108K ./doc/user/build/html/_downloads/nyc_roads.zip
4.0K ./src/extension/restconfig/src/test/java/org/geoserver/catalog/rest/test-data/pds.zip
1.1M ./src/extension/restconfig/src/test/java/org/geoserver/catalog/rest/test-data/usa.zip
4.0K ./src/community/hello_web/minimal.zip
4.0K ./src/community/restconfig_temp/src/test/java/org/geoserver/catalog/rest/test-data/pds.zip
1.1M ./src/community/restconfig_temp/src/test/java/org/geoserver/catalog/rest/test-data/usa.zip
24K ./src/community/cite/wms-1.1.1.zip
4.0K ./src/community/cite/wcs-1.0.0.zip

--
David Winslow
OpenGeo - http://opengeo.org/

Ben_Caradoc-Davies · June 10, 2009, 5:08am

Thanks, Gabriel, for the historical perspective.

Changing OCQL to ECQL would break all existing configuration files. We do not yet have backwards compatibility support for mapping files. I have a task in a private Jira to review this format, but I would like to inflict it on some real end users for a while to gather data[1]. We (the app-schema team) should implement all file format changes resulting from this review in one go, while providing support for the old format. We also need to publish the XSD for the format (currently it is copied for each new configuration [yuck]).

Kind regards,
Ben.

[1] "I've experiments to run.
There is research to be done.
On the people who are still alive."

Gabriel Roldan wrote:

This was implemented before the existence of the CQL module (hence used to use ExpressionBuilder), and choosed the OCQL acronym to mean OGC Commom Query Language, as the original name of the language is.

I think we should change to ECQL now that's there's a reference to the extended dialect.

Cheers,
Gabriel
Ben Caradoc-Davies wrote:

Andrea Aime wrote:
[in an example app-schema mapping file]

- in gml:id mapping, why OCQL? I've heard of ECQL (extended) but never
about OCQL

This was implemented before my time on the project.

Gabriel, Rob A, or Jody. What is the origin of the term OCQL, as used in the community-schemas/app-schema mapping file?

Kind regards,

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

Ben_Caradoc-Davies · June 10, 2009, 5:27am

David Winslow wrote:

That said, a 600kb zip file in SVN will be just fine, especially if we
don't plan on modifying it frequently. In fact, I think this is
encouraged for resources needed in tutorials:

OK. The more I think about it, the more I think that the zip file is not different to any other explanatory binary resource, such as a png. I will put my schema cache on a diet, zip the config, and put it in svn.

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

Justin_Deoliveira · June 10, 2009, 2:06pm

<snip>

The Sphinx documentation for :download: only mentions local files. An examination of the source indicates that shutil.copyfile() is used to copy downloadable files, so it looks to me that URL downloads are not supported.

I am reluctant to include a zip in svn.

Which is less bad: a link to zip on a CSIRO download server, or a 604 KB zip committed to svn?

If committing the zip to svn is preferred, I can reduce the massively bloated OASIS catalog and cut it down to the minimum required for the tutorial. I should be able to get it under 100 KB without too much effort.

I usually try to discourage the storage of binary data in svn but admit that in some cases it is necessary. And 600k is not all that much, I would say go for it.

-Justin

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.