Justin,
my new format data dir was being broken by whitespace around a url entry in datastore.xml. Whose responsibility is it to trim whitespace around connection parameters? The data access (factory) or the GS catalog implementation? Let me know and I'll lodge the Jira issue.
Kind regards,
--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia
Ben Caradoc-Davies ha scritto:
Justin,
my new format data dir was being broken by whitespace around a url entry in datastore.xml. Whose responsibility is it to trim whitespace around connection parameters? The data access (factory) or the GS catalog implementation? Let me know and I'll lodge the Jira issue.
I would say... the catalog (and more than the catalog itself, I'd
say the UI and the persistence layers?)
Cheers
Andrea
--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Good question. Doing this at the catalog level when every string is read would be kind of tough. Also since we use xstream to read kind of hard to do, although we could register a custom converter for String and have it trim the result. That would probably not play all that well with things like titles and descriptions though that may have whitespace.
I am tempted to say the right answer is to have client code be careful about what it puts in the catalog.
-Justin
Ben Caradoc-Davies wrote:
Justin,
my new format data dir was being broken by whitespace around a url entry in datastore.xml. Whose responsibility is it to trim whitespace around connection parameters? The data access (factory) or the GS catalog implementation? Let me know and I'll lodge the Jira issue.
Kind regards,
--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.
the URL is a DataAccess parameter?
if so, the DataAccessFactory shouldn't accept it as valid. That is the correct check I think. This way, the UI doesn't need to guess what's valid and what's not for every data store, but the store factory itself shouldn't accept invalid entries.
if it's not a dataaccess parameter what's it?
btw, I just found a defect in the DataAccessEditPage that it makes saving two datastore parameters that are not actual parameters but some page speficic stuff like "Wicket_Workspace".
Cheers,
Gabriel
Ben Caradoc-Davies wrote:
Justin,
my new format data dir was being broken by whitespace around a url entry in datastore.xml. Whose responsibility is it to trim whitespace around connection parameters? The data access (factory) or the GS catalog implementation? Let me know and I'll lodge the Jira issue.
Kind regards,
--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.
The URL is the location of the app-schema mapping file. This used to be in the old catalog.xml.
Gabriel Roldan wrote:
the URL is a DataAccess parameter?
if so, the DataAccessFactory shouldn't accept it as valid. That is the correct check I think. This way, the UI doesn't need to guess what's valid and what's not for every data store, but the store factory itself shouldn't accept invalid entries.if it's not a dataaccess parameter what's it?
btw, I just found a defect in the DataAccessEditPage that it makes saving two datastore parameters that are not actual parameters but some page speficic stuff like "Wicket_Workspace".
Cheers,
Gabriel
Ben Caradoc-Davies wrote:
Justin,
my new format data dir was being broken by whitespace around a url entry in datastore.xml. Whose responsibility is it to trim whitespace around connection parameters? The data access (factory) or the GS catalog implementation? Let me know and I'll lodge the Jira issue.
Kind regards,
--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia
Ben Caradoc-Davies wrote:
The URL is the location of the app-schema mapping file. This used to be in the old catalog.xml.
cool, but do you agree the place where to put the parameter validation is in the dataaccess factory itself?
I'm not clear if the error happens by extra whitespace is being introduced by the user and not catched up by the application as an invalid parameter, or the application is somehow introducing it.
Cheers,
Gabriel
Gabriel Roldan wrote:
the URL is a DataAccess parameter?
if so, the DataAccessFactory shouldn't accept it as valid. That is the correct check I think. This way, the UI doesn't need to guess what's valid and what's not for every data store, but the store factory itself shouldn't accept invalid entries.if it's not a dataaccess parameter what's it?
btw, I just found a defect in the DataAccessEditPage that it makes saving two datastore parameters that are not actual parameters but some page speficic stuff like "Wicket_Workspace".
Cheers,
Gabriel
Ben Caradoc-Davies wrote:
Justin,
my new format data dir was being broken by whitespace around a url entry in datastore.xml. Whose responsibility is it to trim whitespace around connection parameters? The data access (factory) or the GS catalog implementation? Let me know and I'll lodge the Jira issue.
Kind regards,
--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Justin Deoliveira wrote:
I am tempted to say the right answer is to have client code be careful about what it puts in the catalog.
"Client code" in this case is the test editor used by an end user, manually entering their configuration details. Formatting datastore.xml breaks it. This must be fixed. Is there any case in which leading or trailing whitespace is significant in datstore.xml connection parameters?
--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia
Gabriel Roldan wrote:
Ben Caradoc-Davies wrote:
The URL is the location of the app-schema mapping file. This used to be in the old catalog.xml.
cool, but do you agree the place where to put the parameter validation is in the dataaccess factory itself?
I'm not clear if the error happens by extra whitespace is being introduced by the user and not catched up by the application as an invalid parameter, or the application is somehow introducing it.
The end user is editing the XML and formats the file before saving, introducing whitespace.
(Note: this will be a bit mangled by email:)
This works:
<dataStore>
<id>gsml_MappedFeature_datastore</id>
<name>gsml_MappedFeature</name>
<enabled>true</enabled>
<workspace>
<id>gsml_workspace</id>
</workspace>
<connectionParameters>
<entry key="namespace">urn:cgi:xmlns:CGI:GeoSciML:2.0
</entry>
<entry key="url">file:workspaces/gsml/gsml_MappedFeature/gsml_MappedFeature.xml</entry>
<entry key="dbtype">app-schema</entry>
</connectionParameters>
</dataStore>
This does not:
<dataStore>
<id>gsml_MappedFeature_datastore</id>
<name>gsml_MappedFeature</name>
<enabled>true</enabled>
<workspace>
<id>gsml_workspace</id>
</workspace>
<connectionParameters>
<entry key="namespace">urn:cgi:xmlns:CGI:GeoSciML:2.0
</entry>
<entry key="url">
file:workspaces/gsml/gsml_MappedFeature/gsml_MappedFeature.xml
</entry>
<entry key="dbtype">app-schema</entry>
</connectionParameters>
</dataStore>
--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia
Ben Caradoc-Davies wrote:
Justin Deoliveira wrote:
I am tempted to say the right answer is to have client code be careful about what it puts in the catalog.
"Client code" in this case is the test editor used by an end user, manually entering their configuration details. Formatting datastore.xml breaks it. This must be fixed. Is there any case in which leading or trailing whitespace is significant in datstore.xml connection parameters?
k, now I get it. I seem to recall hearing that XStream does not support (or at least does not use) xml schemas, hence we don't have a schema for the config files nor a schema would be respected by XStream? I guess ability to specify where whitespace should be preserved or not through a schema would be nice but I admit I'm totally ignorant about XStream, but it seems if it had a schema and XStream is configured to use the StAXDriver it should work..
--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Ben Caradoc-Davies ha scritto:
Justin Deoliveira wrote:
I am tempted to say the right answer is to have client code be careful about what it puts in the catalog.
"Client code" in this case is the test editor used by an end user, manually entering their configuration details. Formatting datastore.xml breaks it. This must be fixed.
Then go ahead and fix it. The sources of GeoServer are there for you to
patch no?
I understand your frustration, but going around ordering people to alter
code to solve a problem you're experiencing is not going to speed up
things. In fact, it just makes other developers grumpy and less willing
to help out.
Is there any case in which leading or trailing whitespace is significant in datstore.xml connection parameters?
I don't think so.
Cheers
Andrea
--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Gabriel Roldan ha scritto:
Ben Caradoc-Davies wrote:
Justin Deoliveira wrote:
I am tempted to say the right answer is to have client code be careful about what it puts in the catalog.
"Client code" in this case is the test editor used by an end user, manually entering their configuration details. Formatting datastore.xml breaks it. This must be fixed. Is there any case in which leading or trailing whitespace is significant in datstore.xml connection parameters?
k, now I get it. I seem to recall hearing that XStream does not support (or at least does not use) xml schemas, hence we don't have a schema for the config files nor a schema would be respected by XStream? I guess ability to specify where whitespace should be preserved or not through a schema would be nice but I admit I'm totally ignorant about XStream, but it seems if it had a schema and XStream is configured to use the StAXDriver it should work..
XStream is a XML serialization library, not a XML binding one, that
makes a world of difference. It just serializes and deserializes
objects in a very comfortable xml format and is quite tolerant to
structural changes (e.g., can still load older files if you added
or removed fields in your objects).
So it's a step up over plain java serialization,
but the fact that it's dumping in xml instead of json or any other
format is incidental (in fact, latest version do dump in json as well).
Maybe we should stop using xml and use json instead to make the point
clear.
If you want schema support, well, XStream won't offer that.
The way would be to make xml schemas by hand, build bindings compatible
with gt-xsd (last time I checked it was not possible to keep jaxb and
gt-xsd in the same classpath and have both working) and propose them as
a replacement.
As to how make XStream trim out whitespaces when it's reading
the xml, I looked around a bit but did not find any leads
on that, sorry.
Cheers
Andrea
--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Andrea Aime wrote:
Ben Caradoc-Davies ha scritto:
"Client code" in this case is the test editor used by an end user, manually entering their configuration details. Formatting datastore.xml breaks it. This must be fixed.
Then go ahead and fix it. The sources of GeoServer are there for you to
patch no?
I understand your frustration, but going around ordering people to alter
code to solve a problem you're experiencing is not going to speed up
things. In fact, it just makes other developers grumpy and less willing
to help out.
Andrea! My "must be fixed" was an expression of UI design opinion, not a work instruction.
"this must be fixed" != "you must fix this"
The problem is that it is not clear where this fix can be applied, let alone who we can persuade to do it.
--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia
Ben Caradoc-Davies ha scritto:
Andrea Aime wrote:
Ben Caradoc-Davies ha scritto:
"Client code" in this case is the test editor used by an end user, manually entering their configuration details. Formatting datastore.xml breaks it. This must be fixed.
Then go ahead and fix it. The sources of GeoServer are there for you to
patch no?
I understand your frustration, but going around ordering people to alter
code to solve a problem you're experiencing is not going to speed up
things. In fact, it just makes other developers grumpy and less willing
to help out.Andrea! My "must be fixed" was an expression of UI design opinion, not a work instruction.
"this must be fixed" != "you must fix this"
Sorry, I read too much in your statement.
When I say "this must be fixed" instead of "I'm going to fix it" I
really mean someone else has to fix it (otherwise I propose myself for fixing it to start with), I wrongly assumed it was the same for you.
The problem is that it is not clear where this fix can be applied, let alone who we can persuade to do it.
Well, the jury is still out on whether this "must" be fixed too imho.
The GeoServer configuration files have never been meant to be edited
manually. As I said, I looked into a fix for XStream, but I could
not find one with a quick search.
Maybe Justin has more insight.
Cheers
Andrea
--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Well, the jury is still out on whether this "must" be fixed too imho.
The GeoServer configuration files have never been meant to be edited
manually. As I said, I looked into a fix for XStream, but I could
not find one with a quick search.
Maybe Justin has more insight.
I think my initial statement still holds, if it is the client that is producing "garbage", it is the client that should clean it up. While in this case it would be easy enough to trim() at the xstream persistence level, but that would only work for some cases. What if the client specifies:
<element>
Some pretty bad
ly formatted te
xt in this elem
</element>
A simple trim won't handle that case, and there are probably others. So my preference would be to ensure that the XML being send is clean.
However, I am willing to compromise if you think that is going to be too much of a hassle. We already use a custom Map encoder for encoding and decoding the maps for connection paramaters with xstream. It would be easy enough to throw a trim in there.
Cheers
Andrea
--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.
Andrea Aime wrote:
Ben Caradoc-Davies ha scritto:
Andrea Aime wrote:
Ben Caradoc-Davies ha scritto:
"Client code" in this case is the test editor used by an end user, manually entering their configuration details. Formatting datastore.xml breaks it. This must be fixed.
Then go ahead and fix it. The sources of GeoServer are there for you to
patch no?
I understand your frustration, but going around ordering people to alter
code to solve a problem you're experiencing is not going to speed up
things. In fact, it just makes other developers grumpy and less willing
to help out.Andrea! My "must be fixed" was an expression of UI design opinion, not a work instruction.
"this must be fixed" != "you must fix this"
Sorry, I read too much in your statement.
When I say "this must be fixed" instead of "I'm going to fix it" I
really mean someone else has to fix it (otherwise I propose myself for fixing it to start with), I wrongly assumed it was the same for you.The problem is that it is not clear where this fix can be applied, let alone who we can persuade to do it.
Well, the jury is still out on whether this "must" be fixed too imho.
The GeoServer configuration files have never been meant to be edited
manually. As I said, I looked into a fix for XStream, but I could
not find one with a quick search.
Maybe Justin has more insight.
My 5-minute research tells me this can be easily fixed by:
- having a xml schema for the config files, so we can tell where whitespace is significant
- configuring XStream to use the StAXDriver.
I wish I had a free hour to try it out though.
Cheers,
Gabriel
Cheers
Andrea
--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Gabriel Roldan wrote:
Andrea Aime wrote:
Ben Caradoc-Davies ha scritto:
Andrea Aime wrote:
Ben Caradoc-Davies ha scritto:
"Client code" in this case is the test editor used by an end user, manually entering their configuration details. Formatting datastore.xml breaks it. This must be fixed.
Then go ahead and fix it. The sources of GeoServer are there for you to
patch no?
I understand your frustration, but going around ordering people to alter
code to solve a problem you're experiencing is not going to speed up
things. In fact, it just makes other developers grumpy and less willing
to help out.Andrea! My "must be fixed" was an expression of UI design opinion, not a work instruction.
"this must be fixed" != "you must fix this"
Sorry, I read too much in your statement.
When I say "this must be fixed" instead of "I'm going to fix it" I
really mean someone else has to fix it (otherwise I propose myself for fixing it to start with), I wrongly assumed it was the same for you.The problem is that it is not clear where this fix can be applied, let alone who we can persuade to do it.
Well, the jury is still out on whether this "must" be fixed too imho.
The GeoServer configuration files have never been meant to be edited
manually. As I said, I looked into a fix for XStream, but I could
not find one with a quick search.
Maybe Justin has more insight.My 5-minute research tells me this can be easily fixed by:
- having a xml schema for the config files, so we can tell where whitespace is significant
- configuring XStream to use the StAXDriver.I wish I had a free hour to try it out though.
Coming up with a schema for the config files would take a while, and would be quite painful imo. I have always said that we should discourage as much as possible people reading + writing of configuration files directly, and I stand by that. And how that we have a REST api, I would rather spend my time there.
Also as Andrea stated, XSTream is not meant to be used with a XML schema. If we want more structure to the XML we encode we should indeed abandon xstream for something like XMLBeans, or JAXB, a true "binding" framework.
Cheers,
Gabriel
Cheers
Andrea
--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.
Andrea Aime wrote:
Gabriel Roldan ha scritto:
Ben Caradoc-Davies wrote:
Justin Deoliveira wrote:
I am tempted to say the right answer is to have client code be careful about what it puts in the catalog.
"Client code" in this case is the test editor used by an end user, manually entering their configuration details. Formatting datastore.xml breaks it. This must be fixed. Is there any case in which leading or trailing whitespace is significant in datstore.xml connection parameters?
k, now I get it. I seem to recall hearing that XStream does not support (or at least does not use) xml schemas, hence we don't have a schema for the config files nor a schema would be respected by XStream? I guess ability to specify where whitespace should be preserved or not through a schema would be nice but I admit I'm totally ignorant about XStream, but it seems if it had a schema and XStream is configured to use the StAXDriver it should work..
XStream is a XML serialization library, not a XML binding one, that
makes a world of difference. It just serializes and deserializes
objects in a very comfortable xml format and is quite tolerant to
structural changes (e.g., can still load older files if you added
or removed fields in your objects).So it's a step up over plain java serialization,
but the fact that it's dumping in xml instead of json or any other
format is incidental (in fact, latest version do dump in json as well).
Maybe we should stop using xml and use json instead to make the point
clear.If you want schema support, well, XStream won't offer that.
The way would be to make xml schemas by hand, build bindings compatible
with gt-xsd (last time I checked it was not possible to keep jaxb and
gt-xsd in the same classpath and have both working) and propose them as
a replacement.As to how make XStream trim out whitespaces when it's reading
the xml, I looked around a bit but did not find any leads
on that, sorry.
ok disregard the schema suggestion if it seems like too much grief. I know XStream is not schema aware and that's not what I'm suggesting. Still, it does use a xml parser to read the files and that's why I thought, based on some google results, that the low level parser XStream uses would still parse the strings correctly as far as there's a manually created schema and the parser is the StAX one.
trimming at the custom map encoder seems far easier, though a bit 'corner case dangerous' if its ever used for anything else than dataaccess params.
ah, the easiest solution ever would be for the param value to be an xml attribute instead of an element's content. Uglier model wise though...
Gabriel
Cheers
Andrea
--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Justin Deoliveira wrote:
Gabriel Roldan wrote:
Andrea Aime wrote:
Ben Caradoc-Davies ha scritto:
Andrea Aime wrote:
Ben Caradoc-Davies ha scritto:
"Client code" in this case is the test editor used by an end user, manually entering their configuration details. Formatting datastore.xml breaks it. This must be fixed.
Then go ahead and fix it. The sources of GeoServer are there for you to
patch no?
I understand your frustration, but going around ordering people to alter
code to solve a problem you're experiencing is not going to speed up
things. In fact, it just makes other developers grumpy and less willing
to help out.Andrea! My "must be fixed" was an expression of UI design opinion, not a work instruction.
"this must be fixed" != "you must fix this"
Sorry, I read too much in your statement.
When I say "this must be fixed" instead of "I'm going to fix it" I
really mean someone else has to fix it (otherwise I propose myself for fixing it to start with), I wrongly assumed it was the same for you.The problem is that it is not clear where this fix can be applied, let alone who we can persuade to do it.
Well, the jury is still out on whether this "must" be fixed too imho.
The GeoServer configuration files have never been meant to be edited
manually. As I said, I looked into a fix for XStream, but I could
not find one with a quick search.
Maybe Justin has more insight.My 5-minute research tells me this can be easily fixed by:
- having a xml schema for the config files, so we can tell where whitespace is significant
- configuring XStream to use the StAXDriver.I wish I had a free hour to try it out though.
Coming up with a schema for the config files would take a while, and would be quite painful imo. I have always said that we should discourage as much as possible people reading + writing of configuration files directly, and I stand by that. And how that we have a REST api, I would rather spend my time there.
Also as Andrea stated, XSTream is not meant to be used with a XML schema. If we want more structure to the XML we encode we should indeed abandon xstream for something like XMLBeans, or JAXB, a true "binding" framework.
Justin, Andrea: GOT IT. Just sent out what I found on google for a similar problem.
Gabriel
Cheers,
Gabriel
Cheers
Andrea
--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Gabriel Roldan ha scritto:
ok disregard the schema suggestion if it seems like too much grief. I know XStream is not schema aware and that's not what I'm suggesting. Still, it does use a xml parser to read the files and that's why I thought, based on some google results, that the low level parser XStream uses would still parse the strings correctly as far as there's a manually created schema and the parser is the StAX one.
I had thought in those direction too this morning but could not find
anything to substantiate them. Since XStream is depending on a separate
library to do the xml reading, is there any setting we can force into
such library to force the usual whitespace handling, which is
trim before and after, and collapse every subsequent space in the middle
into just one? (this should be the standard behaviour anyways, and this
confuses me a little, see the long discussions about newlines and extra
spaces in SLD Label element).
Cheers
Andrea
--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.