[Geoserver-devel] Problems with raster data having dots in name

Hi list,
I was playing around today with geoserver trunk code and I noticed
that coverages with dot as part of the name do not work anymore due to
the new config/ui work.
This should be seen as a regression since it is quite frequent to have
raster data generated with dots as part of the name.
Should I opne upa Jira?

Ciao,
Simone.
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder - Software Engineer
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------

Simone Giannecchini ha scritto:

Hi list,
I was playing around today with geoserver trunk code and I noticed
that coverages with dot as part of the name do not work anymore due to
the new config/ui work.
This should be seen as a regression since it is quite frequent to have
raster data generated with dots as part of the name.
Should I opne upa Jira?

I guess so. I know we have had issues with feature type names with dots
as well, as they are common type names in SDE land, but they are not
valid in WFS 1.1 (not sure about WFS 1.0).

The current code in BasicResourceConfig adds a pattern validator
like: [\\w_]\\w* (either a word char or a underscore and then only
word chars).

First off it would be nice to have a list of the valid patterns
for the various services, and then we have to decide if being
lax and allow invalid chars in the name for some services or not
(e.g. say the dot is valid for wfs 1.0 but on wfs 1.1, what do
we do?)

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

Andrea Aime wrote:

Simone Giannecchini ha scritto:

Hi list,
I was playing around today with geoserver trunk code and I noticed
that coverages with dot as part of the name do not work anymore due to
the new config/ui work.
This should be seen as a regression since it is quite frequent to have
raster data generated with dots as part of the name.
Should I opne upa Jira?

I guess so. I know we have had issues with feature type names with dots
as well, as they are common type names in SDE land, but they are not
valid in WFS 1.1 (not sure about WFS 1.0).

The current code in BasicResourceConfig adds a pattern validator
like: [\\w_]\\w* (either a word char or a underscore and then only
word chars).

First off it would be nice to have a list of the valid patterns
for the various services, and then we have to decide if being
lax and allow invalid chars in the name for some services or not
(e.g. say the dot is valid for wfs 1.0 but on wfs 1.1, what do
we do?)

aliasing works (around) just fine.
Irony is, dots _are_ valid characters for an element and attribute name (except for the first character in the name) in XML. They're valid both for xml Name and NCName[1].
Not completely sure, but it looks like they're valid since November 2008[2].
The WFS 1.1 spec, section 7.3, also explicitly quotes the xml spec NCName production rule and dots are included:

#quote
[4] NCName ::= (Letter | '_') (NCNameChar)*
/* An XML Name, minus the ":" */
[5] NCNameChar ::= Letter | Digit | '.' | '-' | '_' | CombiningChar | Extender
[6] QName ::= (Prefix ':')? LocalPart
[7] Prefix ::= NCName
[8] LocalPart ::= NCName
#/quote

and the attached xsd and xml instance do validate, so it's confirmed.
Yet I wonder why we (me included) were so convinced dots were not allowed? No idea..

Cheers,
Gabriel

[1]<http://www.w3.org/TR/2009/CR-xmlschema11-2-20090430/datatypes.html#NCName&gt;
[2]<http://www.w3.org/XML/xml-V10-4e-errata#E09&gt;

Cheers
Andrea

--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.

(attachments)

nametest.xml (489 Bytes)
nametest.xsd (856 Bytes)

Just to clarify,
going from memory I seem to remember that I have many coverages
configured in various installations whose name contains: ^,[,],(,)
etc. etc.
Are we saying that these coverages won't work with GeoServer 2.0? I
hope not, I would have an hard time convincing people to rework their
configurations that contains thousands of layers!

Simone.

-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder - Software Engineer
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------

On Thu, Sep 10, 2009 at 7:23 AM, Gabriel Roldan <groldan@anonymised.com> wrote:

Andrea Aime wrote:

Simone Giannecchini ha scritto:

Hi list,
I was playing around today with geoserver trunk code and I noticed
that coverages with dot as part of the name do not work anymore due to
the new config/ui work.
This should be seen as a regression since it is quite frequent to have
raster data generated with dots as part of the name.
Should I opne upa Jira?

I guess so. I know we have had issues with feature type names with dots
as well, as they are common type names in SDE land, but they are not
valid in WFS 1.1 (not sure about WFS 1.0).

The current code in BasicResourceConfig adds a pattern validator
like: [\\w_]\\w* (either a word char or a underscore and then only
word chars).

First off it would be nice to have a list of the valid patterns
for the various services, and then we have to decide if being
lax and allow invalid chars in the name for some services or not
(e.g. say the dot is valid for wfs 1.0 but on wfs 1.1, what do
we do?)

aliasing works (around) just fine.
Irony is, dots _are_ valid characters for an element and attribute name
(except for the first character in the name) in XML. They're valid both for
xml Name and NCName[1].
Not completely sure, but it looks like they're valid since November 2008[2].
The WFS 1.1 spec, section 7.3, also explicitly quotes the xml spec NCName
production rule and dots are included:

#quote
[4] NCName ::= (Letter | '_') (NCNameChar)*
/* An XML Name, minus the ":" */
[5] NCNameChar ::= Letter | Digit | '.' | '-' | '_' | CombiningChar |
Extender
[6] QName ::= (Prefix ':')? LocalPart
[7] Prefix ::= NCName
[8] LocalPart ::= NCName
#/quote

and the attached xsd and xml instance do validate, so it's confirmed.
Yet I wonder why we (me included) were so convinced dots were not allowed?
No idea..

Cheers,
Gabriel

[1]<http://www.w3.org/TR/2009/CR-xmlschema11-2-20090430/datatypes.html#NCName&gt;
[2]<http://www.w3.org/XML/xml-V10-4e-errata#E09&gt;

Cheers
Andrea

--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus
on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

afaik, coverage names do not have the same restrictions than WFS type names, neither WMS layer names have.
This might be trivial once the resource/publish split is there, but we're not there yet.

_Ideally_, "native" names could be whatever you want, and there might be a publishing configuration where you can assign a published name to a resource (that's LayerInfo). But we're honestly far from there yet.
The least we could do though, for 2.0, is leave coverage names be whatever (or almost?) and FeatureTypes follow the NCName production rule so they're valid for WFS _and_ they can still contain '.' | '-' | '_' among letters and digits?
What do you think Andrea?

Cheers,
Gabriel

Simone Giannecchini wrote:

Just to clarify,
going from memory I seem to remember that I have many coverages
configured in various installations whose name contains: ^,[,],(,)
etc. etc.
Are we saying that these coverages won't work with GeoServer 2.0? I
hope not, I would have an hard time convincing people to rework their
configurations that contains thousands of layers!

Simone.

-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder - Software Engineer
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------

On Thu, Sep 10, 2009 at 7:23 AM, Gabriel Roldan <groldan@anonymised.com> wrote:

Andrea Aime wrote:

Simone Giannecchini ha scritto:

Hi list,
I was playing around today with geoserver trunk code and I noticed
that coverages with dot as part of the name do not work anymore due to
the new config/ui work.
This should be seen as a regression since it is quite frequent to have
raster data generated with dots as part of the name.
Should I opne upa Jira?

I guess so. I know we have had issues with feature type names with dots
as well, as they are common type names in SDE land, but they are not
valid in WFS 1.1 (not sure about WFS 1.0).

The current code in BasicResourceConfig adds a pattern validator
like: [\\w_]\\w* (either a word char or a underscore and then only
word chars).

First off it would be nice to have a list of the valid patterns
for the various services, and then we have to decide if being
lax and allow invalid chars in the name for some services or not
(e.g. say the dot is valid for wfs 1.0 but on wfs 1.1, what do
we do?)

aliasing works (around) just fine.
Irony is, dots _are_ valid characters for an element and attribute name
(except for the first character in the name) in XML. They're valid both for
xml Name and NCName[1].
Not completely sure, but it looks like they're valid since November 2008[2].
The WFS 1.1 spec, section 7.3, also explicitly quotes the xml spec NCName
production rule and dots are included:

#quote
[4] NCName ::= (Letter | '_') (NCNameChar)*
/* An XML Name, minus the ":" */
[5] NCNameChar ::= Letter | Digit | '.' | '-' | '_' | CombiningChar |
Extender
[6] QName ::= (Prefix ':')? LocalPart
[7] Prefix ::= NCName
[8] LocalPart ::= NCName
#/quote

and the attached xsd and xml instance do validate, so it's confirmed.
Yet I wonder why we (me included) were so convinced dots were not allowed?
No idea..

Cheers,
Gabriel

[1]<http://www.w3.org/TR/2009/CR-xmlschema11-2-20090430/datatypes.html#NCName&gt;
[2]<http://www.w3.org/XML/xml-V10-4e-errata#E09&gt;

Cheers
Andrea

--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus
on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.

Gabriel Roldan ha scritto:

Andrea Aime wrote:

Simone Giannecchini ha scritto:

Hi list,
I was playing around today with geoserver trunk code and I noticed
that coverages with dot as part of the name do not work anymore due to
the new config/ui work.
This should be seen as a regression since it is quite frequent to have
raster data generated with dots as part of the name.
Should I opne upa Jira?

I guess so. I know we have had issues with feature type names with dots
as well, as they are common type names in SDE land, but they are not
valid in WFS 1.1 (not sure about WFS 1.0).

The current code in BasicResourceConfig adds a pattern validator
like: [\\w_]\\w* (either a word char or a underscore and then only
word chars).

First off it would be nice to have a list of the valid patterns
for the various services, and then we have to decide if being
lax and allow invalid chars in the name for some services or not
(e.g. say the dot is valid for wfs 1.0 but on wfs 1.1, what do
we do?)

aliasing works (around) just fine.
Irony is, dots _are_ valid characters for an element and attribute name (except for the first character in the name) in XML. They're valid both for xml Name and NCName[1].
Not completely sure, but it looks like they're valid since November 2008[2].
The WFS 1.1 spec, section 7.3, also explicitly quotes the xml spec NCName production rule and dots are included:

#quote
[4] NCName ::= (Letter | '_') (NCNameChar)*
/* An XML Name, minus the ":" */
[5] NCNameChar ::= Letter | Digit | '.' | '-' | '_' | CombiningChar | Extender
[6] QName ::= (Prefix ':')? LocalPart
[7] Prefix ::= NCName
[8] LocalPart ::= NCName
#/quote

and the attached xsd and xml instance do validate, so it's confirmed.
Yet I wonder why we (me included) were so convinced dots were not allowed? No idea..

If I try to validate the following:

<wfs:GetFeature service="WFS" version="1.1.0"
   xmlns:topp="http://www.openplans.org/topp&quot;
   xmlns:wfs="http://www.opengis.net/wfs&quot;
   xmlns:ogc="http://www.opengis.net/ogc&quot;
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot;
   xsi:schemaLocation="http://www.opengis.net/wfs
                       http://schemas.opengis.net/wfs/1.1.0/wfs.xsd&quot;&gt;
   <wfs:Query typeName="topp:states.a.b.c">
     <ogc:Filter>
        <ogc:FeatureId fid="states.3"/>
     </ogc:Filter>
     </wfs:Query>
</wfs:GetFeature>

Eclipse states:

cvc-attribute.3: The value 'topp:states.a.b.c' of attribute 'typeName' on element 'wfs:Query' is not valid with respect to its type, 'TypeNameListType'. WFS_getFeature-1.1.xml /data-directories/release/demo line 8 XML Problem

The same happens if I use any of the other chars in the production,
like - or _. The reason is simple, TypeNameListType has an explicit
definition of the valid pattern:

    <xsd:simpleType name="TypeNameListType">
       <xsd:restriction base="wfs:Base_TypeNameListType">
          <xsd:pattern value="((\w:)?\w(=\w)?){1,}">

which completely forgets about what a QName is.

I've tried to validate a similar document but for wfs 1.0, that worked
fine, as well as similar tests with wcs 1.0 and wcs 1.1

So I guess we can allow the qname allowed chars, but that will result
in invalid WFS 1.1 setups, at least, it will not be possible to make
a schema valid GetFeature XML request.

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

Andrea Aime ha scritto:

Gabriel Roldan ha scritto:

Andrea Aime wrote:

Simone Giannecchini ha scritto:

Hi list,
I was playing around today with geoserver trunk code and I noticed
that coverages with dot as part of the name do not work anymore due to
the new config/ui work.
This should be seen as a regression since it is quite frequent to have
raster data generated with dots as part of the name.
Should I opne upa Jira?

I guess so. I know we have had issues with feature type names with dots
as well, as they are common type names in SDE land, but they are not
valid in WFS 1.1 (not sure about WFS 1.0).

The current code in BasicResourceConfig adds a pattern validator
like: [\\w_]\\w* (either a word char or a underscore and then only
word chars).

First off it would be nice to have a list of the valid patterns
for the various services, and then we have to decide if being
lax and allow invalid chars in the name for some services or not
(e.g. say the dot is valid for wfs 1.0 but on wfs 1.1, what do
we do?)

aliasing works (around) just fine.
Irony is, dots _are_ valid characters for an element and attribute name (except for the first character in the name) in XML. They're valid both for xml Name and NCName[1].
Not completely sure, but it looks like they're valid since November 2008[2].
The WFS 1.1 spec, section 7.3, also explicitly quotes the xml spec NCName production rule and dots are included:

#quote
[4] NCName ::= (Letter | '_') (NCNameChar)*
/* An XML Name, minus the ":" */
[5] NCNameChar ::= Letter | Digit | '.' | '-' | '_' | CombiningChar | Extender
[6] QName ::= (Prefix ':')? LocalPart
[7] Prefix ::= NCName
[8] LocalPart ::= NCName
#/quote

and the attached xsd and xml instance do validate, so it's confirmed.
Yet I wonder why we (me included) were so convinced dots were not allowed? No idea..

If I try to validate the following:

<wfs:GetFeature service="WFS" version="1.1.0"
   xmlns:topp="http://www.openplans.org/topp&quot;
   xmlns:wfs="http://www.opengis.net/wfs&quot;
   xmlns:ogc="http://www.opengis.net/ogc&quot;
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot;
   xsi:schemaLocation="http://www.opengis.net/wfs
                       http://schemas.opengis.net/wfs/1.1.0/wfs.xsd&quot;&gt;
   <wfs:Query typeName="topp:states.a.b.c">
     <ogc:Filter>
        <ogc:FeatureId fid="states.3"/>
     </ogc:Filter>
     </wfs:Query>
</wfs:GetFeature>

Eclipse states:

cvc-attribute.3: The value 'topp:states.a.b.c' of attribute 'typeName' on element 'wfs:Query' is not valid with respect to its type, 'TypeNameListType'. WFS_getFeature-1.1.xml /data-directories/release/demo line 8 XML Problem

The same happens if I use any of the other chars in the production,
like - or _. The reason is simple, TypeNameListType has an explicit
definition of the valid pattern:

    <xsd:simpleType name="TypeNameListType">
       <xsd:restriction base="wfs:Base_TypeNameListType">
          <xsd:pattern value="((\w:)?\w(=\w)?){1,}">

which completely forgets about what a QName is.

I've tried to validate a similar document but for wfs 1.0, that worked
fine, as well as similar tests with wcs 1.0 and wcs 1.1

So I guess we can allow the qname allowed chars, but that will result
in invalid WFS 1.1 setups, at least, it will not be possible to make
a schema valid GetFeature XML request.

Oh, for coverages there are really no limitations, from the WCS 1.1 schema:

<simpleType name="IdentifierType">
    <annotation>
      <documentation>Unambiguous identifier. Although there is no formal restriction on characters included, these identifiers shall be directly usable in GetCoverage operation requests for the specific server, whether those requests are encoded in KVP or XML. Each of these encodings requires that certain characters be avoided, encoded, or escaped (TBR). </documentation>
    </annotation>
    <restriction base="string">
      <pattern value=".+"/>
    </restriction>
  </simpleType>

The coverage identifier can be absolutely anything (even a white space... now this is a bit too much freedom imho). The WCS 1.0
similarly places no restriction... not even that at least one
char is required (who wants the un-named coverage?).

So I guess for coverages we just place a check for at least one
char and avoid a string made only with spaces, and for feature types
we accept the NCName production taking the consequences for WFS 1.1
requests?
Btw, does anybody have a pattern for the NCName production? I don't
know what a combining char is

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.