[Geoserver-devel] Integrated GWC hardcodes GRIDSET_ID VARCHAR(32), while Geoserver has longer strings

Hi,

Hi,

Both lists are OK. It is good that also other users get known to the issue, and because you did not get a working solution from the users list then dev list maybe puts more pressure for the developers to answer.

If the name is automatically created from the EPSG database by concatenating the name and code I believe that the right solution is to modify the table creation script to use for example VARCHAR(64). Change feels small but of course it may have side effects.

-Jukka Rahkonen-

-----Alkuperäinen viesti-----
Lähettäjä: vsinceac <victor.sinceac@anonymised.com>
Lähetetty: keskiviikko 28. lokakuuta 2020 11.41
Vastaanottaja: geoserver-devel@lists.sourceforge.net
Aihe: [Geoserver-devel] Integrated GWC hardcodes GRIDSET_ID VARCHAR(32), while Geoserver has longer strings

Hi,

GeoServer 2.18.0 with embedded GWC: after adding new "Gridset" (here, "WGS
84 / UTM Zone 34N (EPSG:32634)") in section "Tile Caching" of a layer, I got this error:

---------------
ERROR [geoserver.gwc] - Failed to get a quota store, the GeoWebCache disk quota subsystem will stop working now
org.springframework.dao.ConcurrencyFailureException: Failed to create or locate tileset TileSet[messir:countries_and_water#WGS 84 / UTM Zone 34N (EPSG:32634)#image/png#408a9289e97a734867475b5c5178621c355e6e28] after 100 attempts; nested exception is
org.geowebcache.diskquota.jdbc.ParametricDataAccessException: Failed to execute statement INSERT INTO TILESET select :key, :layerName, :gridSetId, :blobFormat, :parametersId, 0 WHERE NOT EXISTS(SELECT 1 FROM TILESET WHERE KEY = :key) with params: {blobFormat=image/png, gridSetId=WGS 84 / UTM Zone 34N (EPSG:32634), layerName=messir:countries_and_water,
parametersId=408a9289e97a734867475b5c5178621c355e6e28,
key=messir:countries_and_water#WGS 84 / UTM Zone 34N (EPSG:32634)#image/png#408a9289e97a734867475b5c5178621c355e6e28}; nested exception is org.springframework.jdbc.UncategorizedSQLException:
PreparedStatementCallback; uncategorized SQLException for SQL [INSERT INTO TILESET select ?, ?, ?, ?, ?, 0 WHERE NOT EXISTS(SELECT 1 FROM TILESET WHERE KEY = ?)]; SQL state [90005]; error code [90005]; Value too long for column GRIDSET_ID VARCHAR(32): 'WGS 84 / UTM Zone 34N (EPSG:32634)' (34); SQL statement:
INSERT INTO TILESET select ?, ?, ?, ?, ?, 0 WHERE NOT EXISTS(SELECT 1 FROM TILESET WHERE KEY = ?) [90005-119]; nested exception is
org.h2.jdbc.JdbcSQLException: Value too long for column GRIDSET_ID
VARCHAR(32): 'WGS 84 / UTM Zone 34N (EPSG:32634)' (34); [... ETC ...] Caused by: org.geowebcache.diskquota.jdbc.ParametricDataAccessException:
Failed to execute statement INSERT INTO TILESET select :key, :layerName, :gridSetId, :blobFormat, :parametersId, 0 WHERE NOT EXISTS(SELECT 1 FROM TILESET WHERE KEY = :key) with params: {blobFormat=image/png, gridSetId=WGS
84 / UTM Zone 34N (EPSG:32634), layerName=messir:countries_and_water,
parametersId=408a9289e97a734867475b5c5178621c355e6e28,
key=messir:countries_and_water#WGS 84 / UTM Zone 34N (EPSG:32634)#image/png#408a9289e97a734867475b5c5178621c355e6e28}; nested exception is org.springframework.jdbc.UncategorizedSQLException:
PreparedStatementCallback; uncategorized SQLException for SQL [INSERT INTO TILESET select ?, ?, ?, ?, ?, 0 WHERE NOT EXISTS(SELECT 1 FROM TILESET WHERE KEY = ?)]; SQL state [90005]; error code [90005]; Value too long for column GRIDSET_ID VARCHAR(32): 'WGS 84 / UTM Zone 34N (EPSG:32634)' (34); SQL statement:
[... ETC ...]
Caused by: org.springframework.jdbc.UncategorizedSQLException:
PreparedStatementCallback; uncategorized SQLException for SQL [INSERT INTO TILESET select ?, ?, ?, ?, ?, 0 WHERE NOT EXISTS(SELECT 1 FROM TILESET WHERE KEY = ?)]; SQL state [90005]; error code [90005]; Value too long for column GRIDSET_ID VARCHAR(32): 'WGS 84 / UTM Zone 34N (EPSG:32634)' (34); SQL statement:
[... ETC ...]
Caused by: org.h2.jdbc.JdbcSQLException: Value too long for column GRIDSET_ID VARCHAR(32): 'WGS 84 / UTM Zone 34N (EPSG:32634)' (34); SQL
statement:
[... ETC ...]
-----------------

Indeed, GWC has hardcoded value "GRIDSET_ID VARCHAR(32)" (see code below), while obviously GeoServer has strings with length > 32 chars to be recorded in this column.

Any idea on how to fix this (without rebuilding GeoServer)?

Many thanks,
Victor Sinceac

PS: I posted the same on "users" forum, but I guess "dev" forum is more appropriate...

geowebcache-master\geowebcache\diskquota\jdbc\src\main\java\org\geowebcache\diskquota\jdbc\SQLDialect.java
    // size guesses: 128 characters should be more than enough for layer name, the gridset id
    // is normally an epsg code so 32 is way more than enough, the blob format
    // is normally a mime plus some extras, again 64 should fit, a param id is
    // a SHA-1 sum that uses 41 chars, the id is the sum of all the above plus
    // connecting chars, 320 is again more than enough
    // bytes is going to be less than a zettabyte(one million petabytes,
10^21) for the
    // foreseeable future
....
    protected static final int GRIDSET_ID_SIZE = 32; .....
    protected final Map<String, List&lt;String>> TABLE_CREATION_MAP =
            new LinkedHashMap<String, List&lt;String>>() {
                {
                    put(
                            "TILESET",
                            Arrays.asList( //
                                    "CREATE TABLE ${schema}TILESET (\n"
                                            + // ....
                                            " GRIDSET_ID VARCHAR("
                                            + GRIDSET_ID_SIZE
                                            + "),\n"
.....

--
Sent from: http://osgeo-org.1560.x6.nabble.com/GeoServer-Dev-f3819232.html

_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Hi,

When using this plugin https://github.com/geoserver/geoserver/commits/main/src/community/schemaless-features on a MongoDB collection where values to be used by a GeoServer layer are declared as Object, the plugin builds a series of features where the feature type is based on the type of the 1st feature.

When layer expects values of type Double and the 1st feature found has type Double, everything goes fine: other values in collection with type Integer are casted to Double (e.g. value “29” is retrieved in the layer as “29.0”).

But, when 1st feature found has type Integer, everything goes bad: if other values have type Double in collection, they are casted to Integer (e.g. value “29.45” is retrieved in the layer as “29”).

This is a blocking issue for a layer based on such a collection…

Any idea how to fix this?

A simple workaround would be to systematically add something like “0.000000001” to every value in each document (this would force the object to have type Double). But this is decidedly ugly, cannot call it a solution…

Many thanks, Victor

Victor.Sinceac

|

Software Developer

Victor.Sinceac@anonymised.com

Tel: +33 1 45 73 60 60

Corobor Systems

|

41 Rue Périer

,

92120 MONTROUGE

,

France

Corobor Systems on LinkedIn

@corobor_systems


This message is subject to and does not create or vary any contractual relationship between Campbell Scientific Limited, its subsidiaries or affiliates (“CSL”) and you. Internet communications are not secure and therefore CSL does not accept legal responsibility for the contents of this message. The message is intended for the addressee only and its contents and any attached files are strictly confidential. If you have received it in error, please telephone +44 (0)1509 601141 or email postmaster@anonymised.com Thank you.
Campbell Scientific Ltd.
Company Registration Number: 1933935 (England and Wales) Registered Address: Campbell Park, 80 Hathern Road, Shepshed, Loughborough, Leicestershire LE12 9GX, UK
Tel: +44 (0) 1509 601141
Fax: +44 (0) 1509 601091
Web: www.campbellsci.co.uk


On Friday, 4 February 2022 8:32:23 PM AEDT Victor. Sinceac wrote:

Hi,

When using this plugin
https://github.com/geoserver/geoserver/commits/main/src/community/schemales
s-features

As indicated, this is a community module, so you probably shouldn't expect it
to work seamlessly for your use cases without some development. Have you tried
debugging it? Have you looked at the mapper code? What changes have you tried?

Brad

Hello Victor,
Currently there is no way to fix this without a code change. I believe the problematic line of code is this one where the value type is forced to be the same of the AttributeType (that indeed is build on the first feature retrieved).

Regards,

Marco Volpini

==
GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Marco Volpini

Software Engineer

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it


Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

On Fri, Feb 4, 2022 at 4:07 PM Victor.Sinceac <Victor.Sinceac@anonymised.com> wrote:

Hi,

When using this plugin https://github.com/geoserver/geoserver/commits/main/src/community/schemaless-features on a MongoDB collection where values to be used by a GeoServer layer are declared as Object, the plugin builds a series of features where the feature type is based on the type of the 1st feature.

When layer expects values of type Double and the 1st feature found has type Double, everything goes fine: other values in collection with type Integer are casted to Double (e.g. value “29” is retrieved in the layer as “29.0”).

But, when 1st feature found has type Integer, everything goes bad: if other values have type Double in collection, they are casted to Integer (e.g. value “29.45” is retrieved in the layer as “29”).

This is a blocking issue for a layer based on such a collection…

Any idea how to fix this?

A simple workaround would be to systematically add something like “0.000000001” to every value in each document (this would force the object to have type Double). But this is decidedly ugly, cannot call it a solution…

Many thanks, Victor

Victor.Sinceac

|

Software Developer

Victor.Sinceac@anonymised.com

Tel: +33 1 45 73 60 60

Corobor Systems

|

41 Rue Périer

,

92120 MONTROUGE

,

France

Corobor Systems on LinkedIn

@corobor_systems


This message is subject to and does not create or vary any contractual relationship between Campbell Scientific Limited, its subsidiaries or affiliates (“CSL”) and you. Internet communications are not secure and therefore CSL does not accept legal responsibility for the contents of this message. The message is intended for the addressee only and its contents and any attached files are strictly confidential. If you have received it in error, please telephone +44 (0)1509 601141 or email postmaster@anonymised.com.7336…. Thank you.
Campbell Scientific Ltd.
Company Registration Number: 1933935 (England and Wales) Registered Address: Campbell Park, 80 Hathern Road, Shepshed, Loughborough, Leicestershire LE12 9GX, UK
Tel: +44 (0) 1509 601141
Fax: +44 (0) 1509 601091
Web: www.campbellsci.co.uk



Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Hi,

If we replace this line of code by:

.binding(value instanceof Number ? Number.class : value.getClass())

It works.

Sylvain.Ladoire

Front End Software Developer

Sylvain.Ladoire@anonymised.com

Tel: +33 1 45 73 60 60

Corobor Systems

41 Rue Périer

(attachments)

image516080.png
image578843.png
image959558.png
image578843.png
image959558.png

···

De : Marco Volpini marco.volpini@anonymised.com
Envoyé : mercredi 9 février 2022 15:18
À : Victor.Sinceac Victor.Sinceac@anonymised.com
Cc : geoserver-devel@lists.sourceforge.net geoserver-devel@lists.sourceforge.net; Sylvain.Ladoire Sylvain.Ladoire@anonymised.com
Objet : Re: [Geoserver-devel] About Community MongoDB plugin “schemaless-features”

Hello Victor,
Currently there is no way to fix this without a code change. I believe the problematic line of code is this one where the value type is forced to be the same of the AttributeType (that indeed is build on the first feature retrieved).

Regards,

Marco Volpini

==
GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Marco Volpini

Software Engineer

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it


Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

On Fri, Feb 4, 2022 at 4:07 PM Victor.Sinceac <Victor.Sinceac@anonymised.com> wrote:

Hi,

When using this plugin https://github.com/geoserver/geoserver/commits/main/src/community/schemaless-features on a MongoDB collection where values to be used by a GeoServer layer are declared as Object, the plugin builds a series of features where the feature type is based on the type of the 1st feature.

When layer expects values of type Double and the 1st feature found has type Double, everything goes fine: other values in collection with type Integer are casted to Double (e.g. value “29” is retrieved in the layer as “29.0”).

But, when 1st feature found has type Integer, everything goes bad: if other values have type Double in collection, they are casted to Integer (e.g. value “29.45” is retrieved in the layer as “29”).

This is a blocking issue for a layer based on such a collection…

Any idea how to fix this?

A simple workaround would be to systematically add something like “0.000000001” to every value in each document (this would force the object to have type Double). But this is decidedly ugly, cannot call it a solution…

Many thanks, Victor

Victor.Sinceac

|

Software Developer

Victor.Sinceac@anonymised.com

Tel: +33 1 45 73 60 60

Corobor Systems

|

41 Rue Périer

,

92120 MONTROUGE

,

France

Corobor Systems on LinkedIn

@corobor_systems

This message is subject to and does not create or vary any contractual relationship between Campbell Scientific Limited, its subsidiaries or affiliates ("CSL") and you. Internet communications are not secure and therefore CSL does not accept legal responsibility for the contents of this message. The message is intended for the addressee only and its contents and any attached files are strictly confidential. If you have received it in error, please telephone +44 (0)1509 601141 or email [postmaster@anonymised.com36...](mailto:postmaster@anonymised.com). Thank you. Campbell Scientific Ltd. Company Registration Number: 1933935 (England and Wales) Registered Address: Campbell Park, 80 Hathern Road, Shepshed, Loughborough, Leicestershire LE12 9GX, UK Tel: +44 (0) 1509 601141 Fax: +44 (0) 1509 601091 Web: [www.campbellsci.co.uk](http://www.campbellsci.co.uk)

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel