[Geoserver-users] Problems with FormatDateTimezone in app-schema mapping

Hello all,

for some reasons I NEED to fiddle around with the formatting of a time stamp in an app-schema mapping.

Everything works like a charm for the cases where the field in the database is populated.

But when it comes to a NULL-Value I get the error message “formatDateTimezone: could not parse date: null”.

With the background knowledge of what happens inside (delegation to java’s SimpleDateFormat) I am pretty sure I know what goes wrong here.

Is there a workaround available for this? Or a better way to format the date?

We already tried formatting it directly in the database (oracle, TO_CHAR-Function) but where not completely happy with that either.

Best regards

G. Dünnebeil

GERHARD DÜNNEBEIL
Secure Information Management
Safety & Security Department

AIT Austrian Institute of Technology GmbH
Donau-City-Straße 1 | 1220 Vienna | Austria
T +43(0) 50550-3173 | M +43(0) 664 2351747 | F +43(0) 50550-4150
gerhard.duennebeil@anonymised.com | http://www.ait.ac.at

FN: 115980 i HG Wien | UID: ATU14703506
http://www.ait.ac.at/Email-Disclaimer

Can you turn verbose logging on and let us see the stack trace that causes that error message to be thrown. Assuming you’re using the 2.8-RC there where some changes to the Date Formatting code that could have introduced a bug but I can’t be sure without seeing the stacktrace.

Ian

···

On 23 September 2015 at 14:14, Dünnebeil Gerhard <Gerhard.Duennebeil@anonymised.com> wrote:

Hello all,

for some reasons I NEED to fiddle around with the formatting of a time stamp in an app-schema mapping.

Everything works like a charm for the cases where the field in the database is populated.

But when it comes to a NULL-Value I get the error message “formatDateTimezone: could not parse date: null”.

With the background knowledge of what happens inside (delegation to java’s SimpleDateFormat) I am pretty sure I know what goes wrong here.

Is there a workaround available for this? Or a better way to format the date?

We already tried formatting it directly in the database (oracle, TO_CHAR-Function) but where not completely happy with that either.

Best regards

G. Dünnebeil

GERHARD DÜNNEBEIL
Secure Information Management
Safety & Security Department

AIT Austrian Institute of Technology GmbH
Donau-City-Straße 1 | 1220 Vienna | Austria
T +43(0) 50550-3173 | M +43(0) 664 2351747 | F +43(0) 50550-4150
gerhard.duennebeil@anonymised.com | http://www.ait.ac.at

FN: 115980 i HG Wien | UID: ATU14703506
http://www.ait.ac.at/Email-Disclaimer


Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
Get real-time metrics from all of your servers, apps and tools
in one place.
SourceForge users - Click here to start your Free Trial of Datadog now!
http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140


Geoserver-users mailing list
Geoserver-users@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Ian Turton

Hi Gerhard,
could you try with a conditional mapping, something along these lines:

if_then_else(isNull(DATETIME_FIELD), Expression.NIL, FormatDateTimezone(‘yyyy-MM-dd’‘T’‘HH:mm:ss’‘Z’‘’, DATETIME_FIELD, ‘UTC’))

Hope this helps.

···

On Wed, Sep 23, 2015 at 3:14 PM, Dünnebeil Gerhard <Gerhard.Duennebeil@anonymised.com> wrote:

Hello all,

for some reasons I NEED to fiddle around with the formatting of a time stamp in an app-schema mapping.

Everything works like a charm for the cases where the field in the database is populated.

But when it comes to a NULL-Value I get the error message “formatDateTimezone: could not parse date: null”.

With the background knowledge of what happens inside (delegation to java’s SimpleDateFormat) I am pretty sure I know what goes wrong here.

Is there a workaround available for this? Or a better way to format the date?

We already tried formatting it directly in the database (oracle, TO_CHAR-Function) but where not completely happy with that either.

Best regards

G. Dünnebeil

GERHARD DÜNNEBEIL
Secure Information Management
Safety & Security Department

AIT Austrian Institute of Technology GmbH
Donau-City-Straße 1 | 1220 Vienna | Austria
T +43(0) 50550-3173 | M +43(0) 664 2351747 | F +43(0) 50550-4150
gerhard.duennebeil@anonymised.com | http://www.ait.ac.at

FN: 115980 i HG Wien | UID: ATU14703506
http://www.ait.ac.at/Email-Disclaimer


Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
Get real-time metrics from all of your servers, apps and tools
in one place.
SourceForge users - Click here to start your Free Trial of Datadog now!
http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140


Geoserver-users mailing list
Geoserver-users@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Best regards,
Stefano Costa

==
GeoServer Professional Services from the experts! Visit
[http://goo.gl/it488V](http://goo.gl/it488V) for more information.
==
Dott. Stefano Costa
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:     +39 0584 1660272

[http://www.geo-solutions.it](http://www.geo-solutions.it)
[http://twitter.com/geosolutions_it](http://twitter.com/geosolutions_it)

-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate.
Il loro utilizzo è consentito esclusivamente al destinatario del
messaggio, per le finalità indicate nel messaggio stesso. Qualora
riceviate questo messaggio senza esserne il destinatario, Vi preghiamo
cortesemente di darcene notizia via e-mail e di procedere alla
distruzione del messaggio stesso, cancellandolo dal Vostro sistema.
Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità
diverse, costituisce comportamento contrario ai principi dettati dal
D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely
for the attention and use of the named addressee(s) and may be
confidential or proprietary in nature or covered by the provisions of
privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New
Data Protection Code).Any use not in accord with its purpose, any
disclosure, reproduction, copying, distribution, or either
dissemination, either whole or partial, is strictly forbidden except
previous formal approval of the named addressee(s). If you are not the
intended recipient, please contact immediately the sender by
telephone, fax or e-mail and delete the information in this message
that has been received in error. The sender does not give any warranty
or accept liability as the content, accuracy or completeness of sent
messages and accepts no responsibility  for changes made after they
were sent or for other risks which arise as a result of e-mail
transmission, viruses, etc.

On Wed, Sep 23, 2015 at 3:14 PM, Dünnebeil Gerhard <
Gerhard.Duennebeil@anonymised.com> wrote:

Hello all,

for some reasons I NEED to fiddle around with the formatting of a time
stamp in an app-schema mapping.

Everything works like a charm for the cases where the field in the
database is populated.

But when it comes to a NULL-Value I get the error message
“formatDateTimezone: could not parse date: null”.

Hum... this message is there on purpose, this is the code:

        Date date = parameters.get(1).evaluate(object, Date.class);
        if (date == null) {
            throw new RuntimeException(getName() + ": could not parse date:
"
                    + (String) parameters.get(1).evaluate(object,
String.class));
        }

Imho this is excessive, it should return a empty string instead...

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

On 24/09/15 03:34, Andrea Aime wrote:

On Wed, Sep 23, 2015 at 3:14 PM, Dünnebeil Gerhard <
Gerhard.Duennebeil@anonymised.com> wrote:

But when it comes to a NULL-Value I get the error message
“formatDateTimezone: could not parse date: null”.

Hum... this message is there on purpose, this is the code:
         Date date = parameters.get(1).evaluate(object, Date.class);
         if (date == null) {
             throw new RuntimeException(getName() + ": could not parse date:
"
                     + (String) parameters.get(1).evaluate(object,
String.class));
         }
Imho this is excessive, it should return a empty string instead...

But this would cause genuine failures to be silenced.

Would it not be better to add an additional top-level guard so that when object is null, null is returned? This would separate error handling from null pass-through, and be consistent with the behaviour of filter functions like centroid.

Is there ever a benefit to using an empty string? The encoder should just ignore elements with null content.

Kind regards,

--
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <http://transient.nz/&gt;
New Zealand

On 24/09/15 08:31, Ben Caradoc-Davies wrote:

Would it not be better to add an additional top-level guard so that when
object is null, null is returned? This would separate error handling
from null pass-through, and be consistent with the behaviour of filter
functions like centroid.

Looking again suggests that the null guard should be on date alone.

--
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <http://transient.nz/&gt;
New Zealand

Gerhard,

I have just pushed a fix to the GeoTools master branch that improves null handling in the FormatDateTimezone function. With this change, FormatDateTimezone returns null if any parameter is null. Rather than returning the exception you reported, the new behaviour should cause the encoder to skip elements for null dates in FormatDateTimezone.

I expect that this change will be included in the next GeoServer master nightly build, in about 9 hours (24 September, 08:30 UTC):
http://ares.boundlessgeo.com/geoserver/master/

When the next build is complete, these links should work:
http://ares.boundlessgeo.com/geoserver/master/geoserver-master-2015-09-24-war.zip
http://ares.boundlessgeo.com/geoserver/master/ext-2015-09-24/geoserver-2.9-SNAPSHOT-app-schema-plugin.zip

Please let us know if the improved null handling behaviour of FormatDateTimezone resolves the problem you encountered.

Kind regards,
Ben.

On 24/09/15 01:14, Dünnebeil Gerhard wrote:

Hello all,

for some reasons I NEED to fiddle around with the formatting of a time stamp in an app-schema mapping.

Everything works like a charm for the cases where the field in the database is populated.
But when it comes to a NULL-Value I get the error message "formatDateTimezone: could not parse date: null".

With the background knowledge of what happens inside (delegation to java's SimpleDateFormat) I am pretty sure I know what goes wrong here.

Is there a workaround available for this? Or a better way to format the date?
We already tried formatting it directly in the database (oracle, TO_CHAR-Function) but where not completely happy with that either.

Best regards
G. Dünnebeil

GERHARD DÜNNEBEIL
Secure Information Management
Safety & Security Department

AIT Austrian Institute of Technology GmbH
Donau-City-Straße 1 | 1220 Vienna | Austria
T +43(0) 50550-3173 | M +43(0) 664 2351747 | F +43(0) 50550-4150
gerhard.duennebeil@anonymised.com<mailto:gerhard.duennebeil@anonymised.com> | http://www.ait.ac.at/&gt;

FN: 115980 i HG Wien | UID: ATU14703506
http://www.ait.ac.at/Email-Disclaimer

------------------------------------------------------------------------------
Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
Get real-time metrics from all of your servers, apps and tools
in one place.
SourceForge users - Click here to start your Free Trial of Datadog now!
http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140

_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <http://transient.nz/&gt;
New Zealand

Hey Ben,
nice improvement! I’m going to cut GeoTools 14.0 (final) Saturday, do you want to backport
this fix before that happens?
Looks sane and simple enough to me.

Cheers
Andrea

···

On Thu, Sep 24, 2015 at 1:51 AM, Ben Caradoc-Davies <ben@anonymised.com> wrote:

Gerhard,

I have just pushed a fix to the GeoTools master branch that improves
null handling in the FormatDateTimezone function. With this change,
FormatDateTimezone returns null if any parameter is null. Rather than
returning the exception you reported, the new behaviour should cause the
encoder to skip elements for null dates in FormatDateTimezone.

I expect that this change will be included in the next GeoServer master
nightly build, in about 9 hours (24 September, 08:30 UTC):
http://ares.boundlessgeo.com/geoserver/master/

When the next build is complete, these links should work:
http://ares.boundlessgeo.com/geoserver/master/geoserver-master-2015-09-24-war.zip
http://ares.boundlessgeo.com/geoserver/master/ext-2015-09-24/geoserver-2.9-SNAPSHOT-app-schema-plugin.zip

Please let us know if the improved null handling behaviour of
FormatDateTimezone resolves the problem you encountered.

Kind regards,
Ben.

On 24/09/15 01:14, Dünnebeil Gerhard wrote:

Hello all,

for some reasons I NEED to fiddle around with the formatting of a time stamp in an app-schema mapping.

Everything works like a charm for the cases where the field in the database is populated.
But when it comes to a NULL-Value I get the error message “formatDateTimezone: could not parse date: null”.

With the background knowledge of what happens inside (delegation to java’s SimpleDateFormat) I am pretty sure I know what goes wrong here.

Is there a workaround available for this? Or a better way to format the date?
We already tried formatting it directly in the database (oracle, TO_CHAR-Function) but where not completely happy with that either.

Best regards
G. Dünnebeil

GERHARD DÜNNEBEIL
Secure Information Management
Safety & Security Department

AIT Austrian Institute of Technology GmbH
Donau-City-Straße 1 | 1220 Vienna | Austria
T +43(0) 50550-3173 | M +43(0) 664 2351747 | F +43(0) 50550-4150
gerhard.duennebeil@anonymised.commailto:[gerhard.duennebeil@anonymised.com](mailto:gerhard.duennebeil@anonymised.com..) | http://www.ait.ac.at<http://www.ait.ac.at/>

FN: 115980 i HG Wien | UID: ATU14703506
http://www.ait.ac.at/Email-Disclaimer


Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
Get real-time metrics from all of your servers, apps and tools
in one place.
SourceForge users - Click here to start your Free Trial of Datadog now!
http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140


Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users


Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <http://transient.nz/>
New Zealand


Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
Get real-time metrics from all of your servers, apps and tools
in one place.
SourceForge users - Click here to start your Free Trial of Datadog now!
http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140


Geoserver-users mailing list
Geoserver-users@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


Done.

Kind regards,
Ben.

On 24/09/15 20:32, Andrea Aime wrote:

Hey Ben,
nice improvement! I'm going to cut GeoTools 14.0 (final) Saturday, do you
want to backport
this fix before that happens?
Looks sane and simple enough to me.

Cheers
Andrea

--
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <http://transient.nz/&gt;
New Zealand

Gerhard,

until you upgrade, Stefano's if_then_else mapping should also work. The fix will also be included in GeoServer 2.8.0.

Kind regards,
Ben.

On 24/09/15 17:23, Dünnebeil Gerhard wrote:

Dear ben,

thank you for the fix.

Without having looked at the code, just from your description, it should show the behavior that I expect.
But I will try it out as soon as possible.

Best regards
Gerhard

--
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <http://transient.nz/&gt;
New Zealand

Ben,

Does Geoserver/App-schema support Oracle data type TimeStamp?

I got an error:
Could not find working property accessor for attribute (START_LIFE) in object

START_LIFE is Timestamp.

If I use if_then_else, I got this:
Filter Function problem for function if_then_else argument #0 - expected type Boolean

If Timestamp is not supported, does that mean I have to use a view to convert it to the right format?

Thanks,
Wen

-----Original Message-----
From: Ben Caradoc-Davies [mailto:ben@…6881…]
Sent: Thursday, September 24, 2015 3:46 PM
To: Dünnebeil Gerhard; geoserver-users@lists.sourceforge.net
Subject: EXT :Re: [Geoserver-users] Problems with FormatDateTimezone in app-schema mapping

Gerhard,

until you upgrade, Stefano's if_then_else mapping should also work. The fix will also be included in GeoServer 2.8.0.

Kind regards,
Ben.

On 24/09/15 17:23, Dünnebeil Gerhard wrote:

Dear ben,

thank you for the fix.

Without having looked at the code, just from your description, it should show the behavior that I expect.
But I will try it out as soon as possible.

Best regards
Gerhard

--
Ben Caradoc-Davies <ben@...6881...>
Director
Transient Software Limited <http://transient.nz/&gt; New Zealand

------------------------------------------------------------------------------
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Wen,

app-schema builds on GeoServer simple feature data stores so only supports Oracle types supported for simple features. I am not familiar with Oracle TIMESTAMP support in GeoServer, but a quick Google suggests that this is a known issue with workarounds:
http://sourceforge.net/p/geoserver/mailman/message/27142636/

I have changed the subject to draw the attention of Oracle users who may be able to advise you.

Kind regards,
Ben.

On 25/09/15 08:23, Yin, Wentao (IS) (Contr) wrote:

Ben,

Does Geoserver/App-schema support Oracle data type TimeStamp?

I got an error:
Could not find working property accessor for attribute (START_LIFE) in object

START_LIFE is Timestamp.

If I use if_then_else, I got this:
Filter Function problem for function if_then_else argument #0 - expected type Boolean

If Timestamp is not supported, does that mean I have to use a view to convert it to the right format?

Thanks,
Wen

--
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <http://transient.nz/&gt;
New Zealand

Thank you, Ben.

Wen

-----Original Message-----
From: Ben Caradoc-Davies [mailto:ben@…6881…]
Sent: Thursday, September 24, 2015 4:50 PM
To: Yin, Wentao (IS) (Contr)
Cc: geoserver-users@lists.sourceforge.net
Subject: EXT :Oracle TIMESTAMP support

Wen,

app-schema builds on GeoServer simple feature data stores so only supports Oracle types supported for simple features. I am not familiar with Oracle TIMESTAMP support in GeoServer, but a quick Google suggests that this is a known issue with workarounds:
http://sourceforge.net/p/geoserver/mailman/message/27142636/

I have changed the subject to draw the attention of Oracle users who may be able to advise you.

Kind regards,
Ben.

On 25/09/15 08:23, Yin, Wentao (IS) (Contr) wrote:

Ben,

Does Geoserver/App-schema support Oracle data type TimeStamp?

I got an error:
Could not find working property accessor for attribute (START_LIFE) in
object

START_LIFE is Timestamp.

If I use if_then_else, I got this:
Filter Function problem for function if_then_else argument #0 -
expected type Boolean

If Timestamp is not supported, does that mean I have to use a view to convert it to the right format?

Thanks,
Wen

--
Ben Caradoc-Davies <ben@...6881...>
Director
Transient Software Limited <http://transient.nz/&gt; New Zealand