[GRASS-dev] Microseconds in timestamps in abstract_space_time_dataset.py

NikosAlexandris · October 21, 2017, 8:08am

Dear Soeren & Co,

the following stumbles in timestamps with microseconds. See:

t.info lst -g

id=lst@lst
..
creator=nik
temporal_type=absolute
creation_time=2017-10-16 09:10:15.689554
modification_time=2017-10-16 09:47:01.646840
semantic_type=mean
start_time=2013-04-13 10:00:09.935625
end_time=2017-04-24 09:57:43.257139
granularity=1 second
..

First item is:

t.rast.list lst -u column=name,start_time |head -1
lst_LC81920282013103LGN01|2013-04-13 10:00:09.935625

The error:

t.rast.aggregate input=lst output=lst_yearly basename=lst_yearly suffix=gran granularity="1 year" method=average --o
0..20..40..60..80..100
0..100
0..Traceback (most recent call last):
  File "/osgeo/grass72/dist.x86_64-pc-linux-gnu/scripts/t.rast.aggregate", line 213, in <module>
    main()
  File "/osgeo/grass72/dist.x86_64-pc-linux-gnu/scripts/t.rast.aggregate", line 203, in main
    sp.get_relative_time_unit(),  dbif)
  File "/osgeo/grass72/dist.x86_64-pc-linux-gnu/etc/python/grass/temporal/register.py", line 499, in register_map_object_list
    file=filename, dbif=dbif)
  File "/osgeo/grass72/dist.x86_64-pc-linux-gnu/etc/python/grass/temporal/register.py", line 341, in register_maps_in_space_time_dataset
    sp.update_from_registered_maps(dbif)
  File "/osgeo/grass72/dist.x86_64-pc-linux-gnu/etc/python/grass/temporal/abstract_space_time_dataset.py", line 2376, in update_from_registered_maps
    tstring, time_format)
  File "/usr/lib64/python2.7/_strptime.py", line 335, in _strptime
    data_string[found.end():])
ValueError: unconverted data remains: .935625

Two possible updates. Remove microseconds altogether:

Index: abstract_space_time_dataset.py

--- abstract_space_time_dataset.py (revision 71469)
+++ abstract_space_time_dataset.py (working copy)
@@ -2368,6 +2368,8 @@
                     # Convert the unicode string into the datetime format
                     if self.is_time_absolute():
                         if tstring.find(":") > 0:
+ if tstring.find('.') > 0:
+ tstring = tstring.split('.')[0]
                             time_format = "%Y-%m-%d %H:%M:%S"
                         else:
                             time_format = "%Y-%m-%d"

Or, retain them:

Index: abstract_space_time_dataset.py

--- abstract_space_time_dataset.py (revision 71469)
+++ abstract_space_time_dataset.py (working copy)
@@ -2367,7 +2367,9 @@
                     tstring = row[0]
                     # Convert the unicode string into the datetime format
                     if self.is_time_absolute():
- if tstring.find(":") > 0:
+ if tstring.find(":") and tstring.find(".") > 0:
+ time_format = "%Y-%m-%d %H:%M:%S.%f"
+ elif tstring.find(":") > 0:
                             time_format = "%Y-%m-%d %H:%M:%S"
                         else:
                             time_format = "%Y-%m-%d"

As ignorant as I am, I'd prefer the second. Would this cause problems to
t.* operations?

Thanks, Nikos

SoerenGebbert · October 21, 2017, 3:04pm

Please try r71573 from grass_trunk, test your case and report any problems.

Best regards
Sören

2017-10-21 10:08 GMT+02:00 Nikos Alexandris <nik@nikosalexandris.net>:

Dear Soeren & Co,

the following stumbles in timestamps with microseconds. See:

t.info lst -g

id=lst@lst
..
creator=nik
temporal_type=absolute
creation_time=2017-10-16 09:10:15.689554
modification_time=2017-10-16 09:47:01.646840
semantic_type=mean
start_time=2013-04-13 10:00:09.935625
end_time=2017-04-24 09:57:43.257139
granularity=1 second
..

First item is:

t.rast.list lst -u column=name,start_time |head -1
lst_LC81920282013103LGN01|2013-04-13 10:00:09.935625

The error:
t.rast.aggregate input=lst output=lst_yearly basename=lst_yearly suffix=gran
granularity="1 year" method=average --o
0..20..40..60..80..100
0..100
0..Traceback (most recent call last):
 File "/osgeo/grass72/dist.x86_64-pc-linux-gnu/scripts/t.rast.aggregate",
line 213, in <module>
   main()
 File "/osgeo/grass72/dist.x86_64-pc-linux-gnu/scripts/t.rast.aggregate",
line 203, in main
   sp.get_relative_time_unit(),  dbif)
 File
"/osgeo/grass72/dist.x86_64-pc-linux-gnu/etc/python/grass/temporal/register.py",
line 499, in register_map_object_list
   file=filename, dbif=dbif)
 File
"/osgeo/grass72/dist.x86_64-pc-linux-gnu/etc/python/grass/temporal/register.py",
line 341, in register_maps_in_space_time_dataset
   sp.update_from_registered_maps(dbif)
 File
"/osgeo/grass72/dist.x86_64-pc-linux-gnu/etc/python/grass/temporal/abstract_space_time_dataset.py",
line 2376, in update_from_registered_maps
   tstring, time_format)
 File "/usr/lib64/python2.7/_strptime.py", line 335, in _strptime
   data_string[found.end():])
ValueError: unconverted data remains: .935625
Two possible updates. Remove microseconds altogether:

Index: abstract_space_time_dataset.py

--- abstract_space_time_dataset.py (revision 71469)
+++ abstract_space_time_dataset.py (working copy)
@@ -2368,6 +2368,8 @@
                    # Convert the unicode string into the datetime format
                    if self.is_time_absolute():
                        if tstring.find(":") > 0:
+ if tstring.find('.') > 0:
+ tstring = tstring.split('.')[0]
                            time_format = "%Y-%m-%d %H:%M:%S"
                        else:
                            time_format = "%Y-%m-%d"

Or, retain them:

Index: abstract_space_time_dataset.py

--- abstract_space_time_dataset.py (revision 71469)
+++ abstract_space_time_dataset.py (working copy)
@@ -2367,7 +2367,9 @@
                    tstring = row[0]
                    # Convert the unicode string into the datetime format
                    if self.is_time_absolute():
- if tstring.find(":") > 0:
+ if tstring.find(":") and tstring.find(".") > 0:
+ time_format = "%Y-%m-%d %H:%M:%S.%f"
+ elif tstring.find(":") > 0:
                            time_format = "%Y-%m-%d %H:%M:%S"
                        else:
                            time_format = "%Y-%m-%d"

As ignorant as I am, I'd prefer the second. Would this cause problems to
t.* operations?

Thanks, Nikos

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

NikosAlexandris · October 21, 2017, 3:22pm

Thanks. Looked at the revision to learn a bit more.
I have no more errors.

Nikos

* Sören Gebbert <soerengebbert@googlemail.com> [2017-10-21 17:04:55 +0200]:

Please try r71573 from grass_trunk, test your case and report any problems.

Best regards
Sören

2017-10-21 10:08 GMT+02:00 Nikos Alexandris <nik@nikosalexandris.net>:
Dear Soeren & Co,

the following stumbles in timestamps with microseconds. See:

t.info lst -g

id=lst@lst
..
creator=nik
temporal_type=absolute
creation_time=2017-10-16 09:10:15.689554
modification_time=2017-10-16 09:47:01.646840
semantic_type=mean
start_time=2013-04-13 10:00:09.935625
end_time=2017-04-24 09:57:43.257139
granularity=1 second
..

First item is:

t.rast.list lst -u column=name,start_time |head -1
lst_LC81920282013103LGN01|2013-04-13 10:00:09.935625

The error:
t.rast.aggregate input=lst output=lst_yearly basename=lst_yearly suffix=gran
granularity="1 year" method=average --o
0..20..40..60..80..100
0..100
0..Traceback (most recent call last):
 File "/osgeo/grass72/dist.x86_64-pc-linux-gnu/scripts/t.rast.aggregate",
line 213, in <module>
   main()
 File "/osgeo/grass72/dist.x86_64-pc-linux-gnu/scripts/t.rast.aggregate",
line 203, in main
   sp.get_relative_time_unit(),  dbif)
 File
"/osgeo/grass72/dist.x86_64-pc-linux-gnu/etc/python/grass/temporal/register.py",
line 499, in register_map_object_list
   file=filename, dbif=dbif)
 File
"/osgeo/grass72/dist.x86_64-pc-linux-gnu/etc/python/grass/temporal/register.py",
line 341, in register_maps_in_space_time_dataset
   sp.update_from_registered_maps(dbif)
 File
"/osgeo/grass72/dist.x86_64-pc-linux-gnu/etc/python/grass/temporal/abstract_space_time_dataset.py",
line 2376, in update_from_registered_maps
   tstring, time_format)
 File "/usr/lib64/python2.7/_strptime.py", line 335, in _strptime
   data_string[found.end():])
ValueError: unconverted data remains: .935625
Two possible updates. Remove microseconds altogether:

Index: abstract_space_time_dataset.py

--- abstract_space_time_dataset.py (revision 71469)
+++ abstract_space_time_dataset.py (working copy)
@@ -2368,6 +2368,8 @@
                    # Convert the unicode string into the datetime format
                    if self.is_time_absolute():
                        if tstring.find(":") > 0:
+ if tstring.find('.') > 0:
+ tstring = tstring.split('.')[0]
                            time_format = "%Y-%m-%d %H:%M:%S"
                        else:
                            time_format = "%Y-%m-%d"

Or, retain them:

Index: abstract_space_time_dataset.py

--- abstract_space_time_dataset.py (revision 71469)
+++ abstract_space_time_dataset.py (working copy)
@@ -2367,7 +2367,9 @@
                    tstring = row[0]
                    # Convert the unicode string into the datetime format
                    if self.is_time_absolute():
- if tstring.find(":") > 0:
+ if tstring.find(":") and tstring.find(".") > 0:
+ time_format = "%Y-%m-%d %H:%M:%S.%f"
+ elif tstring.find(":") > 0:
                            time_format = "%Y-%m-%d %H:%M:%S"
                        else:
                            time_format = "%Y-%m-%d"

As ignorant as I am, I'd prefer the second. Would this cause problems to
t.* operations?

Thanks, Nikos

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

--
Nikos Alexandris | Remote Sensing & Geomatics
GPG Key Fingerprint 6F9D4506F3CA28380974D31A9053534B693C4FB3