Dear all,
just for your information:
I will commit several modification to the temporal framework (grass7
trunk) in the next weeks, that will change the SQL database layout and
the API. The goal is to make the temporal frameworks SQL database
interaction much faster, reducing the number of tables to scale better
with tens of thousand of maps, improving the PostgreSQL
compatibility/performance and make it better maintainable.
The changes will result in backward incompatibility. Hence, the new
framework will not work with existing temporal databases. The changes
are too drastic to provide backward compatibility. I will introduce
new tables and columns and completely remove the creation of tables
for each registered map. I will remove several functions from the API
and introduce new classes and functions. To get an idea of the changes
have a look at [2].
Please use the export tools (t.rast.export and t.vect.export) to
transfer your temporal data into the new temporal database, or simply
use t.*.list[1] tools to create text files that can be used for
re-registering of raster, vector and voxel maps in the new temporal
framework.
I will inform you before the first commit that will introduce the
incompatibility.
Best regards
Soeren
[1] "t.rast.list input=my_strds columns=name,start_time,end_time > map_list.txt"
[2] http://pastebin.de/38117
On 22 December 2013 00:07, Sören Gebbert <soerengebbert@googlemail.com> wrote:
Dear all,
Hi Soeren,
just for your information:
I will commit several modification to the temporal framework (grass7
trunk) in the next weeks, that will change the SQL database layout and
the API. The goal is to make the temporal frameworks SQL database
interaction much faster, reducing the number of tables to scale better
with tens of thousand of maps, improving the PostgreSQL
compatibility/performance and make it better maintainable.
these changes could be useful for create temporal dataset using maps
outside current mapset?
this change for my point of view is really important, if you break the
API could I suggest you add also this "feature" to temporal framework?
Thanks a lot
Best regards
Soeren
--
ciao
Luca
http://gis.cri.fmach.it/delucchi/
www.lucadelu.org
Hi Luca,
i will try to support your suggestion. I will introduce two GRASS
variables to be set via g.gisenv.
Use:
g.gisenv set="TGIS_DISABLE_MAPSET_CHECK=True"
g.gisenv set="TGIS_DISABLE_TIMESTAMP_WRITE=True"
to disable the mapset check and the writing of the timestamps of each
map to the map metadata
in the spatial database as text files. You can set these variables
mapset specific.
Settings these variables "True" should (hopefully, because untested)
allow the registration of maps outside the current mapset, even if you
do not have the permission to modify the maps.
A warning will be printed if these variables are set True.
BUT, be aware that this feature can lead to the corruption of the
temporal database and
unwanted side effects. You can mess up the temporal database if you
are not 100% sure what you are doing.
It is no longer possible to access the timestamp information of these
maps using the C-libraries, because
the timestamp information is not available in the map metadata text files.
Best regards
Soeren
2013/12/28 Luca Delucchi <lucadeluge@gmail.com>:
On 22 December 2013 00:07, Sören Gebbert <soerengebbert@googlemail.com> wrote:
Dear all,
Hi Soeren,
just for your information:
I will commit several modification to the temporal framework (grass7
trunk) in the next weeks, that will change the SQL database layout and
the API. The goal is to make the temporal frameworks SQL database
interaction much faster, reducing the number of tables to scale better
with tens of thousand of maps, improving the PostgreSQL
compatibility/performance and make it better maintainable.
these changes could be useful for create temporal dataset using maps
outside current mapset?
this change for my point of view is really important, if you break the
API could I suggest you add also this "feature" to temporal framework?
Thanks a lot
Best regards
Soeren
--
ciao
Luca
http://gis.cri.fmach.it/delucchi/
www.lucadelu.org
On 28 December 2013 03:33, Sören Gebbert <soerengebbert@googlemail.com> wrote:
Hi Luca,
Hi Soeren
i will try to support your suggestion. I will introduce two GRASS
variables to be set via g.gisenv.
Use:
g.gisenv set="TGIS_DISABLE_MAPSET_CHECK=True"
g.gisenv set="TGIS_DISABLE_TIMESTAMP_WRITE=True"
to disable the mapset check and the writing of the timestamps of each
map to the map metadata
in the spatial database as text files. You can set these variables
mapset specific.
Settings these variables "True" should (hopefully, because untested)
allow the registration of maps outside the current mapset, even if you
do not have the permission to modify the maps.
A warning will be printed if these variables are set True.
this seams really cool, I'll test it when it will be on svn
BUT, be aware that this feature can lead to the corruption of the
temporal database and
unwanted side effects. You can mess up the temporal database if you
are not 100% sure what you are doing.
can you specify a little bit more what can be the problems?
It is no longer possible to access the timestamp information of these
maps using the C-libraries, because
the timestamp information is not available in the map metadata text files.
sure 
Best regards
Soeren
Thanks a lot
--
ciao
Luca
http://gis.cri.fmach.it/delucchi/
www.lucadelu.org
Dear all,
just for your information:
I will commit several modification to the temporal framework (grass7
trunk) in a few minutes. 
The modifications are not backward compatible, hence existing temporal
databases, independently from the used backend (sqlite or postgresql)
WILL NOT WORK with the new framework.
The modifications:
- Removed the creation of a register table for each registered map,
which speeds things up massively
- Added modification time to space time datasets
- Added aggregation type to space time datasets that will be set by
t.rast.aggregate
- Keeping the SQL scripts as simple as possible
- Removed foreign primary keys and created indexes for primary keys in
sqlite3 (postgreql does this automagically)
- Using delete trigger instead of foreign primary keys in postgresql backend
- Removed unneeded SQL scripts
- Using Python dateutil module for time string parsing if installed,
or a much more simpler parser without time zone support
- Added GRASS environmental variables to allow registration of maps
from different mapsets in a space time dataset
- Simplified code for map registration
- Code cleanup ...
Best regards
Soeren
2013/12/22 Sören Gebbert <soerengebbert@googlemail.com>:
Dear all,
just for your information:
I will commit several modification to the temporal framework (grass7
trunk) in the next weeks, that will change the SQL database layout and
the API. The goal is to make the temporal frameworks SQL database
interaction much faster, reducing the number of tables to scale better
with tens of thousand of maps, improving the PostgreSQL
compatibility/performance and make it better maintainable.
The changes will result in backward incompatibility. Hence, the new
framework will not work with existing temporal databases. The changes
are too drastic to provide backward compatibility. I will introduce
new tables and columns and completely remove the creation of tables
for each registered map. I will remove several functions from the API
and introduce new classes and functions. To get an idea of the changes
have a look at [2].
Please use the export tools (t.rast.export and t.vect.export) to
transfer your temporal data into the new temporal database, or simply
use t.*.list[1] tools to create text files that can be used for
re-registering of raster, vector and voxel maps in the new temporal
framework.
I will inform you before the first commit that will introduce the
incompatibility.
Best regards
Soeren
[1] "t.rast.list input=my_strds columns=name,start_time,end_time > map_list.txt"
[2] http://pastebin.de/38117
Hi Luca,
2013/12/30 Luca Delucchi <lucadeluge@gmail.com>:
On 28 December 2013 03:33, Sören Gebbert <soerengebbert@googlemail.com> wrote:
Hi Luca,
Hi Soeren
i will try to support your suggestion. I will introduce two GRASS
variables to be set via g.gisenv.
Use:
g.gisenv set="TGIS_DISABLE_MAPSET_CHECK=True"
g.gisenv set="TGIS_DISABLE_TIMESTAMP_WRITE=True"
to disable the mapset check and the writing of the timestamps of each
map to the map metadata
in the spatial database as text files. You can set these variables
mapset specific.
Settings these variables "True" should (hopefully, because untested)
allow the registration of maps outside the current mapset, even if you
do not have the permission to modify the maps.
A warning will be printed if these variables are set True.
this seams really cool, I'll test it when it will be on svn
BUT, be aware that this feature can lead to the corruption of the
temporal database and
unwanted side effects. You can mess up the temporal database if you
are not 100% sure what you are doing.
can you specify a little bit more what can be the problems?
There are no tests yet that assure the correct function of this
feature and honestly i don't know what the side effects will be for
any circumstances, since the principle design is that space time
datasets are mapset specific, which is well tested and supported.
It is no longer possible to access the timestamp information of these
maps using the C-libraries, because
the timestamp information is not available in the map metadata text files.
sure
Well, unless you already set the timestamps using the C-library
functions or *.timestamp modules before.
Best regards
Soeren
Best regards
Soeren
Thanks a lot
--
ciao
Luca
http://gis.cri.fmach.it/delucchi/
www.lucadelu.org
On 31 December 2013 03:37, Sören Gebbert <soerengebbert@googlemail.com> wrote:
Hi Luca,
Hi Soeren
There are no tests yet that assure the correct function of this
feature and honestly i don't know what the side effects will be for
any circumstances, since the principle design is that space time
datasets are mapset specific, which is well tested and supported.
I tested it, and it failed
t.register -i input=tmean_daily_sicilia file=file_raster.txt
start=2012-01-01 increment="1 days"
WARNING: TGIS_DISABLE_MAPSET_CHECK is True
WARNING: TGIS_DISABLE_TIMESTAMP_WRITE is True
Gathering map informations
100%
Register maps in the temporal database
Register maps in the space time raster dataset
ERROR: Only maps from the same mapset can be registered
the input maps inside the text file are written as follow
lst_2012_001_average@modis_lst_reconstructed_europe_daily
Best regards
Soeren
Thanks a lot
--
ciao
Luca
http://gis.cri.fmach.it/delucchi/
www.lucadelu.org