[GRASS-user] Temporal data issues need resolving

rshepard · February 23, 2017, 5:03pm

Yesterday I created and registered a STRDS. Today grass cannot find it. I
tried re-creating and re-registering the dataset but the latter step fails.

All mapsets are available regardless of the current mapset:

g.mapsets -p
Accessible mapsets:
precipitation PERMANENT analyses features soils topography

In the precipitation mapset the files are present:

g.list type=raster pattern=rainfall_* separator=comma
rainfall_20131115,rainfall_20131116,rainfall_20131117,rainfall_20131118,rai
nfall_20131120,rainfall_20131121,rainfall_20131122,rainfall_20131123,rainfa
ll_20131124,rainfall_20131125,rainfall_20131126,rainfall_20131127,rainfall_
20131128,rainfall_20131129,rainfall_20131130,rainfall_20131201,rainfall_201
31202,rainfall_20131203,rainfall_20131204,rainfall_20131205,rainfall_201312
06,rainfall_20131207,rainfall_20131208,rainfall_20131209

t.create runs with no errors:

t.create output=rainfall type=strds temporaltype=absolute title='Rainfall'
description='Daily rainfall in mm/hr.' --overwrite
WARNING: Overwriting space time raster dataset <rainfall> and unregistering
all maps

But, t.register fails not finding the first rainfall map:

t.register -i input=rainfall maps=`g.list type=raster pattern="rainfall_*"
separator=comma start="2013-11-15" increment="1 days"
Gathering map information...
ERROR: Unable to update raster map <rainfall_20131115@precipitation>. The
map does not exist.

Nothing in grass73 changed from yesterday afternoon to this morning.

I need to resolve this problem so I can run a model. Your help is needed
to understand what happed and how to fix it.

Regards,

Rich

SoerenGebbert · February 23, 2017, 5:07pm

Are strds and raster maps located in the same mapset? A restriction of the temporal framework is, that you can only register maps in stds from the same mapset.

···

Am 23.02.2017 18:03 schrieb “Rich Shepard” <rshepard@appl-ecosys.com>:

Yesterday I created and registered a STRDS. Today grass cannot find it. I
tried re-creating and re-registering the dataset but the latter step fails.

All mapsets are available regardless of the current mapset:

g.mapsets -p
Accessible mapsets:
precipitation PERMANENT analyses features soils topography

In the precipitation mapset the files are present:

g.list type=raster pattern=rainfall_* separator=comma
rainfall_20131115,rainfall_20131116,rainfall_20131117,rainfall_20131118,rai
nfall_20131120,rainfall_20131121,rainfall_20131122,rainfall_20131123,rainfa
ll_20131124,rainfall_20131125,rainfall_20131126,rainfall_20131127,rainfall_
20131128,rainfall_20131129,rainfall_20131130,rainfall_20131201,rainfall_201
31202,rainfall_20131203,rainfall_20131204,rainfall_20131205,rainfall_201312
06,rainfall_20131207,rainfall_20131208,rainfall_20131209

t.create runs with no errors:

t.create output=rainfall type=strds temporaltype=absolute title=‘Rainfall’
description=‘Daily rainfall in mm/hr.’ --overwrite
WARNING: Overwriting space time raster dataset and unregistering
all maps

But, t.register fails not finding the first rainfall map:

t.register -i input=rainfall maps=`g.list type=raster pattern=“rainfall_*”
separator=comma start=“2013-11-15” increment=“1 days”
Gathering map information…
ERROR: Unable to update raster map rainfall_20131115@precipitation. The
map does not exist.

Nothing in grass73 changed from yesterday afternoon to this morning.

I need to resolve this problem so I can run a model. Your help is needed
to understand what happed and how to fix it.

Regards,

Rich

grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

rshepard · February 23, 2017, 5:34pm

On Thu, 23 Feb 2017, Sören Gebbert wrote:

Are strds and raster maps located in the same mapset? A restriction of the
temporal framework is, that you can only register maps in stds from the
same mapset.

Sören,

They should be:

g.mapset -p
precipitation

g.list type=raster pattern=rainfall_* separator=comma
rainfall_20131115,rainfall_20131116,rainfall_20131117,rainfall_20131118,rai
nfall_20131120,rainfall_20131121,rainfall_20131122,rainfall_20131123,rainfa
ll_20131124,rainfall_20131125,rainfall_20131126,rainfall_20131127,rainfall_
20131128,rainfall_20131129,rainfall_20131130,rainfall_20131201,rainfall_201
31202,rainfall_20131203,rainfall_20131204,rainfall_20131205,rainfall_201312
06,rainfall_20131207,rainfall_20131208,rainfall_20131209

t.list type=strds
----------------------------------------------
Space time raster datasets with absolute time available in mapset
<precipitation>:
rainfall@precipitation
Space time raster datasets with absolute time available in mapset
<analyses>:
rainfall@analyses
Space time raster datasets with absolute time available in mapset
<topography>:
rainfall@topography

Regards,

Rich

SoerenGebbert · February 23, 2017, 6:49pm

This seems to be a bug.
However, in your last email you performed this task in the topology mapset.
The module g.list will show all mas from all mapsets. This command
will only show maps from a specific mapset if you force it to: g.list
type=rast sep="," mapset=precipitation.

Why do you need to have three rainfall strds? Are they differen? If
they are equal then you can reference to a single strds using the
@mapset approach like in raster or vector maps.

2017-02-23 18:34 GMT+01:00 Rich Shepard <rshepard@appl-ecosys.com>:

On Thu, 23 Feb 2017, Sören Gebbert wrote:

Are strds and raster maps located in the same mapset? A restriction of
the
temporal framework is, that you can only register maps in stds from the
same mapset.

Sören,

They should be:

g.mapset -p
precipitation

g.list type=raster pattern=rainfall_* separator=comma
rainfall_20131115,rainfall_20131116,rainfall_20131117,rainfall_20131118,rai
nfall_20131120,rainfall_20131121,rainfall_20131122,rainfall_20131123,rainfa
ll_20131124,rainfall_20131125,rainfall_20131126,rainfall_20131127,rainfall_
20131128,rainfall_20131129,rainfall_20131130,rainfall_20131201,rainfall_201
31202,rainfall_20131203,rainfall_20131204,rainfall_20131205,rainfall_201312
06,rainfall_20131207,rainfall_20131208,rainfall_20131209

t.list type=strds
----------------------------------------------
Space time raster datasets with absolute time available in mapset
<precipitation>:
rainfall@precipitation
Space time raster datasets with absolute time available in mapset
<analyses>:
rainfall@analyses
Space time raster datasets with absolute time available in mapset
<topography>:
rainfall@topography

Regards,

Rich
_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

rshepard · February 23, 2017, 7:13pm

On Thu, 23 Feb 2017, Sören Gebbert wrote:

This seems to be a bug.

Sören,

Just my luck!

However, in your last email you performed this task in the topology
mapset.

That's true.

Why do you need to have three rainfall strds? Are they differen? If
they are equal then you can reference to a single strds using the
@mapset approach like in raster or vector maps.

Because Itzi does not like maps to be in different mapsets.

My maps are in separate directories: data/precipitation/,
/data/topography/, and analyses/. I tried running Itzi from the analyses/
subdirectory and the model told me it could not find the DEM or manning's n
maps or the precipitation STRDS. As I wrote earlier, I had to re-create and
re-register the rainfall STRDS because it disappeared since yesterday
afternoon.

Thanks,

Rich

SoerenGebbert · February 23, 2017, 7:22pm

Can you please send me the grass location so that i can investigate the bug?

Best regards
Soeren

2017-02-23 20:13 GMT+01:00 Rich Shepard <rshepard@appl-ecosys.com>:

On Thu, 23 Feb 2017, Sören Gebbert wrote:

This seems to be a bug.

Sören,

  Just my luck!

However, in your last email you performed this task in the topology
mapset.

  That's true.

Why do you need to have three rainfall strds? Are they differen? If
they are equal then you can reference to a single strds using the
@mapset approach like in raster or vector maps.

  Because Itzi does not like maps to be in different mapsets.

  My maps are in separate directories: data/precipitation/,
/data/topography/, and analyses/. I tried running Itzi from the analyses/
subdirectory and the model told me it could not find the DEM or manning's n
maps or the precipitation STRDS. As I wrote earlier, I had to re-create and
re-register the rainfall STRDS because it disappeared since yesterday
afternoon.

Thanks,

Rich
_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Laurent_C · February 23, 2017, 7:22pm

2017-02-23 13:13 GMT-06:00 Rich Shepard <rshepard@appl-ecosys.com>:

On Thu, 23 Feb 2017, Sören Gebbert wrote:

This seems to be a bug.

Sören,

  Just my luck!

However, in your last email you performed this task in the topology
mapset.

  That's true.

Why do you need to have three rainfall strds? Are they differen? If
they are equal then you can reference to a single strds using the
@mapset approach like in raster or vector maps.

  Because Itzi does not like maps to be in different mapsets.

That is not true. You just have to provide Itzï with the fully
qualified ID in the form mapname@mapset, like every other GRASS
module.
The resulting STDS will be written in a single mapset (the current
one), but once again it is standard GRASS behaviour.

Laurent

My maps are in separate directories: data/precipitation/,
/data/topography/, and analyses/. I tried running Itzi from the analyses/
subdirectory and the model told me it could not find the DEM or manning's n
maps or the precipitation STRDS. As I wrote earlier, I had to re-create and
re-register the rainfall STRDS because it disappeared since yesterday
afternoon.

Thanks,

Rich
_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

rshepard · February 23, 2017, 9:22pm

On Thu, 23 Feb 2017, Laurent C. wrote:

Because Itzi does not like maps to be in different mapsets.

That is not true. You just have to provide Itzï with the fully
qualified ID in the form mapname@mapset, like every other GRASS
module.

Laurent,

The parameter file contains:

[input]
dem = open_culvert@topography
friction = manning@topography
rain = rainfall@precipitation

What's wrong with these? The model does not run with this parameter file:

time itzi run itzi-open-culvert.txt
Starting simulation of itzi-open-culvert.txt... WARNING: rainfall@precipitation: invalid topology WARNING: Error during execution: Traceback (most recent call last):
   File "/usr/lib/python2.7/site-packages/itzi/itzi.py", line 183, in
sim_runner_worker
     sim_runner.run()
   File "/usr/lib/python2.7/site-packages/itzi/itzi.py", line 90, in run
     sim_param=self.conf.sim_param)
   File "/usr/lib/python2.7/site-packages/itzi/simulation.py", line 77, in
__init__
     self.gis.read(self.in_map_names)
   File "/usr/lib/python2.7/site-packages/itzi/gis.py", line 189, in read
     if not self.stds_temporal_sanity(strds_id):
   File "/usr/lib/python2.7/site-packages/itzi/gis.py", line 222, in
stds_temporal_sanity
     if stds_start > sim_start:
TypeError: can't compare datetime.datetime to NoneType

Simulations complete. Elapsed times: itzi-open-culvert.txt: 0:00:00 Total: 0:00:00 Average: 0:00:00

real 0m0.871s
user 0m0.751s
sys 0m0.129s

Rich

SoerenGebbert · February 23, 2017, 9:30pm

This error is related to the temporal topology of the rainfall strds, it seems to lack interval time.

Am 23.02.2017 22:22 schrieb “Rich Shepard” <rshepard@appl-ecosys.com>:

On Thu, 23 Feb 2017, Laurent C. wrote:

Because Itzi does not like maps to be in different mapsets.

That is not true. You just have to provide Itzï with the fully
qualified ID in the form mapname@mapset, like every other GRASS
module.

Laurent,

The parameter file contains:

[input]
dem = open_culvert@topography
friction = manning@topography
rain = rainfall@precipitation

What’s wrong with these? The model does not run with this parameter file:

time itzi run itzi-open-culvert.txt
Starting simulation of itzi-open-culvert.txt… WARNING: rainfall@precipitation: invalid topology WARNING: Error during execution: Traceback (most recent call last):
File “/usr/lib/python2.7/site-packages/itzi/itzi.py”, line 183, in
sim_runner_worker
sim_runner.run()
File “/usr/lib/python2.7/site-packages/itzi/itzi.py”, line 90, in run
sim_param=self.conf.sim_param)
File “/usr/lib/python2.7/site-packages/itzi/simulation.py”, line 77, in
init
self.gis.read(self.in_map_names)
File “/usr/lib/python2.7/site-packages/itzi/gis.py”, line 189, in read
if not self.stds_temporal_sanity(strds_id):
File “/usr/lib/python2.7/site-packages/itzi/gis.py”, line 222, in
stds_temporal_sanity
if stds_start > sim_start:
TypeError: can’t compare datetime.datetime to NoneType

Simulations complete. Elapsed times: itzi-open-culvert.txt: 0:00:00 Total: 0:00:00 Average: 0:00:00

real 0m0.871s
user 0m0.751s
sys 0m0.129s

Rich

grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

rshepard · February 23, 2017, 9:48pm

On Thu, 23 Feb 2017, Sören Gebbert wrote:

This error is related to the temporal topology of the rainfall strds, it
seems to lack interval time.

Soeren,

OK. Had a different error before.

Well, the STRDS has again disappeared. t.info shows no times or other
data.

Again, when I try to re-register the strds t.register cannot find the
first map:

t.register -i input=rainfall maps=a.list type=raster pattern="rainfall_*"
separator=comma start="2013-11-15" increment="1 days"
Gathering map information...
ERROR: Unable to update raster map <rainfall_20131115@precipitation>. The
map does not exist.

That map _does_ exist. Now I'll re-create and re-register the rainfall
maps for the third time. Shouldn't need to keep doing this, should I?

Rich

rshepard · February 23, 2017, 9:53pm

On Thu, 23 Feb 2017, Rich Shepard wrote:

That map _does_ exist. Now I'll re-create and re-register the rainfall
maps for the third time. Shouldn't need to keep doing this, should I?

g.list type=raster pattern='rainfall_*' separator=comma
rainfall_20131115,rainfall_20131116,rainfall_20131117,rainfall_20131118,rai
nfall_20131120,rainfall_20131121,rainfall_20131122,rainfall_20131123,rainfa
ll_20131124,rainfall_20131125,rainfall_20131126,rainfall_20131127,rainfall_
20131128,rainfall_20131129,rainfall_20131130,rainfall_20131201,rainfall_201
31202,rainfall_20131203,rainfall_20131204,rainfall_20131205,rainfall_201312
06,rainfall_20131207,rainfall_20131208,rainfall_20131209

Notice that the first map is rainfall_20131115 so why can't t.register
find it?

t.create output=rainfall type=strds temporaltype=absolute title='Rainfall'
description='Daily rainfall in mm/hr.' --overwrite
WARNING: Overwriting space time raster dataset <rainfall> and unregistering
all maps
GRASS 7.3.svn
(Oregon-Dayton-meters):~/projects/oregon/washington-eichler/analyses >
t.register -i input=rainfall maps=`g.list type=raster pattern="rainfall_*"
separator=comma` start="2013-11-15" increment="1 days"
Gathering map information...
ERROR: Unable to update raster map <rainfall_20131115@precipitation>. The
map does not exist.

Rich

SoerenGebbert · February 23, 2017, 10:03pm

Can you please run the following commands in the mapset that has the
error and send me the result?

r.info rainfall_20131115@precipitation
t.info rainfall

2017-02-23 22:53 GMT+01:00 Rich Shepard <rshepard@appl-ecosys.com>:

On Thu, 23 Feb 2017, Rich Shepard wrote:

That map _does_ exist. Now I'll re-create and re-register the rainfall
maps for the third time. Shouldn't need to keep doing this, should I?

g.list type=raster pattern='rainfall_*' separator=comma
rainfall_20131115,rainfall_20131116,rainfall_20131117,rainfall_20131118,rai
nfall_20131120,rainfall_20131121,rainfall_20131122,rainfall_20131123,rainfa
ll_20131124,rainfall_20131125,rainfall_20131126,rainfall_20131127,rainfall_
20131128,rainfall_20131129,rainfall_20131130,rainfall_20131201,rainfall_201
31202,rainfall_20131203,rainfall_20131204,rainfall_20131205,rainfall_201312
06,rainfall_20131207,rainfall_20131208,rainfall_20131209

Notice that the first map is rainfall_20131115 so why can't t.register
find it?

t.create output=rainfall type=strds temporaltype=absolute title='Rainfall'
description='Daily rainfall in mm/hr.' --overwrite
WARNING: Overwriting space time raster dataset <rainfall> and unregistering
all maps
GRASS 7.3.svn
(Oregon-Dayton-meters):~/projects/oregon/washington-eichler/analyses >
t.register -i input=rainfall maps=`g.list type=raster pattern="rainfall_*"
separator=comma` start="2013-11-15" increment="1 days"
Gathering map information...
ERROR: Unable to update raster map <rainfall_20131115@precipitation>. The
map does not exist.

Rich
_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

rshepard · February 23, 2017, 11:46pm

On Thu, 23 Feb 2017, Sören Gebbert wrote:

Can you please run the following commands in the mapset that has the
error and send me the result?
r.info rainfall_20131115@precipitation
t.info rainfall

Sören,

g.mapset -p
precipitation

r.info rainfall_20131115@precipitation
ERROR: Raster map <rainfall_20131115@precipitation> not found

However:
g.list -m type=raster pattern=rainfall_*
rainfall_20131115@topography
rainfall_20131116@topography
rainfall_20131117@topography
rainfall_20131118@topography
...

So they were all created in topography rather than precipitation. I missed
checking the mapset before creating the STRDS. I'll copy/remove all these
files, then run t.create and t.register again with these files in
precipitation and see if Itzi runs from the analysis mapset.

Thanks for catching this!

Rich

rshepard · February 24, 2017, 12:54am

On Thu, 23 Feb 2017, Rich Shepard wrote:

Yesterday I created and registered a STRDS. Today grass cannot find it. I
tried re-creating and re-registering the dataset but the latter step
fails.

Thanks to both Soeren and Laurent Itzi is now happily running. I'll
probably kill the process and start it tomorrow morning from the command
line external to grass and put it in the background.

I found a rainfall data file typo that resulted in a borked STRDS. It is
highly likely that this typo caused all the problems I encountered. Mea
culpa!

I also cleaned up all mapsets so every map is in only one mapset. The Itzi
parameter file specifies the mapset for each input map.

Again, thanks to Soeren and Laurent,

Rich