By running t.rast.out.xyz I would expect to have a file with 146880 rows (number of cells) and columns = the number of maps registered in the raster time series plus x & y coordinates. The latter works, I get the expected number of columns. However, the number of rows is much lower than expected.
GRASS 7.3.svn (latlong_wgs84):~ > t.rast.out.xyz strds=cla_monthly_average | wc -l
100%
Space time raster dataset cla_monthly_average exported to -
12891 << – less than 10% of the region!! No mask applied.
I observe the same behaviour when I vary the region size and also, when I export to a text file. Is this the expected behaviour? Am I missing something?
By running t.rast.out.xyz I would expect to have a file with 146880 rows
(number of cells) and columns = the number of maps registered in the raster
time series plus x & y coordinates. The latter works, I get the expected
number of columns. However, the number of rows is much lower than expected.
GRASS 7.3.svn (latlong_wgs84):~ > t.rast.out.xyz strds=cla_monthly_average |
wc -l
100%
Space time raster dataset cla_monthly_average exported to -
12891 << -- less than 10% of the region!! No mask applied.
I observe the same behaviour when I vary the region size and also, when I
export to a text file. Is this the expected behaviour? Am I missing
something?
Maybe yes... do you have null cells (read r.out.xyz manual )?
So, IIUC, as t.rast.out.xyz is a wraper for r.out.xyz it won’t export cells with NULL values, and from what I learnt just by trial and error, if the time series happens to start with a NULL cell then one gets an empty output. Hence, it’s only useful for time series without gaps…
Would that be too complicated to add support to also export NULL cells with some symbol such as “*” as other modules use??? Is this worth an enhancement ticket?
Anyway, this should also be mentioned in t.rast.out.xyz manual. I will send a diff later today.
By running t.rast.out.xyz I would expect to have a file with 146880 rows
(number of cells) and columns = the number of maps registered in the raster
time series plus x & y coordinates. The latter works, I get the expected
number of columns. However, the number of rows is much lower than expected.
GRASS 7.3.svn (latlong_wgs84):~ > t.rast.out.xyz strds=cla_monthly_average |
wc -l
100%
Space time raster dataset cla_monthly_average exported to -
12891 << – less than 10% of the region!! No mask applied.
I observe the same behaviour when I vary the region size and also, when I
export to a text file. Is this the expected behaviour? Am I missing
something?
Maybe yes… do you have null cells (read r.out.xyz manual )?
On 29 November 2016 at 15:28, Veronica Andreo <veroandreo@gmail.com> wrote:
Ciao Lu,
Ciao,
Yes, of course I have NULL cells... cuac!
So, IIUC, as t.rast.out.xyz is a wraper for r.out.xyz it won't export cells
with NULL values, and from what I learnt just by trial and error, if the
time series happens to start with a NULL cell then one gets an empty output.
Hence, it's only useful for time series without gaps...
yes, most of the temporal module are wrapper for existing modules
Would that be too complicated to add support to also export NULL cells with
some symbol such as "*" as other modules use??? Is this worth an enhancement
ticket?
it seems no, r.out.xyz is a python script using r.stats, r.out.xyz
force to return only no NULL values, but this is a flag in r.stats.
I propose to add a flag to r.out.xyz to permit to export also null
values, what do other devs think?
Anyway, this should also be mentioned in t.rast.out.xyz manual. I will send
a diff later today.
AFAIU, to activate the NULL export in r.out.xyz is then a trivial change, but we do not know which letter to use as flag because -n is already used to indicate the opposite behaviour (?). Is it too difficult to change default behaviour to export NULL values and keep then, the -n flag as in r.stats to not export/report NULL values?
I’m asking from ignorance, I would not know how to implement that myself.
Sorry for bothersome… and thanks much!!
On Wed, Nov 30, 2016 at 2:10 PM, Veronica Andreo <veroandreo@gmail.com> wrote:
Hello devs,
AFAIU, to activate the NULL export in r.out.xyz is then a trivial change,
but we do not know which letter to use as flag because -n is already used to
indicate the opposite behaviour (?). Is it too difficult to change default
behaviour to export NULL values and keep then, the -n flag as in r.stats to
not export/report NULL values?
While this would make sense, it is a kind of API change for r.out.xyz.
The question is: where is it already used, and, would a "no data"
floodin in the output be an issue.
Honestly I was surprised to see that no data is excluded by default,
quite the opposite behaviour as we usually have.
On 30 November 2016 at 14:26, Markus Neteler <neteler@osgeo.org> wrote:
On Wed, Nov 30, 2016 at 2:10 PM, Veronica Andreo <veroandreo@gmail.com> wrote:
Hello devs,
AFAIU, to activate the NULL export in r.out.xyz is then a trivial change,
but we do not know which letter to use as flag because -n is already used to
indicate the opposite behaviour (?). Is it too difficult to change default
behaviour to export NULL values and keep then, the -n flag as in r.stats to
not export/report NULL values?
While this would make sense, it is a kind of API change for r.out.xyz.
yes it is, but we could release this change in 7.2
On Wed, Nov 30, 2016 at 2:10 PM, Veronica Andreo <veroandreo@gmail.com> wrote:
Hello devs,
AFAIU, to activate the NULL export in r.out.xyz is then a trivial change,
but we do not know which letter to use as flag because -n is already used to
indicate the opposite behaviour (?). Is it too difficult to change default
behaviour to export NULL values and keep then, the -n flag as in r.stats to
not export/report NULL values?
attached a trivial (untested) patch make make r.out.xyz behave as the rest.
Please check:
I applied it in my grass trunk and just tested for a map (146880 cells) that contains no data (45757 cells). I get all the pixels with data (101123), but nothing for the pixels with no data. It seems to be the same as before, no? (It is indeed, the same. I tested with the same map in grass72 with no patch)… and if I set the flag -n, I get:
Was that the expected result? or maybe i’m not using it properly… IIUC, I would expect to get the nulls by default and do not get them if I set the -n flag, is that right?
t.rast.out.xyz for the time series containing that map, gives an empty file… as before.
AFAIU, to activate the NULL export in r.out.xyz is then a trivial change,
but we do not know which letter to use as flag because -n is already used to
indicate the opposite behaviour (?). Is it too difficult to change default
behaviour to export NULL values and keep then, the -n flag as in r.stats to
not export/report NULL values?
attached a trivial (untested) patch make make r.out.xyz behave as the rest.
Please check:
I applied it in my grass trunk and just tested for a map (146880 cells)
that contains no data (45757 cells). I get all the pixels with data
(101123), but nothing for the pixels with no data. It seems to be the
same as before, no? (It is indeed, the same. I tested with the same map
in grass72 with no patch)... and if I set the flag -n, I get:
ERROR: r.out.xyz <http://r.out.xyz>: Sorry, <n> is not a valid flag
Was that the expected result? or maybe i'm not using it properly...
This sounds like the patch did not apply correctly. It contains the definition of the -n flag, so you should at least see that when you run r.out.xyz --h.
I applied it in my grass trunk and just tested for a map (146880 cells)
that contains no data (45757 cells). I get all the pixels with data
(101123), but nothing for the pixels with no data. It seems to be the
same as before, no? (It is indeed, the same. I tested with the same map
in grass72 with no patch)… and if I set the flag -n, I get:
Was that the expected result? or maybe i’m not using it properly…
This sounds like the patch did not apply correctly. It contains the definition of the -n flag, so you should at least see that when you run r.out.xyz --h.
On Tue, Dec 6, 2016 at 12:34 PM, Moritz Lennert <
mlennert@club.worldonline.be> wrote:
Le 6 décembre 2016 17:53:48 GMT+01:00, Martin Landa <
landa.martin@gmail.com> a écrit :
>Hi,
>
>2016-12-06 17:45 GMT+01:00 Markus Neteler <neteler@osgeo.org>:
>> Big question: add to 7.2.0 or not? I am in favor.
>
>+1 Ma
I understand the motivation, but IIUC it's definitely an API change that
could break existing scripts, which is against our normal policy.
Why not make the flag mean the contrary, i.e. include null values ? Then
it wouldn't break anything...
I'm pretty sure that most people exporting data via r.out.xyz would
expect null values to be excluded...
r.out.xyz export XY and their values. When there is no value, nothing is
exported. This is the correct counterpart for r.in.xyz where you get NULLs
for cells without any input XY(Z).
I agree with Moritz. Adding a flag to export NULLs looks like the best way.
It doesn't break API nor expectations. NULLs simply not being there seems
like a better default here.
On Tue, Dec 6, 2016 at 7:34 PM, Vaclav Petras <wenzeslaus@gmail.com> wrote:
On Tue, Dec 6, 2016 at 12:34 PM, Moritz Lennert
...
Why not make the flag mean the contrary, i.e. include null values ? Then
it wouldn't break anything...
I'm pretty sure that most people exporting data via r.out.xyz would expect
null values to be excluded...
r.out.xyz export XY and their values. When there is no value, nothing is
exported. This is the correct counterpart for r.in.xyz where you get NULLs
for cells without any input XY(Z).
Ah I see, they kind of belong together.
I agree with Moritz. Adding a flag to export NULLs looks like the best way.
It doesn't break API nor expectations. NULLs simply not being there seems
like a better default here.
Any wording suggestion for such a flag?
And the same flag would go into t.rast.out.xyz then?
On Tue, Dec 6, 2016 at 9:15 PM, Markus Neteler <neteler@osgeo.org> wrote:
On Tue, Dec 6, 2016 at 7:34 PM, Vaclav Petras <wenzeslaus@gmail.com> wrote:
On Tue, Dec 6, 2016 at 12:34 PM, Moritz Lennert
...
Why not make the flag mean the contrary, i.e. include null values ? Then
it wouldn't break anything...
ok
I'm pretty sure that most people exporting data via r.out.xyz would expect
null values to be excluded...
r.out.xyz export XY and their values. When there is no value, nothing is
exported. This is the correct counterpart for r.in.xyz where you get NULLs
for cells without any input XY(Z).
Ah I see, they kind of belong together.
I agree with Moritz. Adding a flag to export NULLs looks like the best way.
It doesn't break API nor expectations. NULLs simply not being there seems
like a better default here.
ok.
I have submitted a new -i flag to include no data values (r70017).
This can be used for t.rast.out.xyz and backported for 7.2.1.
I have submitted a new -i flag to include no data values (r70017).
This can be used for t.rast.out.xyz and backported for 7.2.1.
IMHO, if we wait to backport this new flag in r.out.xyz until 7.2.1, then t.rast.out.xyz could only be modified after that and this might happen in ~3-4 months from now according to a previous email from Martin. What I mean is that the original problem that was regarding exporting time series containing NULL values with t.rast.out.xyz, is then not solved, but postponed some months. Otherwise, we would have an add-on (t.rast.out.xyz) with features that only works along with trunk and that is not desirable, right?
Sorry if I create more noise here, honestly, I am not aware of the drawbacks of backporting right away
best,
Vero
recorded at [1]. Does it mean that we can go ahead and release 7.2.0? Ma
)?
Anyway, I just did and tested again… works as expected!!! Cool!! :))