[GRASS-dev] v.overlay handles mapname@mapset in a wrong way

Hi,

it seems [1] that v.overlay cannot handle dot in mapset name which probably points to a wrong handling of mapset in a parameter.

Code (Python/Bash) to test this separately is welcome.

Vaclav

[1] http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2014-09-17-07-00/report_for_nc_spm_08_grass7_nc/temporal/t.vect.mapcalc/test.t.vect.mapcalc/index.html

On Wed, Sep 17, 2014 at 3:33 PM, Vaclav Petras <wenzeslaus@gmail.com> wrote:

Hi,

it seems [1] that v.overlay cannot handle dot in mapset name which probably
points to a wrong handling of mapset in a parameter.

How did you manage to add @<mapset> to the output name? With the GUI
this is not possible. On the command line you should not add @<mapset>
to a vector output (this might apply also to raster output, I did not
check) because output maps are always created in the current mapset.
The error appears not in the module but in the library, thus affects
all modules who care to check the validity of a vector output name.
IOW, the vector output name must not be a fully qualified name. I have
fixed that in r62023,4 for trunk and relbr_70. The warning is now

WARNING: Illegal vector map name
         <tmp_map_name_27967_1@__temporal_t_vect_mapcalc_test.t.vect.mapcalc>.
         Character '@' not allowed.

because '@' occurs before '.' in the fully qualified name.

Markus M

On Wed, Sep 17, 2014 at 3:50 PM, Markus Metz <markus.metz.giswork@gmail.com>
wrote:

On Wed, Sep 17, 2014 at 3:33 PM, Vaclav Petras <wenzeslaus@gmail.com>
wrote:
> Hi,
>
> it seems [1] that v.overlay cannot handle dot in mapset name which
probably
> points to a wrong handling of mapset in a parameter.

How did you manage to add @<mapset> to the output name? With the GUI
this is not possible. On the command line you should not add @<mapset>
to a vector output (this might apply also to raster output, I did not
check) because output maps are always created in the current mapset.

Oh, I didn't realized that.

The error is caused by t.vect.mapcalc (visible in tests). It seems that it
is just adding the mapset name there for some reason.

http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2014-09-17-07-00/report_for_nc_spm_08_grass7_nc/temporal/t.vect.mapcalc/test.t.vect.mapcalc/index.html

The error appears not in the module but in the library, thus affects
all modules who care to check the validity of a vector output name.
IOW, the vector output name must not be a fully qualified name. I have
fixed that in r62023,4 for trunk and relbr_70. The warning is now

WARNING: Illegal vector map name
         <tmp_map_name_27967_1@
__temporal_t_vect_mapcalc_test.t.vect.mapcalc>.
         Character '@' not allowed.

because '@' occurs before '.' in the fully qualified name.

This is much clearer message, thanks.

Vaclav

Markus M

On Wed, Sep 17, 2014 at 10:28 PM, Vaclav Petras <wenzeslaus@gmail.com> wrote:

On Wed, Sep 17, 2014 at 3:50 PM, Markus Metz <markus.metz.giswork@gmail.com>
wrote:

On Wed, Sep 17, 2014 at 3:33 PM, Vaclav Petras <wenzeslaus@gmail.com>
wrote:
> Hi,
>
> it seems [1] that v.overlay cannot handle dot in mapset name which
> probably
> points to a wrong handling of mapset in a parameter.

How did you manage to add @<mapset> to the output name? With the GUI
this is not possible. On the command line you should not add @<mapset>
to a vector output (this might apply also to raster output, I did not
check) because output maps are always created in the current mapset.

Oh, I didn't realized that.

GRASS is designed for multi-user environments such as research
institutions or public administration institutions. A given user has
write permissions only to its home directory, /tmp or %TMP% and the
current mapset which the current user owns in terms of permissions
(dependent on file system and operating system). Any other mapsets are
assumed to belong to different users. This has been pointed out
repeatedly before and is a security feature. Even if you are the owner
of several different mapsets, you are still allowed to create new data
only in the current mapset. This is a security feature needed e.g. for
HPC (High Performance Computing).

Considering that you are a GRASS developer you should have realized that.

Markus M

The error is caused by t.vect.mapcalc (visible in tests). It seems that it
is just adding the mapset name there for some reason.

http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2014-09-17-07-00/report_for_nc_spm_08_grass7_nc/temporal/t.vect.mapcalc/test.t.vect.mapcalc/index.html

The error appears not in the module but in the library, thus affects
all modules who care to check the validity of a vector output name.
IOW, the vector output name must not be a fully qualified name. I have
fixed that in r62023,4 for trunk and relbr_70. The warning is now

WARNING: Illegal vector map name

<tmp_map_name_27967_1@__temporal_t_vect_mapcalc_test.t.vect.mapcalc>.
         Character '@' not allowed.

because '@' occurs before '.' in the fully qualified name.

This is much clearer message, thanks.

Vaclav

Markus M

Vaclav Petras wrote:

it seems [1] that v.overlay cannot handle dot in mapset name which probably
points to a wrong handling of mapset in a parameter.

Vect_check_input_output_name() doesn't accept fully qualified output
names. It assumes that the output name is an unqualified name (i.e. it
passes it directly to Vect_legal_filename). If the mapset didn't
contain a dot, it would instead complain about the output name
containing an "@".

An attempt was made to fix this in r12350, but this is broken by
design. That change simply makes Vect_legal_filename() accept an "@"
character.

This is wrong on two counts. First, it means that Vect_legal_filename()
no longer performs its original function, i.e. validating an
unqualified map name. Second, it means that it Vect_legal_filename() is
still of no use for validating qualified map names, because it applies
the more restrictive vector names rules[1] to the mapset part of the
name.

I notice that this change has effectively been reverted in r62023
(although the existing version has a redundancy: it explicitly rejects
"@" even though this will already be rejected by virtue of not being an
alphanumeric or underscore).

Either Vect_check_input_output_name() needs to be modified to accept
fully qualified output names (presumably checking that the mapset
component matches the current mapset), or whatever part of TGIS is
passing qualified output names to v.overlay has to stop doing that.

[1] The restrictions on vector names derive from Radim (the developer
of the current vector library) taking the "easy" route.

Each vector map typically has an associated database table for
attributes. So when you create a new vector map, you need to create a
new table for that map. And the table's name is limited by the SQL
syntax (alphanumerics and underscore only, no SQL reserved words).

How to choose an SQL table name which is valid and unique? Easy: just
make vector map names subject to the same restrictions as SQL table
names and use the map name as-is. Well, easy for the developer of the
vector library, not necessarily for everyone else.

--
Glynn Clements <glynn@gclements.plus.com>

On Wed, Sep 17, 2014 at 5:24 PM, Markus Metz <markus.metz.giswork@gmail.com>
wrote:

On Wed, Sep 17, 2014 at 10:28 PM, Vaclav Petras <wenzeslaus@gmail.com>
wrote:
>
>
> On Wed, Sep 17, 2014 at 3:50 PM, Markus Metz <
markus.metz.giswork@gmail.com>
> wrote:
>>
>> On Wed, Sep 17, 2014 at 3:33 PM, Vaclav Petras <wenzeslaus@gmail.com>
>> wrote:
>> > Hi,
>> >
>> > it seems [1] that v.overlay cannot handle dot in mapset name which
>> > probably
>> > points to a wrong handling of mapset in a parameter.
>>
>> How did you manage to add @<mapset> to the output name? With the GUI
>> this is not possible. On the command line you should not add @<mapset>
>> to a vector output (this might apply also to raster output, I did not
>> check) because output maps are always created in the current mapset.
>
>
> Oh, I didn't realized that.

GRASS is designed for multi-user environments such as research
institutions or public administration institutions. A given user has
write permissions only to its home directory, /tmp or %TMP% and the
current mapset which the current user owns in terms of permissions
(dependent on file system and operating system). Any other mapsets are
assumed to belong to different users. This has been pointed out
repeatedly before and is a security feature. Even if you are the owner
of several different mapsets, you are still allowed to create new data
only in the current mapset. This is a security feature needed e.g. for
HPC (High Performance Computing).

Considering that you are a GRASS developer you should have realized that.

Oh, sorry for confusion. I actually know this. I didn't realized that it

is output, not input, and so there shouldn't be any mapset at all.

I was basically just performing the function which the testing framework is
not yet doing: sending emails about what is broken.

Thanks for the explanation anyway, I was recently giving one but next time
I will use yours since it is nicely written. It might and up in the manual
too.

Vaclav

Markus M

>
> The error is caused by t.vect.mapcalc (visible in tests). It seems that
it
> is just adding the mapset name there for some reason.
>
>
http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2014-09-17-07-00/report_for_nc_spm_08_grass7_nc/temporal/t.vect.mapcalc/test.t.vect.mapcalc/index.html
>
>>
>> The error appears not in the module but in the library, thus affects
>> all modules who care to check the validity of a vector output name.
>> IOW, the vector output name must not be a fully qualified name. I have
>> fixed that in r62023,4 for trunk and relbr_70. The warning is now
>>
>> WARNING: Illegal vector map name
>>
>> <tmp_map_name_27967_1@__temporal_t_vect_mapcalc_test.t.vect.mapcalc>.
>> Character '@' not allowed.
>>
>> because '@' occurs before '.' in the fully qualified name.
>>
> This is much clearer message, thanks.
>
> Vaclav
>
>>
>> Markus M
>
>

Sorry for bothering you with this again but now I’m confused. The test

http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2014-10-22-07-00/report_for_nc_spm_08_grass7_nc/temporal/t.vect.mapcalc/test.t.vect.mapcalc/index.html

is still failing but after explanation by Glynn in

http://trac.osgeo.org/grass/ticket/2437#comment:18
http://trac.osgeo.org/grass/ticket/2437#comment:23

I have no idea what is the right fix. (I had no idea before but now it is even worse.)

Thanks,

Vaclav

···

On Wed, Sep 17, 2014 at 6:00 PM, Vaclav Petras <wenzeslaus@gmail.com> wrote:

On Wed, Sep 17, 2014 at 5:24 PM, Markus Metz <markus.metz.giswork@gmail.com> wrote:

On Wed, Sep 17, 2014 at 10:28 PM, Vaclav Petras <wenzeslaus@gmail.com> wrote:

On Wed, Sep 17, 2014 at 3:50 PM, Markus Metz <markus.metz.giswork@gmail.com>
wrote:

On Wed, Sep 17, 2014 at 3:33 PM, Vaclav Petras <wenzeslaus@gmail.com>
wrote:

Hi,

it seems [1] that v.overlay cannot handle dot in mapset name which
probably
points to a wrong handling of mapset in a parameter.

How did you manage to add @ to the output name? With the GUI
this is not possible. On the command line you should not add @
to a vector output (this might apply also to raster output, I did not
check) because output maps are always created in the current mapset.

Oh, I didn’t realized that.

GRASS is designed for multi-user environments such as research
institutions or public administration institutions. A given user has
write permissions only to its home directory, /tmp or %TMP% and the
current mapset which the current user owns in terms of permissions
(dependent on file system and operating system). Any other mapsets are
assumed to belong to different users. This has been pointed out
repeatedly before and is a security feature. Even if you are the owner
of several different mapsets, you are still allowed to create new data
only in the current mapset. This is a security feature needed e.g. for
HPC (High Performance Computing).

Considering that you are a GRASS developer you should have realized that.

Oh, sorry for confusion. I actually know this. I didn’t realized that it is output, not input, and so there shouldn’t be any mapset at all.

I was basically just performing the function which the testing framework is not yet doing: sending emails about what is broken.

Thanks for the explanation anyway, I was recently giving one but next time I will use yours since it is nicely written. It might and up in the manual too.

Vaclav

Markus M

The error is caused by t.vect.mapcalc (visible in tests). It seems that it
is just adding the mapset name there for some reason.

http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2014-09-17-07-00/report_for_nc_spm_08_grass7_nc/temporal/t.vect.mapcalc/test.t.vect.mapcalc/index.html

The error appears not in the module but in the library, thus affects
all modules who care to check the validity of a vector output name.
IOW, the vector output name must not be a fully qualified name. I have
fixed that in r62023,4 for trunk and relbr_70. The warning is now

WARNING: Illegal vector map name

<tmp_map_name_27967_1@__temporal_t_vect_mapcalc_test.t.vect.mapcalc>.
Character ‘@’ not allowed.

because ‘@’ occurs before ‘.’ in the fully qualified name.

This is much clearer message, thanks.

Vaclav

Markus M

On Sat, Oct 25, 2014 at 2:26 AM, Vaclav Petras <wenzeslaus@gmail.com> wrote:

Sorry for bothering you with this again but now I'm confused. The test

http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2014-10-22-07-00/report_for_nc_spm_08_grass7_nc/temporal/t.vect.mapcalc/test.t.vect.mapcalc/index.html

This seems to be a bug either in lib/python/temporal or in
lib/python/pygrass. Somewhere the mapset is appended to the output
name. Output is always created in the current mapset, thus output
names are not allowed to contain '@'. This applies to all GRASS
database elements.

Is it possible that pygrass does not discriminate between opening old
and new vector maps, just like pygrass did not discriminate between
opening old and new raster maps?

Markus M

is still failing but after explanation by Glynn in

http://trac.osgeo.org/grass/ticket/2437#comment:18
http://trac.osgeo.org/grass/ticket/2437#comment:23

I have no idea what is the right fix. (I had no idea before but now it is
even worse.)

Thanks,
Vaclav

On Wed, Sep 17, 2014 at 6:00 PM, Vaclav Petras <wenzeslaus@gmail.com> wrote:

On Wed, Sep 17, 2014 at 5:24 PM, Markus Metz
<markus.metz.giswork@gmail.com> wrote:

On Wed, Sep 17, 2014 at 10:28 PM, Vaclav Petras <wenzeslaus@gmail.com>
wrote:
>
>
> On Wed, Sep 17, 2014 at 3:50 PM, Markus Metz
> <markus.metz.giswork@gmail.com>
> wrote:
>>
>> On Wed, Sep 17, 2014 at 3:33 PM, Vaclav Petras <wenzeslaus@gmail.com>
>> wrote:
>> > Hi,
>> >
>> > it seems [1] that v.overlay cannot handle dot in mapset name which
>> > probably
>> > points to a wrong handling of mapset in a parameter.
>>
>> How did you manage to add @<mapset> to the output name? With the GUI
>> this is not possible. On the command line you should not add @<mapset>
>> to a vector output (this might apply also to raster output, I did not
>> check) because output maps are always created in the current mapset.
>
>
> Oh, I didn't realized that.

GRASS is designed for multi-user environments such as research
institutions or public administration institutions. A given user has
write permissions only to its home directory, /tmp or %TMP% and the
current mapset which the current user owns in terms of permissions
(dependent on file system and operating system). Any other mapsets are
assumed to belong to different users. This has been pointed out
repeatedly before and is a security feature. Even if you are the owner
of several different mapsets, you are still allowed to create new data
only in the current mapset. This is a security feature needed e.g. for
HPC (High Performance Computing).

Considering that you are a GRASS developer you should have realized that.

Oh, sorry for confusion. I actually know this. I didn't realized that it
is output, not input, and so there shouldn't be any mapset at all.

I was basically just performing the function which the testing framework
is not yet doing: sending emails about what is broken.

Thanks for the explanation anyway, I was recently giving one but next time
I will use yours since it is nicely written. It might and up in the manual
too.

Vaclav

Markus M

>
> The error is caused by t.vect.mapcalc (visible in tests). It seems that
> it
> is just adding the mapset name there for some reason.
>
>
> http://fatra.cnr.ncsu.edu/grassgistests/reports_for_date-2014-09-17-07-00/report_for_nc_spm_08_grass7_nc/temporal/t.vect.mapcalc/test.t.vect.mapcalc/index.html
>
>>
>> The error appears not in the module but in the library, thus affects
>> all modules who care to check the validity of a vector output name.
>> IOW, the vector output name must not be a fully qualified name. I have
>> fixed that in r62023,4 for trunk and relbr_70. The warning is now
>>
>> WARNING: Illegal vector map name
>>
>> <tmp_map_name_27967_1@__temporal_t_vect_mapcalc_test.t.vect.mapcalc>.
>> Character '@' not allowed.
>>
>> because '@' occurs before '.' in the fully qualified name.
>>
> This is much clearer message, thanks.
>
> Vaclav
>
>>
>> Markus M
>
>

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Markus Metz wrote:

Output is always created in the current mapset, thus output
names are not allowed to contain '@'.

That's incorrect.

Output is always created in the current mapset. That doesn't mean that
the name cannot be fully qualified, just that any explicit mapset part
must refer to the current mapset.

Rejecting a fully-qualified output name is a bug, and should be fixed
rather than being worked around.

--
Glynn Clements <glynn@gclements.plus.com>

On Sun, Oct 26, 2014 at 12:31 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Metz wrote:

Output is always created in the current mapset, thus output
names are not allowed to contain '@'.

That's incorrect.

Output is always created in the current mapset. That doesn't mean that
the name cannot be fully qualified, just that any explicit mapset part
must refer to the current mapset.

Rejecting a fully-qualified output name is a bug, and should be fixed
rather than being worked around.

Right, I fixed r12350 in r62389,90. test.t.vect.mapcalc should work now.

Markus M