[GRASS5] Layers Clarification

Thanks you all for your comments.

I want to make sure I understand so I will reiterate what I
understand you to have said.

1. What is a vector file "layer".

In a GRASS vector file there is a single collection of spatial objects.
Those spatial objects can be associated with one or more keys (called
layers in current terminology). The use of these multiple keys allow
for grouping of topologically related objects in a single layer that
are thematically different (forests and lakes or lines and nodes). This
also allows for linking different series of attribute data to a single
object.

In no way whatsoever are the spatial objects duplicated by the use of
multiple keys (layers).

2. Naming.

Lots of other software uses this ambiguous terminology so GRASS
developers followed this to remain compatible.

Now my comments.

What this multiple keys feature in GRASS represents is an effort to
provide partial DBMS support for attribute management. The initial
choice of a dbf backend necessitated this as it has no such
functionality. For some users, real databases like PostgreSQL are
overkill and too challenging, so making that the default environment
and letting people manage their multiple tables in that way was not
possible.

Now, as someone who has many years of database programming experience,
I think if this were to be done over, the DBMS end should have been
managed in a single environment like SQLite or PostgreSQL. It is
cleaner and clearer and provides a single interface for functionality
without duplication. Now no matter what we do, users will always face
using the limited DBMS functionality of GRASS for normal use but have
the additional facilities provided by PostgreSQL and SQLite which won't
be directly supported in GRASS proper.

However the situation is that the current limited DBMS functionality
(one to one and many to one joins) is done and working. To rewrite and
reduced database backends to only SQLite and other SQL solutions is
probably too limiting for most users and requires rewriting existing
code that works.

As messy as the current situation remains, I can understand and
support not changing it for the time being, although I would suggest
that in the future, should we need to seriously modify vectors again,
we split off DBMS functionality to make things nice and clean.

What does remain however is what I see as very poor naming. Perhaps if
one is not a native English speaker or one has a regular working
knowledge of the other tools using this ambiguous terminology, then it
maybe it doesn't sound so bad or odd. However to me and I would guess
to most English speakers with some GIS background, but not GRASS,
this would be very confusing. I'm not suggesting that English speakers
should rule the world or anything but since we are discussing the
English naming, I think that normal word usage in English is pertinent
to the discussion.

In terms of naming I can see two ways to look at it. Under the current
GRASS environment, layers was approximately equivalent to files.
However, my understanding of Radim's comments is that under QGIS GRASS
vector keys show up as separate layers when composing a map. If we were
to do the same thing in GRASS, then perhaps using the same or a
modified terminology would be reasonable. However, since we currently
select layers for display as files and then set limits on what is
shown from that file, then using a terminology like keys or classes is
more appropriate and intuitively obvious. So as part of deciding if
"layers" needs to be renamed, we should consider how we want to present
the visual map concept of layers to users as part of that discussion.

I look forward to hearing more comments.

Thanks again.

T
--
Trevor Wiens
twiens@interbaun.com

The significant problems that we face cannot be solved at the same
level of thinking we were at when we created them.
(Albert Einstein)

Trevor,

You have stated the situation very clearly. I agree with your comments.

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
Arizona State University
Tempe, AZ 85287-2402

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: Trevor Wiens <twiens@interbaun.com>
Date: Sat, 4 Mar 2006 14:53:55 -0700
To: GRASS5 <grass5@grass.itc.it>
Cc: Radim Blazek <radim.blazek@gmail.com>, Helena Mitasova
<hmitaso@unity.ncsu.edu>, Martin Landa <landa.martin@gmail.com>, Michael
Barton <michael.barton@asu.edu>
Subject: Layers Clarification

Thanks you all for your comments.

I want to make sure I understand so I will reiterate what I
understand you to have said.

1. What is a vector file "layer".

In a GRASS vector file there is a single collection of spatial objects.
Those spatial objects can be associated with one or more keys (called
layers in current terminology). The use of these multiple keys allow
for grouping of topologically related objects in a single layer that
are thematically different (forests and lakes or lines and nodes). This
also allows for linking different series of attribute data to a single
object.

In no way whatsoever are the spatial objects duplicated by the use of
multiple keys (layers).

2. Naming.

Lots of other software uses this ambiguous terminology so GRASS
developers followed this to remain compatible.

Now my comments.

What this multiple keys feature in GRASS represents is an effort to
provide partial DBMS support for attribute management. The initial
choice of a dbf backend necessitated this as it has no such
functionality. For some users, real databases like PostgreSQL are
overkill and too challenging, so making that the default environment
and letting people manage their multiple tables in that way was not
possible.

Now, as someone who has many years of database programming experience,
I think if this were to be done over, the DBMS end should have been
managed in a single environment like SQLite or PostgreSQL. It is
cleaner and clearer and provides a single interface for functionality
without duplication. Now no matter what we do, users will always face
using the limited DBMS functionality of GRASS for normal use but have
the additional facilities provided by PostgreSQL and SQLite which won't
be directly supported in GRASS proper.

However the situation is that the current limited DBMS functionality
(one to one and many to one joins) is done and working. To rewrite and
reduced database backends to only SQLite and other SQL solutions is
probably too limiting for most users and requires rewriting existing
code that works.

As messy as the current situation remains, I can understand and
support not changing it for the time being, although I would suggest
that in the future, should we need to seriously modify vectors again,
we split off DBMS functionality to make things nice and clean.

What does remain however is what I see as very poor naming. Perhaps if
one is not a native English speaker or one has a regular working
knowledge of the other tools using this ambiguous terminology, then it
maybe it doesn't sound so bad or odd. However to me and I would guess
to most English speakers with some GIS background, but not GRASS,
this would be very confusing. I'm not suggesting that English speakers
should rule the world or anything but since we are discussing the
English naming, I think that normal word usage in English is pertinent
to the discussion.

In terms of naming I can see two ways to look at it. Under the current
GRASS environment, layers was approximately equivalent to files.
However, my understanding of Radim's comments is that under QGIS GRASS
vector keys show up as separate layers when composing a map. If we were
to do the same thing in GRASS, then perhaps using the same or a
modified terminology would be reasonable. However, since we currently
select layers for display as files and then set limits on what is
shown from that file, then using a terminology like keys or classes is
more appropriate and intuitively obvious. So as part of deciding if
"layers" needs to be renamed, we should consider how we want to present
the visual map concept of layers to users as part of that discussion.

I look forward to hearing more comments.

Thanks again.

T
--
Trevor Wiens
twiens@interbaun.com

The significant problems that we face cannot be solved at the same
level of thinking we were at when we created them.
(Albert Einstein)

I can only repeate that the current use of 'layer' fits well to other
packages and OGC standards. The name was already changed once
(note that the original name 'field' was suggested by native english speaker).
In any case we cannot change module options in stable 6.x line
because that would break all users scripts, documentation etc.

If I get time, I would like to add support for dynamic reading from OGR
data sources (without v.external). The OGR data source will be used
as map name and OGR layer as GRASS layer. For example:
r.to.vect input='PG:dbname=mydb' layer=mytable
It would be very confusing to change the name for 'layer' in GRASS,
it is equivalent to OGR layer.

The GRASS layers looks exactly the same in QGIS as various
data types of Coverage in ArcView so it should not confuse users.

Radim

On 3/4/06, Trevor Wiens <twiens@interbaun.com> wrote:

Thanks you all for your comments.

I want to make sure I understand so I will reiterate what I
understand you to have said.

1. What is a vector file "layer".

In a GRASS vector file there is a single collection of spatial objects.
Those spatial objects can be associated with one or more keys (called
layers in current terminology). The use of these multiple keys allow
for grouping of topologically related objects in a single layer that
are thematically different (forests and lakes or lines and nodes). This
also allows for linking different series of attribute data to a single
object.

In no way whatsoever are the spatial objects duplicated by the use of
multiple keys (layers).

2. Naming.

Lots of other software uses this ambiguous terminology so GRASS
developers followed this to remain compatible.

Now my comments.

What this multiple keys feature in GRASS represents is an effort to
provide partial DBMS support for attribute management. The initial
choice of a dbf backend necessitated this as it has no such
functionality. For some users, real databases like PostgreSQL are
overkill and too challenging, so making that the default environment
and letting people manage their multiple tables in that way was not
possible.

Now, as someone who has many years of database programming experience,
I think if this were to be done over, the DBMS end should have been
managed in a single environment like SQLite or PostgreSQL. It is
cleaner and clearer and provides a single interface for functionality
without duplication. Now no matter what we do, users will always face
using the limited DBMS functionality of GRASS for normal use but have
the additional facilities provided by PostgreSQL and SQLite which won't
be directly supported in GRASS proper.

However the situation is that the current limited DBMS functionality
(one to one and many to one joins) is done and working. To rewrite and
reduced database backends to only SQLite and other SQL solutions is
probably too limiting for most users and requires rewriting existing
code that works.

As messy as the current situation remains, I can understand and
support not changing it for the time being, although I would suggest
that in the future, should we need to seriously modify vectors again,
we split off DBMS functionality to make things nice and clean.

What does remain however is what I see as very poor naming. Perhaps if
one is not a native English speaker or one has a regular working
knowledge of the other tools using this ambiguous terminology, then it
maybe it doesn't sound so bad or odd. However to me and I would guess
to most English speakers with some GIS background, but not GRASS,
this would be very confusing. I'm not suggesting that English speakers
should rule the world or anything but since we are discussing the
English naming, I think that normal word usage in English is pertinent
to the discussion.

In terms of naming I can see two ways to look at it. Under the current
GRASS environment, layers was approximately equivalent to files.
However, my understanding of Radim's comments is that under QGIS GRASS
vector keys show up as separate layers when composing a map. If we were
to do the same thing in GRASS, then perhaps using the same or a
modified terminology would be reasonable. However, since we currently
select layers for display as files and then set limits on what is
shown from that file, then using a terminology like keys or classes is
more appropriate and intuitively obvious. So as part of deciding if
"layers" needs to be renamed, we should consider how we want to present
the visual map concept of layers to users as part of that discussion.

I look forward to hearing more comments.

Thanks again.

T
--
Trevor Wiens
twiens@interbaun.com

The significant problems that we face cannot be solved at the same
level of thinking we were at when we created them.
(Albert Einstein)

Trevor Wiens wrote:

Thanks you all for your comments.

I want to make sure I understand so I will reiterate what I
understand you to have said.

1. What is a vector file "layer".

In a GRASS vector file there is a single collection of spatial objects.
Those spatial objects can be associated with one or more keys (called
layers in current terminology). The use of these multiple keys allow
for grouping of topologically related objects in a single layer that
are thematically different (forests and lakes or lines and nodes). This
also allows for linking different series of attribute data to a single
object.

Taking the risk of confusing things a bit more, but I do want to try to understand. In an exchange with Radim on layers in September, we came to the following conclusion of the discussion:

***********
On 9/16/05, Moritz Lennert <mlennert@club.worldonline.be> wrote:
> Ok, so it was a problem of me not understanding layers correctly.
> Sorry...
> Do I then understand correctly that you cannot link one layer to
> different tables simultaneously ?

Yes, you cannot.

Radim
**************

This seems to contradict what Trevor write above, i.e. "This also allows for linking different series of attribute data to a single object."

So if I want to link different data tables simultaneously to the same object, I have to duplicate the object in two layers, or ?

The idea of layers, as I understood it so far, is to be able to have different objects linked to separate tables, but all in the same topological file.

Moritz

You cannot link one layer to 2 tables but you can link
one geometry object to 2 tables. I.e. the geometry must
have 2 categories in different layers.
That means you can link objects in one map to more tables
but objects in one layer are always linked to one table only.

Radim

On 3/6/06, Moritz Lennert <mlennert@club.worldonline.be> wrote:

Trevor Wiens wrote:
> Thanks you all for your comments.
>
> I want to make sure I understand so I will reiterate what I
> understand you to have said.
>
> 1. What is a vector file "layer".
>
> In a GRASS vector file there is a single collection of spatial objects.
> Those spatial objects can be associated with one or more keys (called
> layers in current terminology). The use of these multiple keys allow
> for grouping of topologically related objects in a single layer that
> are thematically different (forests and lakes or lines and nodes). This
> also allows for linking different series of attribute data to a single
> object.

Taking the risk of confusing things a bit more, but I do want to try to
understand. In an exchange with Radim on layers in September, we came to
the following conclusion of the discussion:

***********
On 9/16/05, Moritz Lennert <mlennert@club.worldonline.be> wrote:
> Ok, so it was a problem of me not understanding layers correctly.
> Sorry...
> Do I then understand correctly that you cannot link one layer to
> different tables simultaneously ?

Yes, you cannot.

Radim
**************

This seems to contradict what Trevor write above, i.e. "This also allows
for linking different series of attribute data to a single object."

So if I want to link different data tables simultaneously to the same
object, I have to duplicate the object in two layers, or ?

The idea of layers, as I understood it so far, is to be able to have
different objects linked to separate tables, but all in the same
topological file.

Moritz

Moritz,

I want to embellish on Radim's response a bit.

ONE vector object can join with can be joined with ONE attribute table for
EACH "layer" it has. If it has ONE layer, it can join with ONE table, if it
has TWO layers, it can join with TWO tables.

Each "layer" is a data field or column (integer format) named "cat". Each
vector object has at least 1 cat column ("layer 1") defined by default. You
can optionally add more cat columns ("layer 2", "layer 3", ...). Each cat
column serves as a key field to make a join with an attribute table.

The cat values in different layers can be completely different. For example,
a particular vector point can have cat=1 for layer 1, cat=5 for layer 2, and
cat=38 for layer 3.

The "cat" integers in a "layer" key field must match with a corresponding
key field (integer format) in the attribute table you want to join. To
continue with the above example, the vector point would join with a
line/record in an attribute table whose key field has a value of 1 for layer
1, with a line/record in a DIFFERENT table with whose key field=5 for layer
2, and a line/record in a THIRD table whose key field=38 for layer 3.

In this way, a single vector can be linked with (i.e., one-to-one or
many-to-one) multiple attribute tables, *IF* you have defined multiple
layers. But ONE layer, can only be joined with ONE attribute table.

I find this a very useful feature, but agree with Trevor that it needs to be
documented better. I wrote up something on the GRASS WIKI that could be
added to the docs but I haven't had time to do it yet.

I hope this helps.

Michael
______________________________
Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
Arizona State University
Tempe, AZ 85287-2402
USA

voice: 480-965-6262; fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: Moritz Lennert <mlennert@club.worldonline.be>
Date: Mon, 06 Mar 2006 11:17:14 +0100
To: Trevor Wiens <twiens@interbaun.com>
Cc: GRASS5 <grass5@grass.itc.it>, Radim Blazek <radim.blazek@gmail.com>,
Helena Mitasova <hmitaso@unity.ncsu.edu>, Martin Landa
<landa.martin@gmail.com>, Michael Barton <michael.barton@asu.edu>
Subject: Re: [GRASS5] Layers Clarification

Trevor Wiens wrote:

Thanks you all for your comments.

I want to make sure I understand so I will reiterate what I
understand you to have said.

1. What is a vector file "layer".

In a GRASS vector file there is a single collection of spatial objects.
Those spatial objects can be associated with one or more keys (called
layers in current terminology). The use of these multiple keys allow
for grouping of topologically related objects in a single layer that
are thematically different (forests and lakes or lines and nodes). This
also allows for linking different series of attribute data to a single
object.

Taking the risk of confusing things a bit more, but I do want to try to
understand. In an exchange with Radim on layers in September, we came to
the following conclusion of the discussion:

***********
On 9/16/05, Moritz Lennert <mlennert@club.worldonline.be> wrote:

Ok, so it was a problem of me not understanding layers correctly.
Sorry...
Do I then understand correctly that you cannot link one layer to
different tables simultaneously ?

Yes, you cannot.

Radim
**************

This seems to contradict what Trevor write above, i.e. "This also allows
for linking different series of attribute data to a single object."

So if I want to link different data tables simultaneously to the same
object, I have to duplicate the object in two layers, or ?

The idea of layers, as I understood it so far, is to be able to have
different objects linked to separate tables, but all in the same
topological file.

Moritz

Dear Radim and Michael,

Michael Barton wrote:

Moritz,

I want to embellish on Radim's response a bit.

ONE vector object can join with can be joined with ONE attribute table for
EACH "layer" it has. If it has ONE layer, it can join with ONE table, if it
has TWO layers, it can join with TWO tables.

Each "layer" is a data field or column (integer format) named "cat". Each
vector object has at least 1 cat column ("layer 1") defined by default. You
can optionally add more cat columns ("layer 2", "layer 3", ...). Each cat
column serves as a key field to make a join with an attribute table.

The cat values in different layers can be completely different. For example,
a particular vector point can have cat=1 for layer 1, cat=5 for layer 2, and
cat=38 for layer 3.

The "cat" integers in a "layer" key field must match with a corresponding
key field (integer format) in the attribute table you want to join. To
continue with the above example, the vector point would join with a
line/record in an attribute table whose key field has a value of 1 for layer
1, with a line/record in a DIFFERENT table with whose key field=5 for layer
2, and a line/record in a THIRD table whose key field=38 for layer 3.

In this way, a single vector can be linked with (i.e., one-to-one or
many-to-one) multiple attribute tables, *IF* you have defined multiple
layers. But ONE layer, can only be joined with ONE attribute table.

Thanks for your explanations. I now think that I understand layers
correctly.

One question: is it possible to automatically create a second layer for
all features in a file, or does this have to be done via v.digit ?

I find this a very useful feature, but agree with Trevor that it needs to be
documented better. I wrote up something on the GRASS WIKI that could be
added to the docs but I haven't had time to do it yet.

I agree that it seems an interesting feature, although I do agree with
Trevor that what you explain above is something you should also be able to deal with in the database (either putting all the variables into the same table with some objects having NULL values for some of the values (but I know that NULLs in databases are not acceptable to everyone), or you can work with foreign keys, views and more sophisticated select statements. This should actually not be too complicated with the current way GRASS works. Or am I still misunderstanding something ?

Maybe more concrete usage examples (and a bit more extensive than the
ones that Radim gave) might make it easier to understand the added value
of layers.

Moritz

v.category allow you to create and automatically populate a new layer.

Michael
______________________________
Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
Arizona State University
Tempe, AZ 85287-2402
USA

voice: 480-965-6262; fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: Moritz Lennert <mlennert@club.worldonline.be>
Date: Tue, 07 Mar 2006 10:21:58 +0100
To: Michael Barton <michael.barton@asu.edu>
Cc: Trevor Wiens <twiens@interbaun.com>, GRASS5 <grass5@grass.itc.it>, Radim
Blazek <radim.blazek@gmail.com>, Helena Mitasova <hmitaso@unity.ncsu.edu>,
Martin Landa <landa.martin@gmail.com>
Subject: Re: [GRASS5] Layers Clarification

Dear Radim and Michael,

Michael Barton wrote:

Moritz,

I want to embellish on Radim's response a bit.

ONE vector object can join with can be joined with ONE attribute table for
EACH "layer" it has. If it has ONE layer, it can join with ONE table, if it
has TWO layers, it can join with TWO tables.

Each "layer" is a data field or column (integer format) named "cat". Each
vector object has at least 1 cat column ("layer 1") defined by default. You
can optionally add more cat columns ("layer 2", "layer 3", ...). Each cat
column serves as a key field to make a join with an attribute table.

The cat values in different layers can be completely different. For example,
a particular vector point can have cat=1 for layer 1, cat=5 for layer 2, and
cat=38 for layer 3.

The "cat" integers in a "layer" key field must match with a corresponding
key field (integer format) in the attribute table you want to join. To
continue with the above example, the vector point would join with a
line/record in an attribute table whose key field has a value of 1 for layer
1, with a line/record in a DIFFERENT table with whose key field=5 for layer
2, and a line/record in a THIRD table whose key field=38 for layer 3.

In this way, a single vector can be linked with (i.e., one-to-one or
many-to-one) multiple attribute tables, *IF* you have defined multiple
layers. But ONE layer, can only be joined with ONE attribute table.

Thanks for your explanations. I now think that I understand layers
correctly.

One question: is it possible to automatically create a second layer for
all features in a file, or does this have to be done via v.digit ?

I find this a very useful feature, but agree with Trevor that it needs to be
documented better. I wrote up something on the GRASS WIKI that could be
added to the docs but I haven't had time to do it yet.

I agree that it seems an interesting feature, although I do agree with
Trevor that what you explain above is something you should also be able
to deal with in the database (either putting all the variables into the
same table with some objects having NULL values for some of the values
(but I know that NULLs in databases are not acceptable to everyone), or
you can work with foreign keys, views and more sophisticated select
statements. This should actually not be too complicated with the current
way GRASS works. Or am I still misunderstanding something ?

Maybe more concrete usage examples (and a bit more extensive than the
ones that Radim gave) might make it easier to understand the added value
of layers.

Moritz