[GRASS-dev] Vector file naming

GRASS 6.3.cvs (hamilton2):/usr/src/grass6 > r.to.vect in=elizabeth.los
out=elizabeth.los feature=area
Illegal vector map name <elizabeth.los>. Character <.> not allowed.
ERROR: Map name is not SQL compliant.

How do file names conflict with SQL92? I thought SQL92 only applied to
file contents. Is this also being used as a workaround for DBF
limitations? What I'm getting at is: Why is '.' not allowed? Doing a
quick archive search failed to illuminate any light bulbs in the
immediate vicinity.

--
Brad Douglas <rez touchofmadness com> KB8UYR
Address: 37.493,-121.924 / WGS84 National Map Corps #TNMC-3785

On Thu, Sep 07, 2006 at 11:12:18AM +0000, Brad Douglas wrote:

GRASS 6.3.cvs (hamilton2):/usr/src/grass6 > r.to.vect in=elizabeth.los
out=elizabeth.los feature=area
Illegal vector map name <elizabeth.los>. Character <.> not allowed.
ERROR: Map name is not SQL compliant.

How do file names conflict with SQL92? I thought SQL92 only applied to
file contents. Is this also being used as a workaround for DBF
limitations? What I'm getting at is: Why is '.' not allowed? Doing a
quick archive search failed to illuminate any light bulbs in the
immediate vicinity.

Hi Brad,

AFAIK '.' is reserved for joins. '_' will work.

See
http://grass.itc.it/grass63/manuals/html63_user/sql.html
-> NOTES

It would be nice to have the naiming constraints relaxed but
I am not sure if that's really possible.

Markus

On Thu, 2006-09-07 at 13:35 +0200, Markus Neteler wrote:

On Thu, Sep 07, 2006 at 11:12:18AM +0000, Brad Douglas wrote:
> GRASS 6.3.cvs (hamilton2):/usr/src/grass6 > r.to.vect in=elizabeth.los
> out=elizabeth.los feature=area
> Illegal vector map name <elizabeth.los>. Character <.> not allowed.
> ERROR: Map name is not SQL compliant.
>
> How do file names conflict with SQL92? I thought SQL92 only applied to
> file contents. Is this also being used as a workaround for DBF
> limitations? What I'm getting at is: Why is '.' not allowed? Doing a
> quick archive search failed to illuminate any light bulbs in the
> immediate vicinity.

Hi Brad,

AFAIK '.' is reserved for joins. '_' will work.

See
http://grass.itc.it/grass63/manuals/html63_user/sql.html
-> NOTES

It would be nice to have the naiming constraints relaxed but
I am not sure if that's really possible.

'_' is also reserved for matching, so that can't be it.

Maybe a better question is: Why doesn't GRASS distinguish between tables
and files (database)? I would much prefer to have GRASS automatically
substitute '_' for '.' in the table name so I can keep consistent naming
conventions across rasters and vectors. It's a usability issue and
somewhat annoying. :slight_smile:

I guess the real question is: How big of a job would this be? Radim?
If it isn't an overwhelming job, I'd like to propose it for 7.x.

--
Brad Douglas <rez touchofmadness com> KB8UYR
Address: 37.493,-121.924 / WGS84 National Map Corps #TNMC-3785

On Thu, 7 Sep 2006, Markus Neteler wrote:

On Thu, Sep 07, 2006 at 11:12:18AM +0000, Brad Douglas wrote:

GRASS 6.3.cvs (hamilton2):/usr/src/grass6 > r.to.vect in=elizabeth.los
out=elizabeth.los feature=area
Illegal vector map name <elizabeth.los>. Character <.> not allowed.
ERROR: Map name is not SQL compliant.

How do file names conflict with SQL92? I thought SQL92 only applied to
file contents. Is this also being used as a workaround for DBF
limitations? What I'm getting at is: Why is '.' not allowed? Doing a
quick archive search failed to illuminate any light bulbs in the
immediate vicinity.

Hi Brad,

AFAIK '.' is reserved for joins. '_' will work.

See
http://grass.itc.it/grass63/manuals/html63_user/sql.html
-> NOTES

It would be nice to have the naiming constraints relaxed but
I am not sure if that's really possible.

IIUC this is related to the earlier discussion about database files? With
the current situation of having one database per mapset, each vector map has its own table in the database which matches the name of the map. And so the name of the map must conform to SQL table naming rules. If it was changed to have one database file per map then the SQL naming rules wouldn't be a problem. But TBH it does seem logical to have one database per mapset, as Radim said for queries over multiple maps etc. In fact really, one database for the whole location seems most logical. And some kind of relation to relate the table names in the database to the mapset and map they corresponded with. That would require a terrible lot of programming though I'm sure. Just thinking out loud.

Paul

On Thu, 07 Sep 2006 12:40:46 +0000
Brad Douglas wrote:

On Thu, 2006-09-07 at 13:35 +0200, Markus Neteler wrote:
> On Thu, Sep 07, 2006 at 11:12:18AM +0000, Brad Douglas wrote:
> > GRASS 6.3.cvs (hamilton2):/usr/src/grass6 > r.to.vect in=elizabeth.los
> > out=elizabeth.los feature=area
> > Illegal vector map name <elizabeth.los>. Character <.> not allowed.
> > ERROR: Map name is not SQL compliant.
> >
> > How do file names conflict with SQL92? I thought SQL92 only applied to
> > file contents. Is this also being used as a workaround for DBF
> > limitations? What I'm getting at is: Why is '.' not allowed? Doing a
> > quick archive search failed to illuminate any light bulbs in the
> > immediate vicinity.
>
> Hi Brad,
>
> AFAIK '.' is reserved for joins. '_' will work.
>
> See
> http://grass.itc.it/grass63/manuals/html63_user/sql.html
> -> NOTES
>
> It would be nice to have the naiming constraints relaxed but
> I am not sure if that's really possible.

'_' is also reserved for matching, so that can't be it.

Maybe a better question is: Why doesn't GRASS distinguish between tables
and files (database)? I would much prefer to have GRASS automatically
substitute '_' for '.' in the table name so I can keep consistent naming
conventions across rasters and vectors. It's a usability issue and
somewhat annoying. :slight_smile:

Well... To my knowledge standard SQL requires a letter as the first
character of a table name, and after that letters, numbers, or
underscores are permitted without quotation. If you want to break those
rules double quotes are required. So the error message is correct.
Using a period in a table name without double quoting is a violation of
SQL naming rules. Normally a period is used to related columns to
tables, aliases etc.

For example

select a.x, b.y
from schema_one.table_x a, schema_two.table_y b
where a.q = b.q

Thus it is impossible for SQL to know how to parse this without quotes.

I guess the real question is: How big of a job would this be? Radim?
If it isn't an overwhelming job, I'd like to propose it for 7.x.

Since we want to improve database support it seems reasonable to
provide subtle education for GRASS users about SQL sticking with SQL
naming restrictions. Therefore, it would be more logical to require
rasters to fit the same naming restrictions as vectors for the 7.x
line. There will be many users who will hate this idea as they are
used to using periods in their raster file names, but this would be
consistent.

That said, you are correct that we could implement quoting
of table names to allow rule breaking naming, but considering the
difficulties with quotes and bash, this is just a place we don't want
to go, AFAIC.

So for 7.x we should do one of two things. One, do nothing, as
relaxing the naming rules for vectors wouldn't be difficult, but it
would be unwise and would introduce all new levels frustration for
users not familiar with SQL. Two, enforce SQL and thus vector naming
restrictions for rasters for the sake of consistency.

T
--
Trevor Wiens
twiens@interbaun.com

The significant problems that we face cannot be solved at the same
level of thinking we were at when we created them.
(Albert Einstein)

On Thu, 2006-09-07 at 08:29 -0600, Trevor Wiens wrote:

On Thu, 07 Sep 2006 12:40:46 +0000
Brad Douglas wrote:

> On Thu, 2006-09-07 at 13:35 +0200, Markus Neteler wrote:
> > On Thu, Sep 07, 2006 at 11:12:18AM +0000, Brad Douglas wrote:
> > > GRASS 6.3.cvs (hamilton2):/usr/src/grass6 > r.to.vect in=elizabeth.los
> > > out=elizabeth.los feature=area
> > > Illegal vector map name <elizabeth.los>. Character <.> not allowed.
> > > ERROR: Map name is not SQL compliant.
> > >
> > > How do file names conflict with SQL92? I thought SQL92 only applied to
> > > file contents. Is this also being used as a workaround for DBF
> > > limitations? What I'm getting at is: Why is '.' not allowed? Doing a
> > > quick archive search failed to illuminate any light bulbs in the
> > > immediate vicinity.
> >
> > Hi Brad,
> >
> > AFAIK '.' is reserved for joins. '_' will work.
> >
> > See
> > http://grass.itc.it/grass63/manuals/html63_user/sql.html
> > -> NOTES
> >
> > It would be nice to have the naiming constraints relaxed but
> > I am not sure if that's really possible.
>
> '_' is also reserved for matching, so that can't be it.
>
> Maybe a better question is: Why doesn't GRASS distinguish between tables
> and files (database)? I would much prefer to have GRASS automatically
> substitute '_' for '.' in the table name so I can keep consistent naming
> conventions across rasters and vectors. It's a usability issue and
> somewhat annoying. :slight_smile:

Well... To my knowledge standard SQL requires a letter as the first
character of a table name, and after that letters, numbers, or
underscores are permitted without quotation. If you want to break those
rules double quotes are required. So the error message is correct.
Using a period in a table name without double quoting is a violation of
SQL naming rules. Normally a period is used to related columns to
tables, aliases etc.

For example

select a.x, b.y
from schema_one.table_x a, schema_two.table_y b
where a.q = b.q

Thus it is impossible for SQL to know how to parse this without quotes.

Who says the *file name* has to be the same as the table name? No SQL
spec governs that...it's up to the DBMS. GRASS just doesn't give me
that option, currently.

--
Brad Douglas <rez touchofmadness com> KB8UYR
Address: 37.493,-121.924 / WGS84 National Map Corps #TNMC-3785

Brad Douglas wrote on 09/07/2006 04:46 PM:

Who says the *file name* has to be the same as the table name? No SQL
spec governs that...it's up to the DBMS. GRASS just doesn't give me
that option, currently.

I think it does: v.db.connect...

Markus

Markus Neteler wrote:
> Brad Douglas wrote on 09/07/2006 04:46 PM:
>> Who says the *file name* has to be the same as the table name? No SQL
>> spec governs that...it's up to the DBMS. GRASS just doesn't give me
>> that option, currently.
>>
> I think it does: v.db.connect...

Yes, but r.to.vect doesn't allow you _not_ to create the table...unless you work with point data.

Moritz

On Thu, 2006-09-07 at 16:48 +0200, Markus Neteler wrote:

Brad Douglas wrote on 09/07/2006 04:46 PM:
> Who says the *file name* has to be the same as the table name? No SQL
> spec governs that...it's up to the DBMS. GRASS just doesn't give me
> that option, currently.
>
>
I think it does: v.db.connect...

Vect_legal_filename() says no. :frowning:

I understand that I could move entirely to a spacial database, but then
I'm exclusively working with table names because that's where the
geometry is stored. There is no geometry in a DBF, which allows us the
ability to relax out file naming standards a bit by being able to
reference the table and calling something else (the geometry file) the
intended name.

To illustrate:

mv hamilton2/PERMANENT/vector/elizabeth_los/
hamilton2/PERMANENT/vector/elizabeth.los
mv hamilton2/PERMANENT/dbf/elizabeth_los.dbf
hamilton2/PERMANENT/dbf/elizabeth.los.dbf

[brad@] > grass63
GRASS 6.3.cvs (hamilton2): > v.info elizabeth.los
+----------------------------------------------------------------------------+
| Layer: elizabeth.los Organization:
|
| Mapset: PERMANENT Source Date: Thu Sep 7
03:00:55 |
| Location: hamilton2 Name of creator: brad
|

... [file behaves normally].

Either way, it isn't that big of a deal. :slight_smile:

--
Brad Douglas <rez touchofmadness com> KB8UYR
Address: 37.493,-121.924 / WGS84 National Map Corps #TNMC-3785

Trevor Wiens wrote:

On Thu, 07 Sep 2006 12:40:46 +0000
Brad Douglas wrote:

On Thu, 2006-09-07 at 13:35 +0200, Markus Neteler wrote:
    

On Thu, Sep 07, 2006 at 11:12:18AM +0000, Brad Douglas wrote:
      

GRASS 6.3.cvs (hamilton2):/usr/src/grass6 > r.to.vect in=elizabeth.los
out=elizabeth.los feature=area
Illegal vector map name <elizabeth.los>. Character <.> not allowed.
ERROR: Map name is not SQL compliant.

How do file names conflict with SQL92? I thought SQL92 only applied to
file contents. Is this also being used as a workaround for DBF
limitations? What I'm getting at is: Why is '.' not allowed? Doing a
quick archive search failed to illuminate any light bulbs in the
immediate vicinity.
        

Hi Brad,

AFAIK '.' is reserved for joins. '_' will work.

See
http://grass.itc.it/grass63/manuals/html63_user/sql.html
-> NOTES

It would be nice to have the naiming constraints relaxed but
I am not sure if that's really possible.
      

'_' is also reserved for matching, so that can't be it.

Maybe a better question is: Why doesn't GRASS distinguish between tables
and files (database)? I would much prefer to have GRASS automatically
substitute '_' for '.' in the table name so I can keep consistent naming
conventions across rasters and vectors. It's a usability issue and
somewhat annoying. :slight_smile:
    
Well... To my knowledge standard SQL requires a letter as the first
character of a table name, and after that letters, numbers, or
underscores are permitted without quotation. If you want to break those
rules double quotes are required. So the error message is correct.
Using a period in a table name without double quoting is a violation of
SQL naming rules. Normally a period is used to related columns to
tables, aliases etc.

For example

select a.x, b.y
from schema_one.table_x a, schema_two.table_y b
where a.q = b.q

Thus it is impossible for SQL to know how to parse this without quotes.

I guess the real question is: How big of a job would this be? Radim?
If it isn't an overwhelming job, I'd like to propose it for 7.x.
    
Since we want to improve database support it seems reasonable to
provide subtle education for GRASS users about SQL sticking with SQL
naming restrictions. Therefore, it would be more logical to require
rasters to fit the same naming restrictions as vectors for the 7.x
line. There will be many users who will hate this idea as they are
used to using periods in their raster file names, but this would be
consistent.
  

Count me among those who really hate this idea as I have dot in about every single raster file name.
And there are a lot of scripts around that use dots in raster names.
It is already a pain to have that restriction in vector data. I am with Brad on this one,

Helena

That said, you are correct that we could implement quoting
of table names to allow rule breaking naming, but considering the
difficulties with quotes and bash, this is just a place we don't want
to go, AFAIC.

So for 7.x we should do one of two things. One, do nothing, as
relaxing the naming rules for vectors wouldn't be difficult, but it
would be unwise and would introduce all new levels frustration for
users not familiar with SQL. Two, enforce SQL and thus vector naming
restrictions for rasters for the sake of consistency.

T
  
--
Helena Mitasova
Department of Marine, Earth and Atmospheric Sciences
North Carolina State University
1125 Jordan Hall
NCSU Box 8208
Raleigh, NC 27695-8208
http://skagit.meas.ncsu.edu/~helena/

email: hmitaso@unity.ncsu.edu
ph: 919-513-1327 (no voicemail)
fax 919 515-7802

Moritz Lennert wrote:

Yes, but r.to.vect doesn't allow you _not_ to create the
table...unless you work with point data.

a -t flag could be added, please file as a wish if this is important to
your work.

Hamish

Brad Douglas wrote:

> > GRASS 6.3.cvs (hamilton2):/usr/src/grass6 > r.to.vect in=elizabeth.los
> > out=elizabeth.los feature=area
> > Illegal vector map name <elizabeth.los>. Character <.> not allowed.
> > ERROR: Map name is not SQL compliant.
> >
> > How do file names conflict with SQL92? I thought SQL92 only applied to
> > file contents. Is this also being used as a workaround for DBF
> > limitations? What I'm getting at is: Why is '.' not allowed? Doing a
> > quick archive search failed to illuminate any light bulbs in the
> > immediate vicinity.
>
> Hi Brad,
>
> AFAIK '.' is reserved for joins. '_' will work.
>
> See
> http://grass.itc.it/grass63/manuals/html63_user/sql.html
> -> NOTES
>
> It would be nice to have the naiming constraints relaxed but
> I am not sure if that's really possible.

'_' is also reserved for matching, so that can't be it.

No, _ is perfectly legal in column and table names. It is used as a
wildcard by LIKE, but it isn't significant anywhere else.

Maybe a better question is: Why doesn't GRASS distinguish between tables
and files (database)?

Because it isn't limited to the DBF driver.

I would much prefer to have GRASS automatically
substitute '_' for '.' in the table name so I can keep consistent naming
conventions across rasters and vectors.

That could create a problem if you have both map.1 and map_1, as they
would both have the same name.

It's a usability issue and somewhat annoying. :slight_smile:

FWIW, these issues were pointed out when the new vector architecture
was initially introduced. This isn't an oversight; it was a conscious
choice.

I guess the real question is: How big of a job would this be? Radim?
If it isn't an overwhelming job, I'd like to propose it for 7.x.

It would appear to be a big enough job that Radim chose not to do it
when the issues were originally pointed out.

--
Glynn Clements <glynn@gclements.plus.com>

On Fri, 2006-09-08 at 04:34 +0100, Glynn Clements wrote:

Brad Douglas wrote:

> > > GRASS 6.3.cvs (hamilton2):/usr/src/grass6 > r.to.vect in=elizabeth.los
> > > out=elizabeth.los feature=area
> > > Illegal vector map name <elizabeth.los>. Character <.> not allowed.
> > > ERROR: Map name is not SQL compliant.
> > >
> > > How do file names conflict with SQL92? I thought SQL92 only applied to
> > > file contents. Is this also being used as a workaround for DBF
> > > limitations? What I'm getting at is: Why is '.' not allowed? Doing a
> > > quick archive search failed to illuminate any light bulbs in the
> > > immediate vicinity.
> >
> > Hi Brad,
> >
> > AFAIK '.' is reserved for joins. '_' will work.
> >
> > See
> > http://grass.itc.it/grass63/manuals/html63_user/sql.html
> > -> NOTES
> >
> > It would be nice to have the naiming constraints relaxed but
> > I am not sure if that's really possible.
>
> '_' is also reserved for matching, so that can't be it.

No, _ is perfectly legal in column and table names. It is used as a
wildcard by LIKE, but it isn't significant anywhere else.

> Maybe a better question is: Why doesn't GRASS distinguish between tables
> and files (database)?

Because it isn't limited to the DBF driver.

Perhaps this is what I find most frustrating. DBF is not spatially
aware, so why should I be forced to adhere to unnecessary requirements?

I find it somewhat embarrassing that I can simply move the directory
containing geometry to the name I see fit without negative side effects.

> I would much prefer to have GRASS automatically
> substitute '_' for '.' in the table name so I can keep consistent naming
> conventions across rasters and vectors.

That could create a problem if you have both map.1 and map_1, as they
would both have the same name.

In my experience, I find it much less likely to encounter this error
that the former.

> It's a usability issue and somewhat annoying. :slight_smile:

FWIW, these issues were pointed out when the new vector architecture
was initially introduced. This isn't an oversight; it was a conscious
choice.

> I guess the real question is: How big of a job would this be? Radim?
> If it isn't an overwhelming job, I'd like to propose it for 7.x.

It would appear to be a big enough job that Radim chose not to do it
when the issues were originally pointed out.

I understand the logic behind the present behavior, but I believe that
logic would be best served if GRASS ceased to use it's own format and
went with a real spatially aware SQL backend (there are several to
choose from and SQLite as middleware makes interfacing a breeze).
Perhaps, this will be the next incarnation of the vector architecture as
spatial databases have matured significantly in recent years.

GRASS is plagued with usability issues like this that will prevent it
from being a viable competitor to commercial alternatives outside of the
research community.

Ok. This has been discussed to death. Back to work, everyone. :slight_smile:

--
Brad Douglas <rez touchofmadness com> KB8UYR
Address: 37.493,-121.924 / WGS84 National Map Corps #TNMC-3785

Hamish wrote:

Moritz Lennert wrote:

Yes, but r.to.vect doesn't allow you _not_ to create the
table...unless you work with point data.

a -t flag could be added, please file as a wish if this is important to
your work.

I actually think that maybe an option defining the table name of the table to be created together with the map might be the best solution. But this is valid for more than just r.to.vect (v.in.ogr jumps to mind).

But it has never been a problem for me.

Brad wrote:

I understand the logic behind the present behavior, but I believe that
logic would be best served if GRASS ceased to use it's own format and
went with a real spatially aware SQL backend (there are several to
choose from and SQLite as middleware makes interfacing a breeze).
Perhaps, this will be the next incarnation of the vector architecture as
spatial databases have matured significantly in recent years.

I don't really understand how this solves your problem. In a spatially aware SQL backend you wouldn't be able to use dots in table/map names either, or ?

Moritz