[GRASS-user] Trouble with v.db.join and sqlite

Hello everybody.

I am working with 6.4svn (built yesterday) and got some troubles with
v.db.join and sqlite.
If I try to run the command, this is the error reported:

v.db.join map=aste_geologia@aste_geologia column=link otable=reticolo
ocolumn=link

DBMI-SQLite driver error:
Error in sqlite3_prepare():
duplicate column name: cat
ERROR: Error while executing: 'ALTER TABLE aste_geologia ADD COLUMN cat
INTEGER
'
ERROR: Cannot continue (problem adding column).
ERROR: Cannot continue.

If I try to remove the column cat by hand ( as suggested in a post in the ml),
it goes fine:

GRASS 6.4.svn (wrme):~ > db.dropcol -f aste_geo_bak colum=cat
Forcing ...
WARNING: Deleting <cat> column which may be needed to keep table connected
         to a vector map

Any idea?

Thank you very much

Luca

Luca,

can you replicate this with Spearfish or the NC data set?
Then we could easier test things...

Markus

On Mon, May 26, 2008 at 9:22 AM, Luca Casagrande
<luca.casagrande@gmail.com> wrote:

Hello everybody.

I am working with 6.4svn (built yesterday) and got some troubles with
v.db.join and sqlite.
If I try to run the command, this is the error reported:

v.db.join map=aste_geologia@aste_geologia column=link otable=reticolo
ocolumn=link

DBMI-SQLite driver error:
Error in sqlite3_prepare():
duplicate column name: cat
ERROR: Error while executing: 'ALTER TABLE aste_geologia ADD COLUMN cat
INTEGER
'
ERROR: Cannot continue (problem adding column).
ERROR: Cannot continue.

If I try to remove the column cat by hand ( as suggested in a post in the ml),
it goes fine:

GRASS 6.4.svn (wrme):~ > db.dropcol -f aste_geo_bak colum=cat
Forcing ...
WARNING: Deleting <cat> column which may be needed to keep table connected
        to a vector map

Any idea?

Thank you very much

Luca
_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

--
Open Source Geospatial Foundation
http://www.osgeo.org/
http://www.grassbook.org/

Markus Neteler OSGeo wrote:

Luca,

can you replicate this with Spearfish or the NC data set?
Then we could easier test things...

Markus

Hi Markus,

those are the steps to replicate the problem with NC dataset ( all is done
in a new mapset called sqlite):

db.connect driver=sqlite
database='$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite.db'
g.copy vect=urbanarea@PERMANENT,urbanarea
v.extract -t input=urbanarea output=urban_point type=point,centroid
v.db.addtable urban_point

v.db.join map=urbanarea column=cat otable=urban_point ocolumn=cat
..
DBMI-SQLite driver error:
Error in sqlite3_prepare():
duplicate column name: cat

ERROR: Error while executing: 'ALTER TABLE urban_point ADD COLUMN cat
       INTEGER
       '
ERROR: Cannot continue (problem adding column).
ERROR: Cannot continue.

Thanks for your help

Luca

--
View this message in context: http://www.nabble.com/Trouble-with-v.db.join-and-sqlite-tp17467019p17556544.html
Sent from the Grass - Users mailing list archive at Nabble.com.

On Fri, 30 May 2008 04:17:51 -0700 (PDT)
doktoreas <thedoktor78@gmail.com> wrote:

Markus Neteler OSGeo wrote:
>
> Luca,
>
> can you replicate this with Spearfish or the NC data
set?
> Then we could easier test things...
>
> Markus
>
>

Hi Markus,

those are the steps to replicate the problem with NC
dataset ( all is done
in a new mapset called sqlite):

db.connect driver=sqlite
database='$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite.db'
g.copy vect=urbanarea@PERMANENT,urbanarea
v.extract -t input=urbanarea output=urban_point
type=point,centroid
v.db.addtable urban_point

v.db.join map=urbanarea column=cat otable=urban_point
ocolumn=cat
..
DBMI-SQLite driver error:
Error in sqlite3_prepare():
duplicate column name: cat

ERROR: Error while executing: 'ALTER TABLE urban_point
ADD COLUMN cat
       INTEGER
       '
ERROR: Cannot continue (problem adding column).
ERROR: Cannot continue.

Thanks for your help

Luca

Luca,

v.db.join never worked for me when the columns had the same
name. I rename one of the columns and there is no problem.
I was also wondering if there is a reason for that or if it
is a bug.

in your case rename one of the "cat" columns to lets say
"cat_x" and do:
v.db.join map=urbanarea column=cat otable=urban_point
ocolumn=cat_x

Nikos

On Fri, May 30, 2008 at 1:17 PM, doktoreas <thedoktor78@gmail.com> wrote:

Markus Neteler OSGeo wrote:

Luca,

can you replicate this with Spearfish or the NC data set?
Then we could easier test things...

Markus

Hi Markus,

those are the steps to replicate the problem with NC dataset ( all is done
in a new mapset called sqlite):

db.connect driver=sqlite
database='$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite.db'
g.copy vect=urbanarea@PERMANENT,urbanarea
v.extract -t input=urbanarea output=urban_point type=point,centroid
v.db.addtable urban_point

v.db.join map=urbanarea column=cat otable=urban_point ocolumn=cat
..
DBMI-SQLite driver error:
Error in sqlite3_prepare():
duplicate column name: cat

ERROR: Error while executing: 'ALTER TABLE urban_point ADD COLUMN cat
      INTEGER
      '
ERROR: Cannot continue (problem adding column).
ERROR: Cannot continue.

Thanks for your help

Like this it was easy to replicated.
(Hopefully) fixed in 6.4.develbranch and 7.trunk.

You can grab the fixed script also here:
http://trac.osgeo.org/grass/browser/grass/branches/develbranch_6/scripts/v.db.join/v.db.join
(download at page bottom)

Markus

Markus Neteler OSGeo wrote:

On Fri, May 30, 2008 at 1:17 PM, doktoreas <thedoktor78@gmail.com> wrote:

Markus Neteler OSGeo wrote:

Luca,

can you replicate this with Spearfish or the NC data set?
Then we could easier test things...

Markus

Hi Markus,

those are the steps to replicate the problem with NC dataset ( all is
done
in a new mapset called sqlite):

db.connect driver=sqlite
database='$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite.db'
g.copy vect=urbanarea@PERMANENT,urbanarea
v.extract -t input=urbanarea output=urban_point type=point,centroid
v.db.addtable urban_point

v.db.join map=urbanarea column=cat otable=urban_point ocolumn=cat
..
DBMI-SQLite driver error:
Error in sqlite3_prepare():
duplicate column name: cat

ERROR: Error while executing: 'ALTER TABLE urban_point ADD COLUMN cat
      INTEGER
      '
ERROR: Cannot continue (problem adding column).
ERROR: Cannot continue.

Thanks for your help

Like this it was easy to replicated.
(Hopefully) fixed in 6.4.develbranch and 7.trunk.

You can grab the fixed script also here:
http://trac.osgeo.org/grass/browser/grass/branches/develbranch_6/scripts/v.db.join/v.db.join
(download at page bottom)

Markus

Thanks Markus..
It's funny watching realtime-committing from the IRC channel :slight_smile:

Building it right now.

Luca
--
View this message in context: http://www.nabble.com/Trouble-with-v.db.join-and-sqlite-tp17467019p17558967.html
Sent from the Grass - Users mailing list archive at Nabble.com.

On Fri, 2008-05-30 at 06:32 -0700, doktoreas wrote:

Markus Neteler OSGeo wrote:
>
> On Fri, May 30, 2008 at 1:17 PM, doktoreas <thedoktor78@gmail.com> wrote:
>> Markus Neteler OSGeo wrote:
>>>
>>> Luca,
>>>
>>> can you replicate this with Spearfish or the NC data set?
>>> Then we could easier test things...
>>>
>>> Markus
>>>
>>>
>>
>> Hi Markus,
>>
>> those are the steps to replicate the problem with NC dataset ( all is
>> done
>> in a new mapset called sqlite):
>>
>> db.connect driver=sqlite
>> database='$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite.db'
>> g.copy vect=urbanarea@PERMANENT,urbanarea
>> v.extract -t input=urbanarea output=urban_point type=point,centroid
>> v.db.addtable urban_point
>>
>> v.db.join map=urbanarea column=cat otable=urban_point ocolumn=cat
>> ..
>> DBMI-SQLite driver error:
>> Error in sqlite3_prepare():
>> duplicate column name: cat
>>
>> ERROR: Error while executing: 'ALTER TABLE urban_point ADD COLUMN cat
>> INTEGER
>> '
>> ERROR: Cannot continue (problem adding column).
>> ERROR: Cannot continue.
>>
>> Thanks for your help
>
> Like this it was easy to replicated.
> (Hopefully) fixed in 6.4.develbranch and 7.trunk.
>
> You can grab the fixed script also here:
> http://trac.osgeo.org/grass/browser/grass/branches/develbranch_6/scripts/v.db.join/v.db.join
> (download at page bottom)
>

Does this mean that the failure of join when using columns with the same
name was a bug?

Thank you,

Nikos

Nikos Alexandris wrote:

Does this mean that the failure of join when using columns with the same
name was a bug?

Thank you,

Nikos

So it seem, but I have just tested the fix and now all is fine.

Luca
--
View this message in context: http://www.nabble.com/Trouble-with-v.db.join-and-sqlite-tp17467019p17559701.html
Sent from the Grass - Users mailing list archive at Nabble.com.

On 30/05/08 15:17, Markus Neteler wrote:

On Fri, May 30, 2008 at 1:17 PM, doktoreas <thedoktor78@gmail.com> wrote:

Markus Neteler OSGeo wrote:

Luca,

can you replicate this with Spearfish or the NC data set?
Then we could easier test things...

Markus

Hi Markus,

those are the steps to replicate the problem with NC dataset ( all is done
in a new mapset called sqlite):

db.connect driver=sqlite
database='$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite.db'
g.copy vect=urbanarea@PERMANENT,urbanarea
v.extract -t input=urbanarea output=urban_point type=point,centroid
v.db.addtable urban_point

v.db.join map=urbanarea column=cat otable=urban_point ocolumn=cat
..
DBMI-SQLite driver error:
Error in sqlite3_prepare():
duplicate column name: cat

ERROR: Error while executing: 'ALTER TABLE urban_point ADD COLUMN cat
      INTEGER
      '
ERROR: Cannot continue (problem adding column).
ERROR: Cannot continue.

Thanks for your help

Like this it was easy to replicated.
(Hopefully) fixed in 6.4.develbranch and 7.trunk.

Do I understand correctly that the patch ignores the column if the name already exists ? Wouldn't it be more "user-friendly" to rename it ?

Moritz

Moritz Lennert wrote:

>>> can you replicate this with Spearfish or the NC data set?
>>> Then we could easier test things...
>>>
>>> Markus
>>>
>>>
>> Hi Markus,
>>
>> those are the steps to replicate the problem with NC dataset ( all is done
>> in a new mapset called sqlite):
>>
>> db.connect driver=sqlite
>> database='$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite.db'
>> g.copy vect=urbanarea@PERMANENT,urbanarea
>> v.extract -t input=urbanarea output=urban_point type=point,centroid
>> v.db.addtable urban_point
>>
>> v.db.join map=urbanarea column=cat otable=urban_point ocolumn=cat
>> ..
>> DBMI-SQLite driver error:
>> Error in sqlite3_prepare():
>> duplicate column name: cat
>>
>> ERROR: Error while executing: 'ALTER TABLE urban_point ADD COLUMN cat
>> INTEGER
>> '
>> ERROR: Cannot continue (problem adding column).
>> ERROR: Cannot continue.
>>
>> Thanks for your help
>
> Like this it was easy to replicated.
> (Hopefully) fixed in 6.4.develbranch and 7.trunk.

Do I understand correctly that the patch ignores the column if the name
already exists ? Wouldn't it be more "user-friendly" to rename it ?

I don't know about "user-friendly", but it would be safer to just
abort with an error if the column already exists.

If you're going to replace the contents of an existing column, you
should only do so if --overwrite is given.

--
Glynn Clements <glynn@gclements.plus.com>

On Fri, May 30, 2008 at 10:03 PM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Moritz Lennert wrote:

...

> (Hopefully) fixed in 6.4.develbranch and 7.trunk.

Do I understand correctly that the patch ignores the column if the name
already exists ? Wouldn't it be more "user-friendly" to rename it ?

I don't know about "user-friendly", but it would be safer to just
abort with an error if the column already exists.

If you're going to replace the contents of an existing column, you
should only do so if --overwrite is given.

Sounds reasonable.
It would be good if someone adds both error abort and --overwrite
flag in that script.

Markus

Markus Neteler wrote:

>> > (Hopefully) fixed in 6.4.develbranch and 7.trunk.
>>
>> Do I understand correctly that the patch ignores the column if the name
>> already exists ? Wouldn't it be more "user-friendly" to rename it ?
>
> I don't know about "user-friendly", but it would be safer to just
> abort with an error if the column already exists.
>
> If you're going to replace the contents of an existing column, you
> should only do so if --overwrite is given.

Sounds reasonable.
It would be good if someone adds both error abort and --overwrite
flag in that script.

AIUI, you already have the error abort for free, due to v.db.addcol
failing.

Handling --overwrite would be nice, but in the meantime the main thing
is not to try to circumvent the error by unconditionally deleting the
column. So, I have reverted your recent change.

--
Glynn Clements <glynn@gclements.plus.com>

On 30/05/08 22:03, Glynn Clements wrote:

Moritz Lennert wrote:

can you replicate this with Spearfish or the NC data set?
Then we could easier test things...

Markus

Hi Markus,

those are the steps to replicate the problem with NC dataset ( all is done
in a new mapset called sqlite):

db.connect driver=sqlite
database='$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite.db'
g.copy vect=urbanarea@PERMANENT,urbanarea
v.extract -t input=urbanarea output=urban_point type=point,centroid
v.db.addtable urban_point

v.db.join map=urbanarea column=cat otable=urban_point ocolumn=cat
..
DBMI-SQLite driver error:
Error in sqlite3_prepare():
duplicate column name: cat

ERROR: Error while executing: 'ALTER TABLE urban_point ADD COLUMN cat
      INTEGER
      '
ERROR: Cannot continue (problem adding column).
ERROR: Cannot continue.

Thanks for your help

Like this it was easy to replicated.
(Hopefully) fixed in 6.4.develbranch and 7.trunk.

Do I understand correctly that the patch ignores the column if the name already exists ? Wouldn't it be more "user-friendly" to rename it ?

I don't know about "user-friendly", but it would be safer to just
abort with an error if the column already exists.

If you're going to replace the contents of an existing column, you
should only do so if --overwrite is given.

I'm not speaking about replacing any content. I'm proposing that if you have a column cat in each table, then the second is renamed to something like cat2, so that you can have both in the resulting table.

Moritz

Moritz Lennert wrote:

>>>> those are the steps to replicate the problem with NC dataset ( all is done
>>>> in a new mapset called sqlite):
>>>>
>>>> db.connect driver=sqlite
>>>> database='$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite.db'
>>>> g.copy vect=urbanarea@PERMANENT,urbanarea
>>>> v.extract -t input=urbanarea output=urban_point type=point,centroid
>>>> v.db.addtable urban_point
>>>>
>>>> v.db.join map=urbanarea column=cat otable=urban_point ocolumn=cat
>>>> ..
>>>> DBMI-SQLite driver error:
>>>> Error in sqlite3_prepare():
>>>> duplicate column name: cat
>>>>
>>>> ERROR: Error while executing: 'ALTER TABLE urban_point ADD COLUMN cat
>>>> INTEGER
>>>> '
>>>> ERROR: Cannot continue (problem adding column).
>>>> ERROR: Cannot continue.
>>>>
>>> Like this it was easy to replicated.
>>> (Hopefully) fixed in 6.4.develbranch and 7.trunk.
>> Do I understand correctly that the patch ignores the column if the name
>> already exists ? Wouldn't it be more "user-friendly" to rename it ?
>
> I don't know about "user-friendly", but it would be safer to just
> abort with an error if the column already exists.
>
> If you're going to replace the contents of an existing column, you
> should only do so if --overwrite is given.

I'm not speaking about replacing any content. I'm proposing that if you
have a column cat in each table, then the second is renamed to something
like cat2, so that you can have both in the resulting table.

I think that it would be better to just force the user to pick another
name for the ocolumn= option. That way, the user will know what the
column is named.

Also, in a script, silently renaming the output column may result in
the script silently producing bogus results.

More generally, if there's a problem, you tell the user that there's a
problem, not pretend that there isn't a problem.

AFAICT, the only real flaw with the original script is that the
"duplicate column name: cat" error wasn't especially visible.

Other than that, everything happened as it should have done.

--
Glynn Clements <glynn@gclements.plus.com>

Moritz:

> I'm not speaking about replacing any content.
> I'm proposing that if you have a column cat in each
> table, then the second is renamed to something
> like cat2, so that you can have both in the resulting
> table.

Glynn:

I think that it would be better to just force the user to
pick another name for the ocolumn= option. That way, the user
will know what the column is named.

Also, in a script, silently renaming the output column may
result in the script silently producing bogus results.

see also vector v.overlay/main.c which does:

               if ( input == 0 )
                   db_append_string ( &col_defs, ", a_" );
               else
                   db_append_string ( &col_defs, ", b_" );

to rename all columns to show from which map they came from.
In that module the rename is always done so no "surprises".

This probably isn't a good fit for v.db.join, but FYI another data point.

Hamish