[GRASS-dev] winGRASS 6.4.svn: Attribute manager not opening

Hi,

when loading "roadsmajor" of NC on various windows machines (with
and without white space in the grassdata path, we get

Error:
'columns' is not recognized as an internal or external command,
operable program or batch file
[OK]

Then the attrib. manager opens but it remains empty.

Maybe here?

dbmgr/manager.py
...
       self.listOfCommands.append(('v.db.addcol',
                                    { 'map' : self.vectorName,
                                      'layer' : self.layer,
                                      'columns' : '%s %s' % (name, ctype) }
                                    ))

No idea..

Markus

On Tue, Sep 4, 2012 at 11:04 AM, Markus Neteler <neteler@osgeo.org> wrote:

Hi,

when loading "roadsmajor" of NC on various windows machines (with
and without white space in the grassdata path, we get

Error:
'columns' is not recognized as an internal or external command,
operable program or batch file
[OK]

Then the attrib. manager opens but it remains empty.

Maybe here?

dbmgr/manager.py
...
       self.listOfCommands.append(('v.db.addcol',
                                    { 'map' : self.vectorName,
                                      'layer' : self.layer,
                                      'columns' : '%s %s' % (name, ctype) }
                                    ))

No idea..

Markus

Problem occurs when calling this (line 194, dbmgr/manager.py in grass64)

ret = RunCommand('v.db.select',
                                 quiet = True,
                                 parent = self,
                                 flags = 'c',
                                 map = self.mapDBInfo.map,
                                 layer = layer,
                                 columns = ','.join(columns),
                                 fs = fs,
                                 where = where,
                                 stdout = outFile)

I recently added the fs parameter to be able to open data which
contain | character by changing the separator in preferences. On
Windows it is causing splitting the whole command into 2 parts -
before and after pipe. As a result 'columns' parameter (after pipe) is
treated like another command. As a temporal workaround you can set the
separator to something different in preferences (more characters are
accepted) - this should work immediately. I can also revert the
change.

On Linux the characters in parameters are escaped properly probably in
Popen but not on Windows. Anyone knows how to solve this properly?

Anna

On Tue, Sep 4, 2012 at 2:01 PM, Anna Kratochvílová
<kratochanna@gmail.com> wrote:
...

Problem occurs when calling this (line 194, dbmgr/manager.py in grass64)

...

I recently added the fs parameter to be able to open data which
contain | character by changing the separator in preferences. On
Windows it is causing splitting the whole command into 2 parts -
before and after pipe. As a result 'columns' parameter (after pipe) is
treated like another command. As a temporal workaround you can set the
separator to something different in preferences (more characters are
accepted) - this should work immediately. I can also revert the
change.

Before reverting this important fix, perhaps we manage to find a
solution.

On Linux the characters in parameters are escaped properly probably in
Popen but not on Windows. Anyone knows how to solve this properly?

I see some escape magic in lib/gis/spawn.c and also here
http://docs.python.org/release/3.1.5/library/subprocess.html

Dunno if that's useful at all...

Markus

Hi,

does this have to do with parsing the vector/$MAP/dbln file?

There was an unfortunate thing in the 6.x format spec of that file where
space was used as the delimiter and it was impossible to fix for spaces
in the path, since the result would always be ambiguous. In GRASS 7 I
changed the delim to '|', and AFAIR backported read-support for that back
to grass6. If that sounds like the issue let me know and I'll dig out
more details about it, and how to work around it so it works cleanly
in all branches in a backwarks-forwards compatible way.

Note that for the dbln file the C code looks for and substitutes for
  $GISDBASE/$LOCATION_NAME/$MAPSET/
on all platforms, they are simulated enviro variables and for file-
system portability should be kept in the above form instead of replacing
with a real path. (unless that's really what you want to do)

I recently added the fs parameter to be able to open data
which contain | character by changing the separator in
preferences.

sorry, this email thread got lost in the pile. where/what specifically
are you talking about there?

Hamish

[sorry for the top posting, broken linewrap, and missing authors, too
late to fight with yahoo tonight]

> when loading "roadsmajor" of NC on various windows
machines (with
> and without white space in the grassdata path, we get
>
> Error:
> 'columns' is not recognized as an internal or external
command,
> operable program or batch file
> [OK]
>
> Then the attrib. manager opens but it remains empty.
>
> Maybe here?
>
> dbmgr/manager.py
> ...
>
self.listOfCommands.append(('v.db.addcol',
>
               
     { 'map' :
self.vectorName,
>
               
       'layer' :
self.layer,
>
               
       'columns' : '%s %s' % (name,
ctype) }
>
               
     ))
>
> No idea..
>
> Markus

Problem occurs when calling this (line 194, dbmgr/manager.py
in grass64)

ret = RunCommand('v.db.select',
               
   quiet = True,
               
   parent = self,
               
   flags = 'c',
               
   map = self.mapDBInfo.map,
               
   layer = layer,
               
   columns = ','.join(columns),
               
   fs = fs,
               
   where = where,
               
   stdout = outFile)

I recently added the fs parameter to be able to open data
which contain | character by changing the separator in
preferences. On
Windows it is causing splitting the whole command into 2
parts -
before and after pipe. As a result 'columns' parameter
(after pipe) is
treated like another command. As a temporal workaround you
can set the
separator to something different in preferences (more
characters are
accepted) - this should work immediately. I can also
revert the
change.

On Linux the characters in parameters are escaped properly
probably in
Popen but not on Windows. Anyone knows how to solve this
properly?

Hi,

On Sun, Sep 9, 2012 at 1:15 PM, Hamish <hamish_b@yahoo.com> wrote:

Hi,

does this have to do with parsing the vector/$MAP/dbln file?

There was an unfortunate thing in the 6.x format spec of that file where
space was used as the delimiter and it was impossible to fix for spaces
in the path, since the result would always be ambiguous. In GRASS 7 I
changed the delim to '|', and AFAIR backported read-support for that back
to grass6. If that sounds like the issue let me know and I'll dig out
more details about it, and how to work around it so it works cleanly
in all branches in a backwarks-forwards compatible way.

Note that for the dbln file the C code looks for and substitutes for
  $GISDBASE/$LOCATION_NAME/$MAPSET/
on all platforms, they are simulated enviro variables and for file-
system portability should be kept in the above form instead of replacing
with a real path. (unless that's really what you want to do)

This doesn't seem related. I explained the problem in the mail above
but I can try to explain it more clearly if needed.

I recently added the fs parameter to be able to open data
which contain | character by changing the separator in
preferences.

sorry, this email thread got lost in the pile. where/what specifically
are you talking about there?

Here is the ticket:
http://trac.osgeo.org/grass/ticket/1633

Anna

Hamish

[sorry for the top posting, broken linewrap, and missing authors, too
late to fight with yahoo tonight]

> when loading "roadsmajor" of NC on various windows
machines (with
> and without white space in the grassdata path, we get
>
> Error:
> 'columns' is not recognized as an internal or external
command,
> operable program or batch file
> [OK]
>
> Then the attrib. manager opens but it remains empty.
>
> Maybe here?
>
> dbmgr/manager.py
> ...
>
self.listOfCommands.append(('v.db.addcol',
>

     { 'map' :
self.vectorName,
>

       'layer' :
self.layer,
>

       'columns' : '%s %s' % (name,
ctype) }
>

     ))
>
> No idea..
>
> Markus

Problem occurs when calling this (line 194, dbmgr/manager.py
in grass64)

ret = RunCommand('v.db.select',

   quiet = True,

   parent = self,

   flags = 'c',

   map = self.mapDBInfo.map,

   layer = layer,

   columns = ','.join(columns),

   fs = fs,

   where = where,

   stdout = outFile)

I recently added the fs parameter to be able to open data
which contain | character by changing the separator in
preferences. On
Windows it is causing splitting the whole command into 2
parts -
before and after pipe. As a result 'columns' parameter
(after pipe) is
treated like another command. As a temporal workaround you
can set the
separator to something different in preferences (more
characters are
accepted) - this should work immediately. I can also
revert the
change.

On Linux the characters in parameters are escaped properly
probably in
Popen but not on Windows. Anyone knows how to solve this
properly?

Anna wrote:

This doesn't seem related. I explained the problem in the
mail above but I can try to explain it more clearly if needed.

sorry, I reread you post and it was clear enough. the confusion
was all mine.

Here is the ticket:
http://trac.osgeo.org/grass/ticket/1633

it seems a bit strange that the contents of a varchar string
are able to break the GUI. maybe in this case it's possible to
work around it, but in general seems like a deeper problem,
vulnerable to sql injection style issues and the current parsing
method may need to be revisited.
  http://en.wikipedia.org/wiki/Sql_injection

(not that grass is in any way safe from buffer overflows and
sql or shell script injections, but we should try to fix these
where we notice them since they make the code more robust)

Hamish