[GRASS-dev] Re: [GRASS-SVN] r30420 - grass/trunk/scripts/v.db.renamecol

Markus Neteler pisze:

On Sun, Mar 2, 2008 at 2:02 PM, <svn_grass@osgeo.org> wrote:

Author: msieczka
Date: 2008-03-02 08:02:33 -0500 (Sun, 02 Mar 2008)
New Revision: 30420

Modified:
   grass/trunk/scripts/v.db.renamecol/v.db.renamecol
Log:
Fix: protect key column, allow renaming 'cat' when it isn't a key column.
Get rid of awk dependency.
Minor fixes.

are you sure that this is portable?

if [ "$driver" = "dbf" ] ; then
- NAMELEN=`echo "$newcol" | wc -c | awk '{print $1}'`
+ NAMELEN=`echo "$newcol" | wc -c`

I darkly remember that some "wc" programs insert odd spaces which
would break the script.

I haven't heard of it. Maybe you are right. Let's ask on the dev ML. Thoughts, Anybody?

Maciek

Maciej Sieczka wrote:

>> New Revision: 30420
>>
>> Modified:
>> grass/trunk/scripts/v.db.renamecol/v.db.renamecol
>> Log:
>> Fix: protect key column, allow renaming 'cat' when it isn't a key
>> column.
>> Get rid of awk dependency.
>> Minor fixes.

Markus:

> are you sure that this is portable?
>
> if [ "$driver" = "dbf" ] ; then
> - NAMELEN=`echo "$newcol" | wc -c | awk '{print $1}'`
> + NAMELEN=`echo "$newcol" | wc -c`
>
> I darkly remember that some "wc" programs insert odd spaces which
> would break the script.

I haven't heard of it. Maybe you are right. Let's ask on the dev ML.
Thoughts, Anybody?

'wc -c filename' will produce output like: '17 filename' (needing awk
or cut) while 'wc -c < filename' or 'cat filename | wc -c' will produce
output like: '17'. Maybe that is the dark memory?

For curiosity, what's so bad about using awk? speed? It cannot be
removed from all scripts* so why remove from any? [*] needed for FP
math (and less hard to find than bc), fancy stuff like v.in.mapgen, ...

#### setup temporary file
TMP="`g.tempfile pid=$$`"
if [ $? -ne 0 ] || [ -z "$TMP" ] ; then
- g.message -e "Unable to create temporary files"
+ g.message -e "Unable to create temporary files."
     exit 1
fi

IIUC the message standardization has been to not end single sentence
warnings and errors with a ".", only to do that with module
descriptions. (I don't know if I agree with that all the time, but a
lot of effort has gone in to making it that way)

if [ -z "$table" ] ; then
- g.message 'There is no table connected to this map! Cannot rename
any column.'
+ g.message -e "There is no table connected to input vector map!
Cannot rename any column."
    cleanup
    exit 1
fi

and r30418: v.db.dropcol
@@ -91,5 +80,5 @@
exitprocedure()
{
- g.message -e 'User break!'
+ g.message -e "User break!"
  cleanup
  exit 1

"!" is a special shell char and must be quoted in 'single quotes'.

GRASS> g.message -e "this will not work!, ok?"
bash: !,: event not found

Hamish

      ____________________________________________________________________________________
Never miss a thing. Make Yahoo your home page.
http://www.yahoo.com/r/hs

Hamish <hamish_b@yahoo.com> writes:

[...]

>>> I darkly remember that some "wc" programs insert odd spaces which
>>> would break the script.

>> I haven't heard of it. Maybe you are right. Let's ask on the dev ML.
>> Thoughts, Anybody?

> 'wc -c filename' will produce output like: '17 filename' (needing awk
> or cut) while 'wc -c < filename' or 'cat filename | wc -c' will
> produce output like: '17'. Maybe that is the dark memory?

--cut: http://opengroup.org/onlinepubs/007908799/xcu/wc.html--
    If no input file operands are specified, no name will be written and
    no blank characters preceding the pathname will be written.
--cut: http://opengroup.org/onlinepubs/007908799/xcu/wc.html--

[...]

> and r30418: v.db.dropcol
> @@ -91,5 +80,5 @@
> exitprocedure()
> {
> - g.message -e 'User break!'
> + g.message -e "User break!"
> cleanup
> exit 1

  In general, I'd recommend using single quotes for text strings,
  unless there's a reason to use double ones (such as when one
  needs to substitute a variable within the text.)

> "!" is a special shell char and must be quoted in 'single quotes'.

> GRASS> g.message -e "this will not work!, ok?"
> bash: !,: event not found

  It is not quite so. One may think of `!' as of a ``Bash special
  char''. Compare, e. g.:

bash $ echo "hello, world!"
bash: !": event not found
bash $

  and

dash $ echo "hello, world!"
hello, world!
dash $

  Moreover, it's an optional feature even in Bash (check the
  ``History expansion'' section in bash(1)):

bash $ set +H
bash $ echo "hello, world!"
hello, world!
bash $

  and it is /not/ enabled by default when in a non-interactive
  mode:

bash $ bash -c 'echo "hello, world!"'
hello, world!
bash $

  In particular, this feature never gets enabled by default for
  Shell scripts.

Ivan Shmakov wrote:

bash $ echo "hello, world!"
bash: !": event not found
bash $

...

  Moreover, it's an optional feature even in Bash (check the
  ``History expansion'' section in bash(1)):

...

  In particular, this feature never gets enabled by default for
  Shell scripts.

apparently "never" doesn't include Debian, where (IIRC) I had seen the
"!" error oddly happening mid-script, before all the quoting fixes.

Hamish

      ____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

>> Markus wrote:
> Maciek wrote:
Hamish pisze:

are you sure that this is portable?

if [ "$driver" = "dbf" ] ; then
- NAMELEN=`echo "$newcol" | wc -c | awk '{print $1}'`
+ NAMELEN=`echo "$newcol" | wc -c`

I darkly remember that some "wc" programs insert odd spaces which
would break the script.

I haven't heard of it. Maybe you are right. Let's ask on the dev ML. Thoughts, Anybody?

'wc -c filename' will produce output like: '17 filename' (needing awk
or cut) while 'wc -c < filename' or 'cat filename | wc -c' will produce
output like: '17'. Maybe that is the dark memory?

That could explain the " awk '{print $1}' ". Markus?

For curiosity, what's so bad about using awk?

Invoking the allmighty awk just to do what cut is designed for seems an overkill to me. Awk is not needed for anything in v.db.renamecol. Cut suffices. Keep it simple, the UNIX way of specialized tools and stuff.

speed?

Actually cut itself seems slightly slower than awk for this sort of task:

$ x=0; time while [ $x -lt 1000 ]; do echo "raz dwa" | cut -d" " -f2 > /dev/null ; x=`expr $x + 1`; done

on my machine is about 7% slower than

$ x=0; time while [ $x -lt 1000 ]; do echo "raz dwa" | awk '{print $2}' > /dev/null ; x=`expr $x + 1`; done

On the other hand, after replacing all awk with cut in the script, removing the test for awk's presence:

#### check if we have awk
if [ ! -x "`which awk`" ] ; then
     g.message -e "awk required, please install awk or gawk first"
     exit 1
fi

resulted in a speed boost. In the end, v.db.renamcol without awk is as fast as it was with awk. I you want to test, try the following:

$ v.random pts n=10 --o; v.db.addtable pts col="huha1 integer
$ x=0; time while [ $x -lt 50 ]; do v.db.renamecol map=pts col=huha1,huha2; v.db.renamecol map=pts col=huha2,huha1; x=`expr $x + 1`; done

using v.db.renamecol as it was with awk, and with all of awk removed or replaced by cut.

It cannot be removed from all scripts* so why remove from any? [*] needed for FP
math (and less hard to find than bc), fancy stuff like v.in.mapgen, ...

I know it is used widely in GRASS scripts. Yet there is no reason to require it in a script which does not need it (for instance unless there is FP maths in v.db.renamecol).

#### setup temporary file
TMP="`g.tempfile pid=$$`"
if [ $? -ne 0 ] || [ -z "$TMP" ] ; then
- g.message -e "Unable to create temporary files" + g.message -e "Unable to create temporary files."
     exit 1
fi

IIUC the message standardization has been to not end single sentence
warnings and errors with a ".", only to do that with module
descriptions. (I don't know if I agree with that all the time, but a
lot of effort has gone in to making it that way)

A sentence without a period? Strange and not grammatical. What would be the rationalle?

Even if it was agreed, I can't find it documented in SUBMITTING, SUBMITTING_SCRIPTS nor g.message man. Actually, the SUBMITTING_SCRIPTS examples indirectly suggest that both with and without period are allowed (see section 11).

The WIKI confirms the latter: "Be consistent with periods. Either end all phrases with a period or none", "Punctuated events, such as errors, deserve a period" [1]. This is contrary to what you suggest in your email now.

FWIW, the original v.db.renamecol used both conventions - with and without ".". I added periods to all sentences - correct in terms of grammar and GRASS standards I guess.

if [ -z "$table" ] ; then
- g.message 'There is no table connected to this map! Cannot rename
any column.'
+ g.message -e "There is no table connected to input vector map!
Cannot rename any column."
    cleanup
    exit 1
fi

and r30418: v.db.dropcol
@@ -91,5 +80,5 @@
exitprocedure()
{
- g.message -e 'User break!'
+ g.message -e "User break!"
  cleanup
  exit 1

"!" is a special shell char and must be quoted in 'single quotes'.

GRASS> g.message -e "this will not work!, ok?"
bash: !,: event not found

My bad. Very sorry. Correcting it right away.

Maciek

[1]http://grass.gdf-hannover.de/wiki/Development_Specs#Standard_messages_sandbox

Hamish pisze:

Ivan Shmakov wrote:

bash $ echo "hello, world!" bash: !": event not found
bash $

...

  Moreover, it's an optional feature even in Bash (check the
  ``History expansion'' section in bash(1)):

...

  In particular, this feature never gets enabled by default for
  Shell scripts.

apparently "never" doesn't include Debian, where (IIRC) I had seen the
"!" error oddly happening mid-script, before all the quoting fixes.

On Ubuntu Gutsy it's like Ivan says - echo "!" is ok in scripts, fails on command line.

I'm about to switch to Debian. I'll check this if not forget to.

Maciek

Maciej Sieczka wrote:

On Ubuntu Gutsy it's like Ivan says - echo "!" is ok in scripts,
fails on command line.

I'm about to switch to Debian. I'll check this if not forget to.

It could very well be user error & bad memory on my behalf. But
regardless it can only be a good thing to '' quote !s.

H

      ____________________________________________________________________________________
Looking for last minute shopping deals?
Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping

Hamish pisze:

Maciej Sieczka wrote:

On Ubuntu Gutsy it's like Ivan says - echo "!" is ok in scripts,
fails on command line.

I'm about to switch to Debian. I'll check this if not forget to.

It could very well be user error & bad memory on my behalf. But
regardless it can only be a good thing to '' quote !s.

Fully aggreed. I'll try not to forget to stick to that.

Maciek

On Mon, Mar 3, 2008 at 11:53 AM, Maciej Sieczka <tutey@o2.pl> wrote:

>> Markus wrote:
  > Maciek wrote:
Hamish pisze:

>>> are you sure that this is portable?
>>>
>>> if [ "$driver" = "dbf" ] ; then
>>> - NAMELEN=`echo "$newcol" | wc -c | awk '{print $1}'`
>>> + NAMELEN=`echo "$newcol" | wc -c`
>>>
>>> I darkly remember that some "wc" programs insert odd spaces which
>>> would break the script.

>> I haven't heard of it. Maybe you are right. Let's ask on the dev ML.
>> Thoughts, Anybody?

> 'wc -c filename' will produce output like: '17 filename' (needing awk
> or cut) while 'wc -c < filename' or 'cat filename | wc -c' will produce
> output like: '17'. Maybe that is the dark memory?

That could explain the " awk '{print $1}' ". Markus?

Possibly yes. As said: darkly remembered...

> For curiosity, what's so bad about using awk?

Invoking the allmighty awk just to do what cut is designed for seems an
overkill to me. Awk is not needed for anything in v.db.renamecol. Cut
suffices. Keep it simple, the UNIX way of specialized tools and stuff.

> speed?

Actually cut itself seems slightly slower than awk for this sort of task:

$ x=0; time while [ $x -lt 1000 ]; do echo "raz dwa" | cut -d" " -f2 >
/dev/null ; x=`expr $x + 1`; done

on my machine is about 7% slower than

$ x=0; time while [ $x -lt 1000 ]; do echo "raz dwa" | awk '{print $2}'
  > /dev/null ; x=`expr $x + 1`; done

On the other hand, after replacing all awk with cut in the script,
removing the test for awk's presence:

#### check if we have awk
if [ ! -x "`which awk`" ] ; then
     g.message -e "awk required, please install awk or gawk first"
     exit 1
fi

resulted in a speed boost. In the end, v.db.renamcol without awk is as
fast as it was with awk.

At this point I don't see why do this effort.
There is the risk to introduce new bugs (as seen here) and
there is no point in saving 7% of 10 nanoseconds. Since
awk is needed in GRASS it can also be used here.

Why not using efforts for more important things?

Markus

On Mon, Mar 3, 2008 at 11:53 AM, Maciej Sieczka <tutey@o2.pl> wrote:

>> Markus wrote:
  > Maciek wrote:
Hamish pisze:

...

> #### setup temporary file
> TMP="`g.tempfile pid=$$`"
> if [ $? -ne 0 ] || [ -z "$TMP" ] ; then
> - g.message -e "Unable to create temporary files"
> + g.message -e "Unable to create temporary files."
> exit 1
> fi
>
> IIUC the message standardization has been to not end single sentence
> warnings and errors with a ".", only to do that with module
> descriptions. (I don't know if I agree with that all the time, but a
> lot of effort has gone in to making it that way)

A sentence without a period? Strange and not grammatical. What would be
the rationalle?

...

[1]http://grass.gdf-hannover.de/wiki/Development_Specs#Standard_messages_sandbox

The key point is not not break again all translations.

Markus

Hi,

2008/3/3, Markus Neteler <neteler@osgeo.org>:
Maciek:

> > IIUC the message standardization has been to not end single sentence
> > warnings and errors with a ".", only to do that with module
> > descriptions. (I don't know if I agree with that all the time, but a
> > lot of effort has gone in to making it that way)
>
> A sentence without a period? Strange and not grammatical. What would be
> the rationalle?
> [1]http://grass.gdf-hannover.de/wiki/Development_Specs#Standard_messages_sandbox

Markus:

The key point is not not break again all translations.

In the last decade most of single sentences w/e were changed to be
without '.' at the end of the sentence so I would prefer not to break
this 'rule'. Anyway it should be documented somewhere (SUBMITTING*)

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa *

Markus Neteler pisze:

On Mon, Mar 3, 2008 at 11:53 AM, Maciej Sieczka <tutey@o2.pl> wrote:

Markus wrote:

Maciek wrote:

Hamish pisze:

'wc -c filename' will produce output like: '17 filename' (needing awk
or cut) while 'wc -c < filename' or 'cat filename | wc -c' will produce
output like: '17'. Maybe that is the dark memory?

That could explain the " awk '{print $1}' ". Markus?

Possibly yes. As said: darkly remembered...

In the end, v.db.renamcol without awk is as
fast as it was with awk.

At this point I don't see why do this effort.
There is the risk to introduce new bugs

Well yes there is as always.

(as seen here)

My awk to cut switch didn't introduce any bugs. No any new bugs were introduced at all. The only controversial change was:

- g.message -e 'User break!'
+ g.message -e "User break!"

which could (as Hamish says) but unlikely (as Ivan says) cause shell to expand the exclamation mark. Anyway - I reverted that, as single quotes are 100% safe. Glad Hamish pointed it out to me and sorry again.

Let me remind that the main point of my patches for v.db.renamecol and v.db.dropcol was to correct a bug that the key column was always assumed to be "cat". Fixed that. Other changes were BTW.

> and there is no point in saving 7% of 10 nanoseconds.

There is no speed gain at all.

Since awk is needed in GRASS it can also be used here.
Why not using efforts for more important things?

Like I said, awk to cut change that was just done BTW fixing bugs.

Maciek

Markus Neteler pisze:

On Mon, Mar 3, 2008 at 11:53 AM, Maciej Sieczka <tutey@o2.pl> wrote:

A sentence without a period? Strange and not grammatical. What would be
the rationalle?

[1]http://grass.gdf-hannover.de/wiki/Development_Specs#Standard_messages_sandbox

The key point is not not break again all translations.

Did my adding periods break any translations? Should I delete theses periods? Sorry if a stupid question.

Maciek

Hi,

2008/3/3, Maciej Sieczka <tutey@o2.pl>:

>> A sentence without a period? Strange and not grammatical. What would be
>> the rationalle?

>> [1]http://grass.gdf-hannover.de/wiki/Development_Specs#Standard_messages_sandbox

> The key point is not not break again all translations.

Did my adding periods break any translations? Should I delete theses
periods? Sorry if a stupid question.

yes, it can turn translated messages to fuzzy. I would prefer to put
'.' at the end of single sentence message too, but since last message
standardization effort follows rule 'single sentence message ->
without '.') I would prefer to continue in this way and to change
'rules' every year;-) Anyway it should be documented somewhere.

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa *

Martin Landa pisze:

2008/3/3, Maciej Sieczka <tutey@o2.pl>:

Did my adding periods break any translations? Should I delete theses
periods? Sorry if a stupid question.

yes, it can turn translated messages to fuzzy. I would prefer to put
'.' at the end of single sentence message too, but since last message
standardization effort follows rule 'single sentence message ->
without '.') I would prefer to continue in this way and to change
'rules' every year;-)

OK. I'm deleting them. Though a sentence with no period is crippled. Reader might suppose the message is incomplete or broken. Could this be changed in GRASS 7?

Anyway it should be documented somewhere.

Please. As you can see I don't know how the translation mechanism works, so I better don't do it myself.

Maciek

Maciej Sieczka pisze:

Martin Landa pisze:

2008/3/3, Maciej Sieczka <tutey@o2.pl>:

Did my adding periods break any translations? Should I delete theses
periods? Sorry if a stupid question.

yes, it can turn translated messages to fuzzy. I would prefer to put
'.' at the end of single sentence message too, but since last message
standardization effort follows rule 'single sentence message ->
without '.') I would prefer to continue in this way and to change
'rules' every year;-)

OK. I'm deleting them.

Oh, and what about other punctuation symbols? Are !?,;: within/at the end of the message? [1] does not explain.

[1]http://grass.gdf-hannover.de/wiki/Development_Specs#Message_standardization

Maciek

Maciek,

2008/3/3, Maciej Sieczka <tutey@o2.pl>:

>> Did my adding periods break any translations? Should I delete theses
>> periods? Sorry if a stupid question.

> yes, it can turn translated messages to fuzzy. I would prefer to put
> '.' at the end of single sentence message too, but since last message
> standardization effort follows rule 'single sentence message ->
> without '.') I would prefer to continue in this way and to change
> 'rules' every year;-)

OK. I'm deleting them. Though a sentence with no period is crippled.
Reader might suppose the message is incomplete or broken. Could this be
changed in GRASS 7?

I would really avoid it, GRASS 7 should be focused on other things
than changing all messages which have been changed in the last decade
back;-) Look at changesets with log ~ "Message standardization" from
the last few months, now it is too late to change this rule. I can
repeat that I would prefer to put '.' at the end of each (even single
sentence) message, but it would be really waste of energy to change
all messages back. Carlos?

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa *

Maciej:
[thanks for the cat != key fix; not mentioned as it was all fine...]

> resulted in a speed boost. In the end, v.db.renamcol without
> awk is as fast as it was with awk.

Markus:

At this point I don't see why do this effort.
There is the risk to introduce new bugs (as seen here) and
there is no point in saving 7% of 10 nanoseconds. Since
awk is needed in GRASS it can also be used here.

Why not using efforts for more important things?

just a thought on the other side of the coin: it is nice to have
reference code to copy from once we all figure out best practices. I
think the issue is similar to removing redundant `cat`s. maybe it's
pedantic but it makes us better next time we have to write a script--
as long as it is done for self education and not for useless 'OMG!
wasted 2ns!' crusades.

Hamish

      ____________________________________________________________________________________
Looking for last minute shopping deals?
Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping

Maciek:

Oh, and what about other punctuation symbols? Are !?,;: within/at the
end of the message? [1] does not explain.

[1]http://grass.gdf-hannover.de/wiki/Development_Specs#Message_standardization

the g.message help page tries to explain.
I do not know the full list of chars which must be with 'single'
quotes.

Hamish

      ____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

On Mon, Mar 3, 2008 at 5:29 PM, Maciej Sieczka <tutey@o2.pl> wrote:

Martin Landa pisze:

> 2008/3/3, Maciej Sieczka <tutey@o2.pl>:

>> Did my adding periods break any translations? Should I delete theses
>> periods? Sorry if a stupid question.

> yes, it can turn translated messages to fuzzy.

It will most likely do since the messages are different now from
the previous version.
We should be really careful with the messages. Once someone has
translated 5000 messages s/he will be "happy" to do it again just for the
introduction of a period.

> I would prefer to put
> '.' at the end of single sentence message too, but since last message
> standardization effort follows rule 'single sentence message ->
> without '.') I would prefer to continue in this way and to change
> 'rules' every year;-)

OK. I'm deleting them. Though a sentence with no period is crippled.

Well, it's not literature...

Reader might suppose the message is incomplete or broken. Could this be
changed in GRASS 7?

I don't think so - too much wasted effort.

> Anyway it should be documented somewhere.

Please. As you can see I don't know how the translation mechanism works,
so I better don't do it myself.

See here how it works:
http://trac.osgeo.org/grass/browser/grass/trunk/locale/README

It's fairly easy.

Markus