[GRASS-dev] [GRASS GIS] #2136: Create standard options for map or file base name (prefix)

GRASS GIS GRASS GIS developer
1

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------
A lot of modules creates several map while the input is only one name and
the actual map names are combined from the name provided by user,
separator and some additional information. For example:

{{{
r.mystats out=elevation_stats

# produces:
elevation_stats_mean
elevation_stats_stddev
elevation_stats_variance
}}}

Example modules are r.texture, i.pca, some import tools and probably some
others. With the temporal data we can expect more and more modules
outputting something like `water_level_12`, `water_level_13`, ...
(correction: we do not expect, it is already happening).

Name provided by user is usually referred as ''prefix'' by older modules
or base name (''base_name'' or ''basename'') by newer modules. The other
possibilities are just a name such as ''output'' and using input name as a
base name for output.

I consider the last two possibilities as confusing. I don't like
''prefix'' because it is not a prefix. In my point of view we are
suffixing the analyses name or state description, not prefixing the name
of map to it. I prefer ''basename'' because it is what it is. Although it
is longer then ''prefix'' and there is an issue with different spellings.
I prefer ''basename'' over ''base_name'' because it can be used in both
text and as parameter although my spell checker does not like it.

The general way would be to let user choose the separator (e.g. by not
putting there any) but it is not practical. We are using now both the dot
(usually older modules) and underscore (usually newer modules). My choice
is the underscore because it works for both rasters and vectors and
moreover, it is more practical for files.

We need to create standard options for this and use it in the modules. It
will improve standardization in module interfaces, enable parser in adding
overwrite flag and it will provide a general way to GUI to generate the
form correctly.

See also [http://lists.osgeo.org/pipermail/grass-
dev/2013-November/066392.html GRASS-dev i.pca doesn't have the '--o'
flag].

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2136&gt;
GRASS GIS <http://grass.osgeo.org>

2

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by mmetz):

Replying to [ticket:2136 wenzeslaus]:
> A lot of modules creates several map while the input is only one name
and the actual map names are combined from the name provided by user,
separator and some additional information.

[...]

> We need to create standard options for this and use it in the modules.
It will improve standardization in module interfaces, enable parser in
adding overwrite flag and it will provide a general way to GUI to generate
the form correctly.

It seems that this is already implemented, see r.mapcalc [0] for an
example.

[0]
https://trac.osgeo.org/grass/browser/grass/trunk/raster/r.mapcalc/main.c#L114

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2136#comment:1&gt;
GRASS GIS <http://grass.osgeo.org>

3

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by wenzeslaus):

{{{
module->overwrite = 1;
}}}

[http://grass.osgeo.org/programming7/structGModule.html GModule] overwrite
solves only one of the problems. It adds the overwrite flag and even GUI
respects it this is great. But it does not standardize the option name
(key), label and description.

So, r58311 and [source:grass/trunk/raster/r.mapcalc/main.c#L114 L114] in
`r.mapcalc` solves the main problem for now but still leave here the
confusion (see [http://lists.osgeo.org/pipermail/grass-
dev/2013-November/066402.html GRASS-dev i.pca doesn't have the '--o'
flag]).

Moreover, documentation to the standard option would define if underscore,
dot or nothing is used to separate basename/prefix from the suffix/result
name. So, user would know what to expect and programmer what to implement.

Finally, I hope that in future, parser, GUI or both will use the
information from (or defined by) standard flag to provide some checking
(such as validity of name as map name) or some additional functionality
(such a visual warning in GUI if maps with the same basename/prefix
already exists).

Anyway, thank you for the fix for `i.pca` and `r.texture`.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2136#comment:2&gt;
GRASS GIS <http://grass.osgeo.org>

4

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by wenzeslaus):

One more name which is used for this thing: ''name prefix''. Coming form
G7:r.ros this time.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2136#comment:3&gt;
GRASS GIS <http://grass.osgeo.org>

5

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by zarch):

Replying to [comment:2 wenzeslaus]:
> Moreover, documentation to the standard option would
> define if underscore, dot or nothing is used to separate
> basename/prefix from the suffix/result name. So, user
> would know what to expect and programmer what to implement.

What do you think if we use two underscore (__) to join the basename with
the rest?

basename__01, basename__02, etc.

{{{
#!python
>>> import re
>>> m = re.match("^(?P<name>[a-zA-Z0-9]+)__(?P<id>\w+)$", "horizon__000")
>>> m.groups()
('horizon', '000')
>>> m.groupdict()
{'id': '000', 'name': 'pippo'}
>>> m = re.match("^(?P<name>[a-zA-Z0-9]+)__(?P<id>\w+)$", "tile__000_001")
>>> m.groups()
('tile', '000_001')
>>> m.groupdict()
{'name': 'tile', 'id': '000_001'}
}}}

So maybe in the future the GUI would be able to group maps following the
rule "basename__*" and visualize them in the wxGUI Animation or something
else...

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2136#comment:4&gt;
GRASS GIS <http://grass.osgeo.org>

6

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by zarch):

The list of the modules using the word "prefix" as name of the parameter
or in the parameter description is (module name => parameter name,
parameter description):

     - g.extension => prefix: Prefix where to install extension (ignored
when flag -s is given)
     - i.cca => output: Output raster map prefix name
     - i.landsat.acca => input_prefix: Example: 'B.' for B.1, B.2, ...
     - i.landsat.toar => input_prefix: Example: 'B.' for B.1, B.2, ...
     - i.landsat.toar => output_prefix: Example: 'B.toar.' generates
B.toar.1, B.toar.2, ...
     - i.pansharpen => output_prefix: Prefix for output raster maps
     - i.pca => output_prefix: A numerical suffix will be added for each
component map
     - i.tasscap => output_prefix: Prefix for output raster maps
     - i.topo.corr => output: Name (flag -i) or prefix for output raster
maps
     - m.nviz.script => name: Prefix of output images (default = NVIZ)
     - r.blend => output_prefix: Prefix for red, green and blue output
raster maps containing
     - r.rgb => output_prefix: Prefix for output raster maps (default:
input)
     - r.ros => output: Prefix for output raster maps (.base, .max,
.maxdir, .spotdist)
     - r.texture => prefix: Prefix for output raster map(s)
     - v.out.postgis => dsn: Starts with 'PG' prefix, eg. 'PG:dbname=grass'
     - v.rast.stats => column_prefix: Column prefix for new attribute
columns

Just for the record I attached the code to look for a word in all the
GRASS modules:

{{{
#!python
from __future__ import print_function
import sys
from grass.pygrass.modules.shortcuts import (display, database, general,
                                              imagery, miscellaneous,
                                              postscript, raster, raster3D,
                                              temporal, vector)

mods = [display, database, general, imagery, miscellaneous, postscript,
         raster, raster3D, temporal, vector]

SKIP = ['v.pack', 'v.parallel']

def look(modules, word, skip, file=sys.stdout):
     mlen = len(modules.__dir__())
     for i, module in enumerate(modules.__dir__()):
         try:
             md = "%s.%s" % (modules.prefix, module)
             print("%03d/%03d - %s" % (i + 1, mlen, md))
             if md in skip:
                 print("SKIPED: %s" % md)
             else:
                 m = getattr(modules, module)
                 for p in m.params_list:
                     if ((p.name is not None and word in p.name.lower()) or
                             (p.description is not None and
                              word in p.description.lower())):
                         print("%s => %s, %s" % (m.name, p.name,
p.description),
                               file=file)
                 del(m)
         except:
             print("Not able to read: %s.%s" % (modules.prefix, module),
                   file=file)
             #import ipdb; ipdb.set_trace()
     del(modules)

with open("prefix.txt", "w") as file:
     for m in mods:
         look(m, "prefix", skip=SKIP, file=file)

}}}

In this way I found out that pygrass modules interface crashed with these
two modules:

     - v.pack
     - v.parallel

and raise an exception trying to read:

     - g.parser
     - i.eb_h_sebal01
     - t.rast_accdetect
     - t.rast_accumulate
     - t.rast_aggregate
     - t.rast_aggregate_ds
     - t.rast_extract
     - t.rast_gapfill
     - t.rast_import
     - t.rast_mapcalc
     - t.rast_neighbors
     - t.rast3d_mapcalc
     - t.vect_extract
     - v.generalize

So I will look into it during the next days.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2136#comment:5&gt;
GRASS GIS <http://grass.osgeo.org>

7

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by annakrat):

Replying to [comment:4 zarch]:
> Replying to [comment:2 wenzeslaus]:
> > Moreover, documentation to the standard option would
> > define if underscore, dot or nothing is used to separate
> > basename/prefix from the suffix/result name. So, user
> > would know what to expect and programmer what to implement.
>
> What do you think if we use two underscore (`__`) to join the basename
with the rest?
>
> basename`__`01, basename`__`02, etc.
>
  {{{
  #!python
  >>> import re
  >>> m = re.match("^(?P<name>[a-zA-Z0-9]+)__(?P<id>\w+)$", "horizon__000")
  >>> m.groups()
  ('horizon', '000')
  >>> m.groupdict()
  {'id': '000', 'name': 'pippo'}
  >>> m = re.match("^(?P<name>[a-zA-Z0-9]+)__(?P<id>\w+)$",
"tile__000_001")
  >>> m.groups()
  ('tile', '000_001')
  >>> m.groupdict()
  {'name': 'tile', 'id': '000_001'}
  }}}
>
> So maybe in the future the GUI would be able to group maps following the
rule "basename`__`*" and visualize them in the wxGUI Animation or
something else...

I agree with double underscore.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2136#comment:6&gt;
GRASS GIS <http://grass.osgeo.org>

8

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by zarch):

Replying to [comment:5 zarch]:
> In this way I found out that pygrass modules
> interface crashed with these two modules:
>
> - v.pack
> - v.parallel
>
> and raise an exception trying to read:
>
> - g.parser
> - i.eb_h_sebal01
> - t.rast_accdetect
> - t.rast_accumulate
> - t.rast_aggregate
> - t.rast_aggregate_ds
> - t.rast_extract
> - t.rast_gapfill
> - t.rast_import
> - t.rast_mapcalc
> - t.rast_neighbors
> - t.rast3d_mapcalc
> - t.vect_extract
> - v.generalize
>
> So I will look into it during the next days.

Ok, fixed ([http://trac.osgeo.org/grass/changeset/60940 r60940],
[http://trac.osgeo.org/grass/changeset/60941 r60941]) and added a
testsuite ([http://trac.osgeo.org/grass/changeset/60942 r60942]) to read
the XML interface of all GRASS modules, now all modules are working except
for g.parser.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2136#comment:7&gt;
GRASS GIS <http://grass.osgeo.org>

9

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by wenzeslaus):

Replying to [comment:6 annakrat]:
> Replying to [comment:4 zarch]:
> > Replying to [comment:2 wenzeslaus]:
> > > Moreover, documentation to the standard option would
> > > define if underscore, dot or nothing is used to separate
> > > basename/prefix from the suffix/result name. So, user
> > > would know what to expect and programmer what to implement.
> >
> > What do you think if we use two underscore (`__`) to join the basename
with the rest?
> >
> > basename`__`01, basename`__`02, etc.
> >
> {{{
> #!python
> >>> import re
> >>> m = re.match("^(?P<name>[a-zA-Z0-9]+)__(?P<id>\w+)$",
"horizon__000")
> >>> m.groups()
> ('horizon', '000')
> >>> m.groupdict()
> {'id': '000', 'name': 'pippo'}
> >>> m = re.match("^(?P<name>[a-zA-Z0-9]+)__(?P<id>\w+)$",
"tile__000_001")
> >>> m.groups()
> ('tile', '000_001')
> >>> m.groupdict()
> {'name': 'tile', 'id': '000_001'}
> }}}
> >
> > So maybe in the future the GUI would be able to group maps following
the rule "basename`__`*" and visualize them in the wxGUI Animation or
something else...
>
> I agree with double underscore.

I'm not sure about double underscore. I don't like how it looks like. I'm
afraid if it will be understandable for users, for example when visualized
in some GUI, you don't know how the font will show it (with mono spaced
font, you can see two underscores even if they are connected with each
other, with other fonts, you don't know). Moreover, so systems can try to
interpret the double underscore too (see what Trac did to commit message
in r60944).

On the other hand, I see the advantages too. It is much harder or more
fragile to achieve the same with one underscore (or other character).

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2136#comment:8&gt;
GRASS GIS <http://grass.osgeo.org>

10

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by zarch):

Replying to [comment:8 wenzeslaus]:
> Replying to [comment:6 annakrat]:
> > Replying to [comment:4 zarch]:
> > > So maybe in the future the GUI would be able to group maps
> > > following the rule "basename`__`*" and visualize them in
> > > the wxGUI Animation or something else...
> >
> > I agree with double underscore.
>
> I'm not sure about double underscore. I don't like how it
> looks like. I'm afraid if it will be understandable for users,
> for example when visualized in some GUI, you don't know how the
> font will show it (with mono spaced font, you can see two
> underscores even if they are connected with each other, with
> other fonts, you don't know). Moreover, so systems can try
> to interpret the double underscore too (see what Trac did to
> commit message in r60944).
>
> On the other hand, I see the advantages too. It is much harder
> or more fragile to achieve the same with one underscore
> (or other character).

I see two options:
     - avoid to use any separator;
     - choose a "standard" separator for map names;

I think that define a standard separator could open new possibilities to
GRASS GUI/interaction. So I believe we should choose one.

Concerning which separator we should use, personally I've not strong
opinion about double underscore, my idea is that one underscore is quite
common in the raster/vector names so I don't think it is a good candidate
to be used as separator.

Other options could be:
     - basename:01, basename:02, etc.
     - basename::01, basename::02, etc.
     - basename(01), basename(02), etc.
     - basename[01], basename[02], etc.
     - basename{01}, basename{02}, etc.
     - basename»01, basename»02, etc.

That became with two ids, like the output of r.tile:
     - basename:00:00, basename:00:01, etc.
     - basename::00::00, basename::00::01, etc.
     - basename(00)(00), basename(00)(01), etc.
     - basename[00][00], basename[00][01], etc.
     - basename{00}{00}, basename{00}{01}, etc.
     - basename»00»00, basename»00»01, etc.

Using the parenthesis is maybe a little more difficult to parse than just
a separator and parenthesis are often interpreted by other program, see
how are rendered here!

Personally I like the use of a peculiar character like: », ■, or other
because user should never insert such characters manually and clearly mark
which maps are automatically generated my GRASS modules, and as far as I
know they have not particular meaning in other system like shell, trac,
etc..

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2136#comment:9&gt;
GRASS GIS <http://grass.osgeo.org>

11

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by annakrat):

Replying to [comment:9 zarch]:
> Replying to [comment:8 wenzeslaus]:
> > Replying to [comment:6 annakrat]:
> > > Replying to [comment:4 zarch]:
> > > > So maybe in the future the GUI would be able to group maps
> > > > following the rule "basename`__`*" and visualize them in
> > > > the wxGUI Animation or something else...
> > >
> > > I agree with double underscore.
> >
> > I'm not sure about double underscore. I don't like how it
> > looks like. I'm afraid if it will be understandable for users,
> > for example when visualized in some GUI, you don't know how the
> > font will show it (with mono spaced font, you can see two
> > underscores even if they are connected with each other, with
> > other fonts, you don't know). Moreover, so systems can try
> > to interpret the double underscore too (see what Trac did to
> > commit message in r60944).
> >
> > On the other hand, I see the advantages too. It is much harder
> > or more fragile to achieve the same with one underscore
> > (or other character).
>
>
> I see two options:
> - avoid to use any separator;
> - choose a "standard" separator for map names;
>
> I think that define a standard separator could open new possibilities to
GRASS GUI/interaction. So I believe we should choose one.
>
> Concerning which separator we should use, personally I've not strong
opinion about double underscore, my idea is that one underscore is quite
common in the raster/vector names so I don't think it is a good candidate
to be used as separator.
> Personally I like the use of a peculiar character like: », ■, or other
because user should never insert such characters manually and clearly mark
which maps are automatically generated my GRASS modules, and as far as I
know they have not particular meaning in other system like shell, trac,
etc..
>
>

Map names can't have this weird characters, check `G_legal_filename`. I
still think '`__`' is the best option.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2136#comment:10&gt;
GRASS GIS <http://grass.osgeo.org>

12

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by zarch):

Replying to [comment:10 annakrat]:
> Replying to [comment:9 zarch]:
> > Personally I like the use of a peculiar character like: », ■,
> > or other because user should never insert such characters
> > manually and clearly mark which maps are automatically generated
> > my GRASS modules, and as far as I know they have not particular
> > meaning in other system like shell, trac, etc..
> >
> >
>
> Map names can't have this weird characters, check `G_legal_filename`.
> I still think '`__`' is the best option.

Yes, I know... But we can always change what is considered legal and what
is not...

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2136#comment:11&gt;
GRASS GIS <http://grass.osgeo.org>

13

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by wenzeslaus):

Replying to [comment:11 zarch]:
> Replying to [comment:10 annakrat]:
> > Replying to [comment:9 zarch]:
> > > Personally I like the use of a peculiar character like: », ■,
> > > or other because user should never insert such characters
> > > manually and clearly mark which maps are automatically generated
> > > my GRASS modules, and as far as I know they have not particular
> > > meaning in other system like shell, trac, etc..
> > >
> > >
> >
> > Map names can't have this weird characters, check `G_legal_filename`.
> > I still think '`__`' is the best option.
>
> Yes, I know... But we can always change what is considered legal and
what is not...
>
I would not risk anything what is non-ASCII. Moreover, it might be a good
idea to avoid also all characters which has some meaning or must be
escaped/quoted in various command lines or SQL. These include
`(){}:.|$!&*`.

Perhaps one underscore is enough. For GUI, it would be a big improvement
to know that there was a basename.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2136#comment:12&gt;
GRASS GIS <http://grass.osgeo.org>

14

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by hcho):

I don't like those special characters (», ■,) because they are not easy to
type and we should not include non-ASCII characters in legal names.

Personally, I prefer to append a separator to basenames myself. What about
a GRASS variable like BASENAME_SEPARATOR? It can be empty or whatever we
want.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2136#comment:13&gt;
GRASS GIS <http://grass.osgeo.org>

15

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by zarch):

Replying to [comment:12 wenzeslaus]:
> Replying to [comment:11 zarch]:
> > Replying to [comment:10 annakrat]:
> > > Map names can't have this weird characters, check
`G_legal_filename`.
> > > I still think '`__`' is the best option.
> >
> > Yes, I know... But we can always change what is considered legal and
> > what is not...
> >
> I would not risk anything what is non-ASCII.

Just for the record this are ASCII characters » (175), ■ (254)...

> Moreover, it might be a good idea to avoid also all characters which
> has some meaning or must be escaped/quoted in various command lines
> or SQL. These include `(){}:.|$!&*`.

Works fine with SQL, and has not special meaning elsewhere, and so far it
is working quite well also in my bash scripts without problems.

{{{
$ sqlite3 /tmp/temp»db.sqlite
SQLite version 3.7.11 2012-03-20 11:35:50
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> CREATE TABLE my»table (cat INTEGER PRIMARY KEY, my»col INTEGER);
sqlite> INSERT INTO my»table (cat, my»col) VALUES (1, 1234);
sqlite> SELECT * FROM my»table;
1|1234
}}}

> Perhaps one underscore is enough.

I don't think so, a single underscore is too common, it is too prone to
casual errors.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2136#comment:14&gt;
GRASS GIS <http://grass.osgeo.org>

16

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by glynn):

Replying to [comment:12 wenzeslaus]:

> > Yes, I know... But we can always change what is considered legal and
what is not...

Sort of. We can't change which characters exist in various encodings, nor
can we change which characters are legal (at the OS level) in a filename.

> I would not risk anything what is non-ASCII. Moreover, it might be a
good idea to avoid also all characters which has some meaning or must be
escaped/quoted in various command lines or SQL. These include
`(){}:.|$!&*`.

Realistically, "characters which has some meaning" is anything except
alphanumerics and underscore. Everything else meaning to something or
another. Actually, even an underscore isn't 100% safe.

Even a dash (minus, ASCII 45 = 0x2d) creates issues for r.mapcalc: foo-bar
is parsed as the subtraction of bar from foo; if you want to use a map
named foo-bar (which is valid as per G_legal_filename), the name has to be
quoted in the r.mapcalc expression).

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2136#comment:15&gt;
GRASS GIS <http://grass.osgeo.org>

17

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by wenzeslaus):

Replying to [comment:14 zarch]:
> Replying to [comment:12 wenzeslaus]:
> > Replying to [comment:11 zarch]:
> > > Replying to [comment:10 annakrat]:
> > > > Map names can't have this weird characters, check
`G_legal_filename`.
> > > > I still think '`__`' is the best option.
> > >
> > > Yes, I know... But we can always change what is considered legal and
> > > what is not...
> > >
> > I would not risk anything what is non-ASCII.
>
> Just for the record this are ASCII characters » (175), ■ (254)...
>
I wish this to be true but it is not. They might be part of some
[http://en.wikipedia.org/wiki/Extended_ASCII extended ASCII]; I can see
them at [http://www.asciitable.com/ asciitable] but not at [http://www
.ascii-code.com/ ascii-code]. More importantly, they are not part of
widely supported (and thus safe) [http://en.wikipedia.org/wiki/ASCII
ASCII] which has only 128 printable and non-prinatble characters (to fit
into 7 bits). ASCII corresponds to first 128 characters of
[http://en.wikipedia.org/wiki/UTF-8 UTF-8].

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2136#comment:16&gt;
GRASS GIS <http://grass.osgeo.org>

18

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by zarch):

Replying to [comment:16 wenzeslaus]:
> Replying to [comment:14 zarch]:
> > Just for the record this are ASCII characters » (175), ■ (254)...
>
>
> I wish this to be true but it is not. They might be part of some
> [http://en.wikipedia.org/wiki/Extended_ASCII extended ASCII]; I can
> see them at [http://www.asciitable.com/ asciitable] but not at
> [http://www.ascii-code.com/ ascii-code]. More importantly, they
> are not part of widely supported (and thus safe)
> [http://en.wikipedia.org/wiki/ASCII ASCII] which has only
> 128 printable and non-prinatble characters (to fit into 7 bits).
> ASCII corresponds to first 128 characters of
> [http://en.wikipedia.org/wiki/UTF-8 UTF-8].

yes, you are right, thanks for clarification.

Replying to [comment:14 zarch]:
> Replying to [comment:16 wenzeslaus]:
> > Perhaps one underscore is enough.
>
> I don't think so, a single underscore is too common,
> it is too prone to casual errors.

For example, we decide to not write the dot in float number
(basename_000.05) as in raster maps generated by r.horizon using a single
underscore (basename_000_05), but in this way is not clear which one is
the basename which the id.

If in the future we will have to generate raster maps with two floating
points (basename_000.05_000.10), if we consider to not use the dot, it is
not readable (basename_000_05_000_10). Use two underscore
(basename__000_05__000_10) at least avoid these problems (but it is not
working in trac!).

So I don't see too many options...

Maybe we can define a GBASENAME_SEP variable in gis.h containing the
default separator, and as suggested by hco, in the future add the
possibility to set an environmental variable to customize the character or
the sequence of characters that will be consider as basename separator.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2136#comment:17&gt;
GRASS GIS <http://grass.osgeo.org>

19

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by lucadelu):

Replying to [comment:17 zarch]:

>
> Maybe we can define a GBASENAME_SEP variable in gis.h containing the
default separator, and as suggested by hco, in the future add the
possibility to set an environmental variable to customize the character or
the sequence of characters that will be consider as basename separator.

+1 for this solution

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2136#comment:18&gt;
GRASS GIS <http://grass.osgeo.org>

20

#2136: Create standard options for map or file base name (prefix)
-----------------------------------------+----------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
     Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Parser | Version: svn-releasebranch64
Keywords: base name, prefix, basename | Platform: All
      Cpu: All |
-----------------------------------------+----------------------------------

Comment(by zarch):

Replying to [comment:17 zarch]:
> Maybe we can define a GBASENAME_SEP variable in gis.h containing
> the default separator, and as suggested by hco, in the future
> add the possibility to set an environmental variable to customize
> the character or the sequence of characters that will be consider
> as basename separator.

OK, I wrote the following functions that you can be found
[https://github.com/zarch/grass_basename here]:

     - G_get_numb_of_decimals, return the number of decimal in a string;
     - G_double_to_str, return a formatted string substituting . with _ in
double number;
     - G_get_basename_sep, return the basename separator using the default
or the one find in the GRASS_BASENAME_SEP variable;
     - G_join_basename_strings, takes an array of strings and return a
joined string using the basename separator;
     - G_get_format_name, takes a basename string and a double number and
return a formatted string;

I'm using ``malloc`` and ``free`` instead of ``G_malloc`` and ``G_free``
just to make it easier to test outside GRASS

Do you have any comments?

Should I create a directory in trunk/lib/basename containing
[https://github.com/zarch/grass_basename/blob/master/basename.c
basename.c] and put the
[https://github.com/zarch/grass_basename/blob/master/basename.h
basename.h] in trunk/include/defs?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2136#comment:19&gt;
GRASS GIS <http://grass.osgeo.org>