[GRASS-dev] g.search.module: shell style output

GRASS GIS GRASS GIS developer
, ,
1

Hi Jachym,

the g.search.module -g flag (shell style output) outputs just names. Do you have a particular reason for it? My use case is something like that:

g.search.modules keyword=“support” -g | sed -e “s/|[^|]*$//g” | sed -e “s/|/\t/g”

with the following desired output (name + keywords, description removed by sed):

g.version general,support,citing,copyright,version,license
t.support temporal,metadata,time
r.support raster,metadata
r.support.stats raster,statistics
r.out.gdal raster,export
v.out.ogr vector,export,OGR
r3.support raster3d,metadata,voxel
g.findetc general,map management,scripts
v.external vector,import,external,OGR,PostGIS
g.message general,support,scripts
g.tempfile general,support,scripts
v.support vector,metadata
r.external raster,import,external

I can actually see that outputting just module names can be advantageous in some cases. But I want to get something like, so I can throw sed and grep on it:

v.support|vector,metadata|Updates vector map metadata.

If we permit change of the interface, I think -g could do the output above. This would make the -g output more like the others: same information as by default and with -j, so we can even consider it fixing a bug.

The current output with -g can be generated with some other flag. -n* for “names only” perhaps?

Best,

Vaclav

2

Hi,

no special reason for not listing the module description too, just did not came to my mind

Just do it [1]

J

[1] https://www.youtube.com/watch?v=ZXsQAXx_ao0

čt 18. 8. 2016 v 20:32 odesílatel Vaclav Petras <wenzeslaus@gmail.com> napsal:

Hi Jachym,

the g.search.module -g flag (shell style output) outputs just names. Do you have a particular reason for it? My use case is something like that:

g.search.modules keyword=“support” -g | sed -e “s/|[^|]*$//g” | sed -e “s/|/\t/g”

with the following desired output (name + keywords, description removed by sed):

g.version general,support,citing,copyright,version,license
t.support temporal,metadata,time
r.support raster,metadata
r.support.stats raster,statistics
r.out.gdal raster,export
v.out.ogr vector,export,OGR
r3.support raster3d,metadata,voxel
g.findetc general,map management,scripts
v.external vector,import,external,OGR,PostGIS
g.message general,support,scripts
g.tempfile general,support,scripts
v.support vector,metadata
r.external raster,import,external

I can actually see that outputting just module names can be advantageous in some cases. But I want to get something like, so I can throw sed and grep on it:

v.support|vector,metadata|Updates vector map metadata.

If we permit change of the interface, I think -g could do the output above. This would make the -g output more like the others: same information as by default and with -j, so we can even consider it fixing a bug.

The current output with -g can be generated with some other flag. -n* for “names only” perhaps?

Best,

Vaclav

3

On Fri, Aug 19, 2016 at 2:15 AM, Jachym Cepicky <jachym.cepicky@gmail.com>
wrote:

Hi,

no special reason for not listing the module description too, just did not
came to my mind

Thanks. Good to know.

Just do it [1]

While using the modified version, I actually realized that "shell script
style" usually produces key-value pairs which can which can be evaluated by
shell's eval or grass.script.parse_command. Not all modules comply with
this, e.g. `g.extension -g` produces multiple key-values with same keys and
order matters, so this must be parsed in a special way. The result is
actually exactly the information g.search.modules is producing:

$ g.extension -g
...
name=v.habitat.dem
description=Calculates DEM derived characteristics of habitats.
keywords=vector,raster,terrain,statistics,sun,zonal statistics
name=v.in.gbif
description=importing of GBIF species distribution data
keywords=vector,geometry

`g.extension -l` produces list of modules in the same way as currently
`g.search.modules -g` produces:

$ g.extension -l

v.habitat.dem
v.in.gbif

As a result, I don't know what to do with -g, at this point I would just
replace the letter by -n (names only) or -s (short output with names only)
and add -t for table output (that's in the attached patch). -g can go to
renamed options for compatibility reasons for now.

For the future, we should try to keep in mind that g.extension and
g.search.module should have unified interfaces and/or outputs. And more
generally, we should define what -g "shell script style" means.

J

[1] https://www.youtube.com/watch?v=ZXsQAXx_ao0

čt 18. 8. 2016 v 20:32 odesílatel Vaclav Petras <wenzeslaus@gmail.com>
napsal:

Hi Jachym,

the g.search.module -g flag (shell style output) outputs just names. Do
you have a particular reason for it? My use case is something like that:

g.search.modules keyword="support" -g | sed -e "s/|[^|]*$//g" | sed -e
"s/|/\t/g"

with the following desired output (name + keywords, description removed
by sed):

g.version general,support,citing,copyright,version,license
t.support temporal,metadata,time
r.support raster,metadata
r.support.stats raster,statistics
r.out.gdal raster,export
v.out.ogr vector,export,OGR
r3.support raster3d,metadata,voxel
g.findetc general,map management,scripts
v.external vector,import,external,OGR,PostGIS
g.message general,support,scripts
g.tempfile general,support,scripts
v.support vector,metadata
r.external raster,import,external

I can actually see that outputting just module names can be advantageous
in some cases. But I want to get something like, so I can throw sed and
grep on it:

v.support|vector,metadata|Updates vector map metadata.

If we permit change of the interface, I think -g could do the output
above. This would make the -g output more like the others: same information
as by default and with -j, so we can even consider it fixing a bug.

The current output with -g can be generated with some other flag. -n* for
"names only" perhaps?

Best,
Vaclav

* https://lists.osgeo.org/pipermail/grass-dev/2016-August/081556.html

(attachments)

g_search_modules_table.diff (1.85 KB)

4

Vasku,

IIRC, “-g” flag was introduced first time in 2007 for g.region module and only reason for taking “g” letter was, that “s” for “shell script” was taken (or some similar reason). AFAIK there was never defined official GRASS rule for using -g flag for parsable output as well as how the output should look like - people simple started to adopt it (like usually in GRASS).

Putting rule to developer guidelines for parsable output as well as promoting the “-g” flag for all modules would certainly be an option

Jachym

po 22. 8. 2016 v 4:39 odesílatel Vaclav Petras <wenzeslaus@gmail.com> napsal:

···

On Fri, Aug 19, 2016 at 2:15 AM, Jachym Cepicky <jachym.cepicky@gmail.com> wrote:

Hi,

no special reason for not listing the module description too, just did not came to my mind

Thanks. Good to know.

Just do it [1]

While using the modified version, I actually realized that “shell script style” usually produces key-value pairs which can which can be evaluated by shell’s eval or grass.script.parse_command. Not all modules comply with this, e.g. g.extension -g produces multiple key-values with same keys and order matters, so this must be parsed in a special way. The result is actually exactly the information g.search.modules is producing:

$ g.extension -g

…
name=v.habitat.dem
description=Calculates DEM derived characteristics of habitats.
keywords=vector,raster,terrain,statistics,sun,zonal statistics
name=v.in.gbif
description=importing of GBIF species distribution data
keywords=vector,geometry

g.extension -l produces list of modules in the same way as currently g.search.modules -g produces:

$ g.extension -l

v.habitat.dem
v.in.gbif

As a result, I don’t know what to do with -g, at this point I would just replace the letter by -n (names only) or -s (short output with names only) and add -t for table output (that’s in the attached patch). -g can go to renamed options for compatibility reasons for now.

For the future, we should try to keep in mind that g.extension and g.search.module should have unified interfaces and/or outputs. And more generally, we should define what -g “shell script style” means.

J

[1] https://www.youtube.com/watch?v=ZXsQAXx_ao0

čt 18. 8. 2016 v 20:32 odesílatel Vaclav Petras <wenzeslaus@gmail.com> napsal:

Hi Jachym,

the g.search.module -g flag (shell style output) outputs just names. Do you have a particular reason for it? My use case is something like that:

g.search.modules keyword=“support” -g | sed -e “s/|[^|]*$//g” | sed -e “s/|/\t/g”

with the following desired output (name + keywords, description removed by sed):

g.version general,support,citing,copyright,version,license
t.support temporal,metadata,time
r.support raster,metadata
r.support.stats raster,statistics
r.out.gdal raster,export
v.out.ogr vector,export,OGR
r3.support raster3d,metadata,voxel
g.findetc general,map management,scripts
v.external vector,import,external,OGR,PostGIS
g.message general,support,scripts
g.tempfile general,support,scripts
v.support vector,metadata
r.external raster,import,external

I can actually see that outputting just module names can be advantageous in some cases. But I want to get something like, so I can throw sed and grep on it:

v.support|vector,metadata|Updates vector map metadata.

If we permit change of the interface, I think -g could do the output above. This would make the -g output more like the others: same information as by default and with -j, so we can even consider it fixing a bug.

The current output with -g can be generated with some other flag. -n* for “names only” perhaps?

Best,

Vaclav

5

Hi,

As a side note:

The shell script output of v.db.connect is not “parsable” like in g.region (n=…) either.

However, it is well formatted as a kind of table output…

Cheers

Stefan

···

Vasku,

IIRC, “-g” flag was introduced first time in 2007 for g.region module and only reason for taking “g” letter was, that “s” for “shell script” was taken (or some similar reason). AFAIK there was never defined official GRASS rule for using -g flag for parsable output as well as how the output should look like - people simple started to adopt it (like usually in GRASS).

Putting rule to developer guidelines for parsable output as well as promoting the “-g” flag for all modules would certainly be an option

Jachym

po 22. 8. 2016 v 4:39 odesílatel Vaclav Petras <wenzeslaus@gmail.com> napsal:

On Fri, Aug 19, 2016 at 2:15 AM, Jachym Cepicky <jachym.cepicky@gmail.com> wrote:

Hi,

no special reason for not listing the module description too, just did not came to my mind

Thanks. Good to know.

Just do it [1]

While using the modified version, I actually realized that “shell script style” usually produces key-value pairs which can which can be evaluated by shell’s eval or grass.script.parse_command. Not all modules comply with this, e.g. g.extension -g produces multiple key-values with same keys and order matters, so this must be parsed in a special way. The result is actually exactly the information g.search.modules is producing:

$ g.extension -g

…
name=v.habitat.dem
description=Calculates DEM derived characteristics of habitats.
keywords=vector,raster,terrain,statistics,sun,zonal statistics
name=v.in.gbif
description=importing of GBIF species distribution data
keywords=vector,geometry

g.extension -l produces list of modules in the same way as currently g.search.modules -g produces:

$ g.extension -l

v.habitat.dem
v.in.gbif

As a result, I don’t know what to do with -g, at this point I would just replace the letter by -n (names only) or -s (short output with names only) and add -t for table output (that’s in the attached patch). -g can go to renamed options for compatibility reasons for now.

For the future, we should try to keep in mind that g.extension and g.search.module should have unified interfaces and/or outputs. And more generally, we should define what -g “shell script style” means.

J

[1] https://www.youtube.com/watch?v=ZXsQAXx_ao0

čt 18. 8. 2016 v 20:32 odesílatel Vaclav Petras <wenzeslaus@gmail.com> napsal:

Hi Jachym,

the g.search.module -g flag (shell style output) outputs just names. Do you have a particular reason for it? My use case is something like that:

g.search.modules keyword=“support” -g | sed -e “s/|[^|]*$//g” | sed -e “s/|/\t/g”

with the following desired output (name + keywords, description removed by sed):

g.version general,support,citing,copyright,version,license
t.support temporal,metadata,time
r.support raster,metadata
r.support.stats raster,statistics
r.out.gdal raster,export
v.out.ogr vector,export,OGR
r3.support raster3d,metadata,voxel
g.findetc general,map management,scripts
v.external vector,import,external,OGR,PostGIS
g.message general,support,scripts
g.tempfile general,support,scripts
v.support vector,metadata
r.external raster,import,external

I can actually see that outputting just module names can be advantageous in some cases. But I want to get something like, so I can throw sed and grep on it:

v.support|vector,metadata|Updates vector map metadata.

If we permit change of the interface, I think -g could do the output above. This would make the -g output more like the others: same information as by default and with -j, so we can even consider it fixing a bug.

The current output with -g can be generated with some other flag. -n* for “names only” perhaps?

Best,

Vaclav