[GRASS-dev] Python Scripting

I'm trying to automate some modules in GRASS 6.3.2 using Python scripts
and g.parser. I've had a look at the examples in /Scripts, but I'm
having some trouble. I want to pass arguments to a GRASS module by
iterating over a list of values. Here's what I have;

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

#copied the g.parser headers and appropriately modified so that a GUI
#with the correct OPT fields does come up, but script fails when run

import os
import sys

#read cat file export (from v.category) and create a list of cat values
a=open("/home/wchris/Desktop/test.txt")
b=
for line in a:
  b.append(line)

#generate the catlist and clear \n from the input file
catlist=
for i in b:
  catlist.append(i[:-1])

#call GRASS module and iterate over catlist using it as input for
#v.extract argument list
for i in catlist:
  list=i
  v.extract input="$GIS_OPT_INPUT" output="$GIS_OPT_OUTPUT""_"i \
type="point" layer=1 new=-1 list=i

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I've also looked into some shell scripting that was mentioned in a
previous mailing at http://www.tldp.org/LDP/abs/html/loops1.html so
apparently iterating over a list and passing those elements as arguments
to a function is possible, but I'm lost. I'm also fairly new to Python.

Thanks for the help if you can offer any,

Chris

W. Chris Carleton wrote:

I'm trying to automate some modules in GRASS 6.3.2 using Python scripts
and g.parser. I've had a look at the examples in /Scripts, but I'm
having some trouble. I want to pass arguments to a GRASS module by
iterating over a list of values. Here's what I have;

  v.extract input="$GIS_OPT_INPUT" output="$GIS_OPT_OUTPUT""_"i \
type="point" layer=1 new=-1 list=i

The above is shell syntax.

In Python, it would look something like:

  import subprocess

  ...

  subprocess.call([
    "v.extract",
    "input=%s" % os.getenv("GIS_OPT_INPUT"),
    "output=%s_%s" % (os.getenv("GIS_OPT_OUTPUT"), i),
    "type=point", "layer=1", "new=-1", "list=%s" % i])

The main differences between Python and the shell:

1. All strings must be quoted.

2. Environment variables have to be accessed via os.getenv().

3. String manipulation requires explicit operators, e.g. +
(concatenation) or % (substitution).

4. Executing external programs requires the functions in the
subprocess module.

--
Glynn Clements <glynn@gclements.plus.com>

I'm curious if any though has been given to developing an actual GRASS
library for Python that goes beyond calling executables with system calls?

I'm thinking about a model that encapsulates the GRASS environment and
allows for both low-level processing like the C library, and high-level
use of existing modules like shell scripts.

I'll admit I have not given this a lot of though, but a hypothetical,
and quickly thought-out, example could be something like:

from grass import module, layer, cell, parser

def r_add(inLayers, outLayer):
  for outCell in outLayer.cells:
    sum = cell(type=outLayer.cellType)
    for l in inLayers:
      c = l.getCellAtSameLocation(outCell)
      if c.value==cell.null:
        sum.value = cell.null
        break
      sum.value += c.value
    outCell.value = c.value
  outLayer.commit()
    
if __name__ == "__main__":
  # Set up module information here
  # Set up and run parser
  # Open input layers
  # Create new layer for output
  # call r_add()
  # close layers

I don't know if this would even be feasible, but I think it would make a nice addition to GRASS's Python support. If done right it would even allow other python-based GRASS modules to be called-on without having to make a system-call (eg. from raster_add import r_add).

As to existing modules, what about a helper function to access then?

module.executeModule( name="r.stats", options={ "input": "elevation.dem,slope,aspect", "fs": ",", "output": "elev.csv"}, flags=["q", "1", "n", "g"] )

It was just a thought, and would require a lot more thought and work to
design properly. I, frankly, do not have enough understanding of GRASS internals to even know where to begin.

-- Dan.

Dan D'Alimonte wrote:

I'm curious if any though has been given to developing an actual GRASS
library for Python that goes beyond calling executables with system calls?

I'm thinking about a model that encapsulates the GRASS environment and
allows for both low-level processing like the C library, and high-level
use of existing modules like shell scripts.

I'll admit I have not given this a lot of though, but a hypothetical,
and quickly thought-out, example could be something like:

from grass import module, layer, cell, parser

def r_add(inLayers, outLayer):
  for outCell in outLayer.cells:
    sum = cell(type=outLayer.cellType)
    for l in inLayers:
      c = l.getCellAtSameLocation(outCell)
      if c.value==cell.null:
        sum.value = cell.null
        break
      sum.value += c.value
    outCell.value = c.value
  outLayer.commit()
    
if __name__ == "__main__":
  # Set up module information here
  # Set up and run parser
  # Open input layers
  # Create new layer for output
  # call r_add()
  # close layers

I don't know if this would even be feasible, but I think it would make a
nice addition to GRASS's Python support. If done right it would even
allow other python-based GRASS modules to be called-on without having to
make a system-call (eg. from raster_add import r_add).

Raster processing in Python is likely to be too slow to be practical.

As to existing modules, what about a helper function to access then?

module.executeModule( name="r.stats", options={ "input":
"elevation.dem,slope,aspect", "fs": ",", "output": "elev.csv"},
flags=["q", "1", "n", "g"] )

This idea has occurred to me. Some comments:

Pass argument values as Python values, e.g. passing multiple values as
lists, passing numeric types directly, etc, and have the interface
convert them to strings. Pass the flags as a single string.

module.execute( "r.stats",
                options = { "input": ["elevation.dem", "slope", "aspect"],
                            "fs": ",",
                            "output": "elev.csv" },
                flags = "q1ng" )

Provide a lower-level function which simply generates the command to
pass to Popen(), for cases where you want to interact with the child
process.

Provide an intermediate function which starts the process, returning
an object from which you can obtain structured progress information:
percentage complete, any messages along with their type (debug, info,
warning), etc.

--
Glynn Clements <glynn@gclements.plus.com>

Isn't there a SWIG? I think there is a 'swig' folder in the grass
installation and I remember reading about it on the web.
Stefano.

2008/7/18 Glynn Clements <glynn@gclements.plus.com>:

Dan D'Alimonte wrote:

I'm curious if any though has been given to developing an actual GRASS
library for Python that goes beyond calling executables with system calls?

I'm thinking about a model that encapsulates the GRASS environment and
allows for both low-level processing like the C library, and high-level
use of existing modules like shell scripts.

I'll admit I have not given this a lot of though, but a hypothetical,
and quickly thought-out, example could be something like:

from grass import module, layer, cell, parser

def r_add(inLayers, outLayer):
      for outCell in outLayer.cells:
              sum = cell(type=outLayer.cellType)
              for l in inLayers:
                      c = l.getCellAtSameLocation(outCell)
                      if c.value==cell.null:
                              sum.value = cell.null
                              break
                      sum.value += c.value
              outCell.value = c.value
      outLayer.commit()

if __name__ == "__main__":
      # Set up module information here
      # Set up and run parser
      # Open input layers
      # Create new layer for output
      # call r_add()
      # close layers

I don't know if this would even be feasible, but I think it would make a
nice addition to GRASS's Python support. If done right it would even
allow other python-based GRASS modules to be called-on without having to
make a system-call (eg. from raster_add import r_add).

Raster processing in Python is likely to be too slow to be practical.

As to existing modules, what about a helper function to access then?

module.executeModule( name="r.stats", options={ "input":
"elevation.dem,slope,aspect", "fs": ",", "output": "elev.csv"},
flags=["q", "1", "n", "g"] )

This idea has occurred to me. Some comments:

Pass argument values as Python values, e.g. passing multiple values as
lists, passing numeric types directly, etc, and have the interface
convert them to strings. Pass the flags as a single string.

module.execute( "r.stats",
               options = { "input": ["elevation.dem", "slope", "aspect"],
                           "fs": ",",
                           "output": "elev.csv" },
               flags = "q1ng" )

Provide a lower-level function which simply generates the command to
pass to Popen(), for cases where you want to interact with the child
process.

Provide an intermediate function which starts the process, returning
an object from which you can obtain structured progress information:
percentage complete, any messages along with their type (debug, info,
warning), etc.

--
Glynn Clements <glynn@gclements.plus.com>
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Glynn Clements wrote:

> As to existing modules, what about a helper function to access then?
>
> module.executeModule( name="r.stats", options={ "input":
> "elevation.dem,slope,aspect", "fs": ",", "output": "elev.csv"},
> flags=["q", "1", "n", "g"] )

This idea has occurred to me. Some comments:

Pass argument values as Python values, e.g. passing multiple values as
lists, passing numeric types directly, etc, and have the interface
convert them to strings. Pass the flags as a single string.

module.execute( "r.stats",
                options = { "input": ["elevation.dem", "slope", "aspect"],
                            "fs": ",",
                            "output": "elev.csv" },
                flags = "q1ng" )

Provide a lower-level function which simply generates the command to
pass to Popen(), for cases where you want to interact with the child
process.

I have attached a first draft of such a module, which also includes a
wrapper function for g.parser (for which an example script is also
attached).

--
Glynn Clements <glynn@gclements.plus.com>

(attachments)

grass.py (1.77 KB)
test.py (957 Bytes)

I have something similar in PyWPS code [1], but this looks much nicer
to me. Looking forward to adopt this.

Jachym

[1] http://wald.intevation.org/plugins/scmsvn/viewcvs.php/trunk/pywps/Process/Process.py?rev=541&root=pywps&view=markup
look after "def cmd"

2008/7/18 Glynn Clements <glynn@gclements.plus.com>:

Glynn Clements wrote:

> As to existing modules, what about a helper function to access then?
>
> module.executeModule( name="r.stats", options={ "input":
> "elevation.dem,slope,aspect", "fs": ",", "output": "elev.csv"},
> flags=["q", "1", "n", "g"] )

This idea has occurred to me. Some comments:

Pass argument values as Python values, e.g. passing multiple values as
lists, passing numeric types directly, etc, and have the interface
convert them to strings. Pass the flags as a single string.

module.execute( "r.stats",
                options = { "input": ["elevation.dem", "slope", "aspect"],
                            "fs": ",",
                            "output": "elev.csv" },
                flags = "q1ng" )

Provide a lower-level function which simply generates the command to
pass to Popen(), for cases where you want to interact with the child
process.

I have attached a first draft of such a module, which also includes a
wrapper function for g.parser (for which an example script is also
attached).

--
Glynn Clements <glynn@gclements.plus.com>

import os
import os.path
import sys
import types
import subprocess

def _make_val(val):
   if isinstance(val, types.StringType):
       return val
   if isinstance(val, types.ListType):
       return ",".join(map(_make_val, val))
   if isinstance(val, types.TupleType):
       return _make_val(list(val))
   return str(val)

def make_command(prog, options = , flags = "", overwrite = False, quiet = False, verbose = False):
   args = [prog]
   if overwrite:
       args.append("--o")
   if quiet:
       args.append("--q")
   if verbose:
       args.append("--v")
   if flags:
       args.append("-%s" % flags)
   for opt, val in options.iteritems():
       args.append("%s=%s" % (opt, _make_val(val)))
   return args

def start_command(prog, options = , flags = "", overwrite = False, quiet = False, verbose = False, **kwargs):
   args = make_command(prog, options, flags, overwrite, quiet, verbose)
   return subprocess.Popen(args, **kwargs)

def run_command(*args, **kwargs):
   ps = start_command(*args, **kwargs)
   return ps.wait()

def _parse_env():
   options = {}
   flags = {}
   for var, val in os.environ.iteritems():
       if var.startswith("GIS_OPT_"):
           opt = var.replace("GIS_OPT_", "", 1).lower()
           options[opt] = val;
       if var.startswith("GIS_FLAG_"):
           flg = var.replace("GIS_FLAG_", "", 1).lower()
           flags[flg] = bool(int(val));
   return (options, flags)

def parser():
   if not os.getenv("GISBASE"):
       print >> sys.stderr, "You must be in GRASS GIS to run this program."
       sys.exit(1)

   if len(sys.argv) > 1 and sys.argv[1] == "@ARGS_PARSED@":
       return _parse_env()

   argv = sys.argv[:]
   name = argv[0]
   if not os.path.isabs(name):
       argv[0] = os.path.normpath(os.path.join(sys.path[0], name))

   os.execvp("g.parser", [name] + argv)
   raise OSError("error executing g.parser")

#!/usr/bin/env python

# g.parser demo script for python programing

import grass

#%Module
#% description: g.parser test script (python)
#%End
#%flag
#% key: f
#% description: a flag
#%END
#%option
#% key: raster
#% type: string
#% gisprompt: old,cell,raster
#% description: raster input map
#% required : yes
#%end
#%option
#% key: vector
#% type: string
#% gisprompt: old,vector,vector
#% description: vector input map
#% required : yes
#%end
#%option
#% key: option1
#% type: string
#% description: an option
#% required : no
#%end

if __name__ == "__main__":
   (options, flags) = grass.parser()

   print ""
   if flags['f']:
       print "Flag -f set"
   else:
       print "Flag -f not set"

   #test if parameter present:
   if options['option1']:
       print "Value of option1: '%s'" % options['option1']

   print "Value of raster= option: '%s'" % options['raster']
   print "Value of vector= option: '%s'" % options['vector']

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

--
Jachym Cepicky
e-mail: jachym.cepicky gmail com
URL: http://les-ejk.cz
GPG: http://www.les-ejk.cz/pgp/jachym_cepicky-gpg.pub