#3639: Write table like output directly to database
--------------------------+----------------------------------
Reporter: sbl | Owner: grass-dev@…
Type: enhancement | Status: new
Priority: normal | Milestone: 8.0.0
Component: Database | Version: svn-trunk
Resolution: | Keywords: table,database,write
CPU: Unspecified | Platform: Unspecified
--------------------------+----------------------------------
Comment (by sbl):
I am considering adding a function to the Python scripting library for
parsing table-like stdout from GRASS modules into Numpy arrays. Nothing
complicated, but probably a convenient wrapper function!?
See:
{{{
def stdout2numpy(stdout=None, sep=',', names=False, null_value=None,
fill_value=None, comments='#', usecols=None):
"""Read table-like output from grass modules as Numpy array;
format instructions are handed down to Numpys genfromtxt function
param str|byte stdout: tabular stdout from GRASS GIS module call
param str sep: Separator delimiting columns
param list names: List of strings with names for columns
param str null_value: Characters representing the no-data value
param str fill_value: Value to fill no-data with
param str comments: Character that
param list usecols: List of columns to import
"""
import numpy as np
from io import BytesIO
if type(stdout) == str:
stdout = gscript.encode(stdout)
elif type(stdout) != byte:
gscript.fatal(_('Unsupported data type'))
np_array = np.genfromtxt(BytesIO(stdout),
missing_values=null_value,
filling_values=fill_value,
usecols=usecols,
names=names,
dtype=None, delimiter=sep)
return np_array
}}}
or alternatively as np_parse_command (equivalent to parse_command()):
{{{
def np_parse_command(*args, **kwargs):
"""Passes all arguments to read_command, then parses the output
using Numpys genfromtxt() function to generate a Numpy array from
table like output.
Parsing options in Numpys genfromtxt() function can be
optionally given by <em>parse</em> dictionary, e.g.
::
parse_command(..., parse = { 'delimiter' : '|' }))
As far as possible, at least standard parser options in GRASS
commands are handed down and translated to genfromtxt()
automatically, e.g. separator (GRASS) -> delimiter (Numpy).
import numpy as np
from io import BytesIO
stdout = read_command(*args, **kwargs)
if type(stdout) == str:
stdout = gscript.encode(stdout)
elif type(stdout) != byte:
gscript.fatal(_('Unsupported data type'))
filling_values = parse['filling_values'] if 'filling_values' in parse
else np.nan
usecols = parse['usecols'] if 'usecols' in parse else None
names = parse['names'] if 'names' in parse else None
dtype = parse['dtype'] if 'dtype' in parse else None
# Check if separator is specified in GRASS command
if 'delimiter' not in parse:
if 'separator' in args:
parse['delimiter'] = args['separator']
elif 'separator' in kwargs:
parse['delimiter'] = kwargs['separator']
else:
parse['delimiter'] = ','
# Check if null_value is specified in GRASS command
if 'missing_values' not in parse:
if 'null_value' in args:
parse['missing_values'] = args['null_value']
elif 'null_value' in kwargs:
parse['missing_values'] = kwargs['null_value']
else:
parse['missing_values'] = '*'
np_array = np.genfromtxt(BytesIO(stdout), **parse)
return np_array
}}}
That would probably be a first step for a Python function that can write
output to DB...
Any thoughts/opinions?
--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3639#comment:2>
GRASS GIS <https://grass.osgeo.org>