[GRASS-dev] New CLI GSoC Idea: Comments, Mentors, Students Needed

Dear list,

I just compiled a another idea for this year GSoC:

https://trac.osgeo.org/grass/wiki/GSoC/2018#Neweasy-to-useCLIandAPIforGRASSGIS

All: You may remember this idea from 2015. Since then I implemented --exec which, I think, is successful since I can see it used in different posts online. However, I also think that --exec is far from an ideal interface. I seek comments for the ideas and newly proposed CLI and potential Python API design which is presented at the wiki.

Mentors: I’m seeking an additional mentor for this idea. I put myself as first, but you can be first or second mentor as you wish.

Students: This email applies to all potential GSoC students, but I’m including those of you who posted on the mailing list so far to CC just to let you know that you can apply for more than one idea which is more than appropriate when there is more than one student interested in the same idea. Please find details about this idea at the wiki.

Thank you and please let me if you have any questions or comments,

Vaclav

Dear Vaclav,

···

On Sun, Mar 11, 2018 at 4:33 AM, Vaclav Petras <wenzeslaus@gmail.com> wrote:

I just compiled a another idea for this year GSoC:

https://trac.osgeo.org/grass/wiki/GSoC/2018#Neweasy-to-useCLIandAPIforGRASSGIS

I like the idea! It is still not clear to me if you think to have a persistent storage for the “temporary” location/mapset or if they are generated every time.

Mentors: I’m seeking an additional mentor for this idea. I put myself as first, but you can be first or second mentor as you wish.

I’m available to co-mentoring this GSoC.

Best regards

Pietro

On Sun, Mar 11, 2018 at 4:33 AM, Vaclav Petras <wenzeslaus@gmail.com> wrote:

Dear list,

I just compiled a another idea for this year GSoC:

https://trac.osgeo.org/grass/wiki/GSoC/2018#Neweasy-to-useCLIandAPIforGRASSGIS

All: You may remember this idea from 2015. Since then I implemented --exec which, I think, is successful since I can see it used in different posts online. However, I also think that --exec is far from an ideal interface. I seek comments for the ideas and newly proposed CLI and potential Python API design which is presented at the wiki.

The example in the wiki is based on a GeoTIFF file with a single raster band. IMHO, the input should be more general in the form “GDAL_raster_datasource raster_band” and “GDAL_vector_datasource layer_name”. Specifying a raster band/layer name should probably be mandatory. The new parser would then use GDAL to test if the given datasource is valid and if the given raster band/vector layer exists in this datasource.

I guess the main part of this project would be to write a parser for the grass run arguments that translates them to an actual GRASS command.

Mentors: I’m seeking an additional mentor for this idea. I put myself as first, but you can be first or second mentor as you wish.

I would be interested in the general design and concept for the implementation of this project.

A very interesting idea that would facilitate the use of GRASS as a toolbox similar to GDAL or (ambitious reference) OTB.

Markus M

Students: This email applies to all potential GSoC students, but I’m including those of you who posted on the mailing list so far to CC just to let you know that you can apply for more than one idea which is more than appropriate when there is more than one student interested in the same idea. Please find details about this idea at the wiki.

Thank you and please let me if you have any questions or comments,
Vaclav


grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Hi Pietro,

···

On Mon, Mar 12, 2018 at 6:05 AM, Pietro <peter.zamb@gmail.com> wrote:

I’m not sure what you mean by “persistent storage”. The current system of connecting to an existing “D/L/M” stays in place. I mean temporary as a temporary directory, e.g. in /tmp:

“”"
GRASS Database would be created with an appropriate Location (projection based on input files or additional CLI input).
The GRASS GIS Database, Location and Mapset should be created on the fly and deleted afterwards (the .grassrc wouldn’t be used).

Add --tmp-location which runs --exec in a database/location/mapset which are created at the beginning and deleted at the end.
“”"

So, basically combination of current -c, --exec, and rm -r.

If you have ideas about how it make it more efficient that would be great since the overhead for the run option/subcommand will be high. Currently, the idea is that user can use an existing “D/L/M” if higher efficiency is needed (i.e. same as now).

On Sun, Mar 11, 2018 at 4:33 AM, Vaclav Petras <wenzeslaus@gmail.com> wrote:

I just compiled a another idea for this year GSoC:

https://trac.osgeo.org/grass/wiki/GSoC/2018#Neweasy-to-useCLIandAPIforGRASSGIS

I like the idea! It is still not clear to me if you think to have a persistent storage for the “temporary” location/mapset or if they are generated every time.

Mentors: I’m seeking an additional mentor for this idea. I put myself as first, but you can be first or second mentor as you wish.

I’m available to co-mentoring this GSoC.

Awesome!

Hi Markus,

On Mon, Mar 12, 2018 at 3:03 PM, Markus Metz <markus.metz.giswork@gmail.com> wrote:

On Sun, Mar 11, 2018 at 4:33 AM, Vaclav Petras <wenzeslaus@gmail.com> wrote:

The example in the wiki is based on a GeoTIFF file with a single raster band. IMHO, the input should be more general in the form “GDAL_raster_datasource raster_band” and “GDAL_vector_datasource layer_name”.

Yes, I agree, the input files in the examples are trivial. Ideally, this should also work with things like PostGIS or NetCDF for time series. I don’t have proposal for the syntax at this point, space from your suggestion, colon, and a separate …-layer pseudo-option come to my mind.

For vectors we already have a similar case with @OGR mapset where we use vector map name for GDAL_vector_datasource and layer for layer_name.

There we hit some terminology differences but that could be mitigated by adding something like OGC compliant descriptions which might be anyway needed for the better integration with QGIS.

https://trac.osgeo.org/grass/wiki/GSoC/2018#ImproveGRASSintegrationinQGIS3

Specifying a raster band/layer name should probably be mandatory.

I’m not sure if it should be mandatory or not. I think @OGR requires the layer, but if user provides a single band GeoTIFF or a Shapefile, it seems as expected behavior that the only band/layer is picked automatically.

The new parser would then use GDAL to test if the given datasource is valid and if the given raster band/vector layer exists in this datasource.

These and other tests will be big part of the backend: “The system behind the interface will be inherently fragile, so it is necessary to write large amount of tests which would check different combinations of data types and projections.”

I guess the main part of this project would be to write a parser for the grass run arguments that translates them to an actual GRASS command.

Yes, there are two types of arguments, those related to grass run like --mask and then those which are part of module interface like input which translate to some import commands and also the module option.

In case somebody wonders, yes, --mask could be a general option for all the module, in fact --region was already discussed (r.relief input=… --v --r=“n=5520 s=4000…”). Further, input could be written like --input to follow POSIX and finally, even to modules themselves could accept the general files (like with @OGR), rather than going through run. But any of these are outside of scope for this GSoC and it is probably a good idea to leave them for the next iteration. Same goes for the full subcommand interface idea and Grassfile (directory rc file) idea mentioned in the comment 15 of #2579.

https://trac.osgeo.org/grass/ticket/2579#comment:15

Mentors: I’m seeking an additional mentor for this idea. I put myself as first, but you can be first or second mentor as you wish.

I would be interested in the general design and concept for the implementation of this project.

Great, this will be very helpful.

A very interesting idea that would facilitate the use of GRASS as a toolbox similar to GDAL or (ambitious reference) OTB.

Thanks and thanks for the feedback too!

Vaclav

Hi to all,

I like the proposal and the discussion.

Would it be a good idea to schedule a virtual meeting during the code sprint next week ?

Moritz

Am 13. März 2018 02:16:09 MEZ schrieb Vaclav Petras <wenzeslaus@gmail.com>:

Hi Markus,

On Mon, Mar 12, 2018 at 3:03 PM, Markus Metz
<markus.metz.giswork@gmail.com>
wrote:

On Sun, Mar 11, 2018 at 4:33 AM, Vaclav Petras <wenzeslaus@gmail.com>

wrote:

The example in the wiki is based on a GeoTIFF file with a single

raster
band. IMHO, the input should be more general in the form
"GDAL_raster_datasource raster_band" and "GDAL_vector_datasource
layer_name".

Yes, I agree, the input files in the examples are trivial. Ideally,
this
should also work with things like PostGIS or NetCDF for time series. I
don't have proposal for the syntax at this point, space from your
suggestion, colon, and a separate ...-layer pseudo-option come to my
mind.

For vectors we already have a similar case with @OGR mapset where we
use
vector map name for GDAL_vector_datasource and layer for layer_name.

There we hit some terminology differences but that could be mitigated
by
adding something like OGC compliant descriptions which might be anyway
needed for the better integration with QGIS.

https://trac.osgeo.org/grass/wiki/GSoC/2018#ImproveGRASSintegrationinQGIS3

Specifying a raster band/layer name should probably be mandatory.

I'm not sure if it should be mandatory or not. I think @OGR requires
the
layer, but if user provides a single band GeoTIFF or a Shapefile, it
seems
as expected behavior that the only band/layer is picked automatically.

The new parser would then use GDAL to test if the given datasource is

valid and if the given raster band/vector layer exists in this
datasource.

These and other tests will be big part of the backend: "The system
behind
the interface will be inherently fragile, so it is necessary to write
large
amount of tests which would check different combinations of data types
and
projections."

I guess the main part of this project would be to write a parser for

the
`grass run` arguments that translates them to an actual GRASS command.

Yes, there are two types of arguments, those related to `grass run`
like
`--mask` and then those which are part of module interface like `input`
which translate to some import commands and also the module option.

In case somebody wonders, yes, `--mask` could be a general option for
all
the module, in fact `--region` was already discussed (r.relief
input=...
--v --r="n=5520 s=4000..."). Further, `input` could be written like
`--input` to follow POSIX and finally, even to modules themselves could
accept the general files (like with @OGR), rather than going through
`run`.
But any of these are outside of scope for this GSoC and it is probably
a
good idea to leave them for the next iteration. Same goes for the full
subcommand interface idea and Grassfile (directory rc file) idea
mentioned
in the comment 15 of #2579.

https://trac.osgeo.org/grass/ticket/2579#comment:15

> Mentors: I'm seeking an additional mentor for this idea. I put

myself
as first, but you can be first or second mentor as you wish.

I would be interested in the general design and concept for the

implementation of this project.

Great, this will be very helpful.

A very interesting idea that would facilitate the use of GRASS as a

toolbox similar to GDAL or (ambitious reference) OTB.

Thanks and thanks for the feedback too!

Vaclav

On Tue, Mar 13, 2018 at 2:56 AM, Moritz Lennert <mlennert@club.worldonline.be> wrote:

Would it be a good idea to schedule a virtual meeting during the code sprint next week ?

Yes, please. I plan to be available on Monday to do calls.

Ciao Pietro and all,

···

On Mon, Mar 12, 2018 at 11:05 AM, Pietro <peter.zamb@gmail.com> wrote:

may I remind to subscribe as a mentor filling the form https://goo.gl/forms/kddSWrLkna84oXyb2 , it’s not too late for new mentors, but the sooner the better. Thanks!

I’m available to co-mentoring this GSoC.

Margherita Di Leo