[GRASS-dev] GRASS & QGIS: the future

GRASS GIS GRASS GIS developer
, ,
1

Hi all.
I learned during dinner that GRASS7 RC1 is due very soon. This opens the
issue of its functioning in QGIS. IMHO:

* the qgis-grass-plugin might stop working (this has to be tested)
* some of the module options will be different
* new modules will not be available in QGIS.

I think we can deal with this in several ways:

* dropping the plugin, and concentrate the work on Processing
* upgrading both the plugin and Processing.

In the first case, we have two major issues:

* upgrading Processing GRASS modules
* changing the current Processing behaviour, avoiding the import-export
phase when piping consecutive GRASS commands; this makes GRASS modules
slower than the equivalent commands of other backends.

While the first issue can be solved easily by a couple of people in a
few days, the second one is more tricky, and requires hard coding skills.
In addition, we'll no longer be able to edit GRASS vectors directly.

In the second case, we'll have more work, and a not-so-nice duplication.

I would like to have an open discussion on this, avoiding things to just
happen, with the possible negative consequences.

All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html

2

Hi Paolo,

(note: I'm not ready qgis-dev, not sure if this email reaches it)

On Thu, Mar 27, 2014 at 11:18 AM, Paolo Cavallini <cavallini@faunalia.it> wrote:

Hi all.
I learned during dinner that GRASS7 RC1 is due very soon. This opens the
issue of its functioning in QGIS. IMHO:

* the qgis-grass-plugin might stop working (this has to be tested)
* some of the module options will be different

For the time being people can certainly continue to work with GRASS 6.

I think that the plugin needs to be "cloned" for GRASS 7.

* new modules will not be available in QGIS.

There might be only a few which could be of interest here (e.g. the
highly specialized evapotranspiration modules not but some others
yes). For an overview, see

http://trac.osgeo.org/grass/wiki/Grass7/NewFeatures

I think we can deal with this in several ways:

* dropping the plugin, and concentrate the work on Processing
* upgrading both the plugin and Processing.

(the second: yes)

In the first case, we have two major issues:

* upgrading Processing GRASS modules

I'll do that. I already started with Pirmin and Victor to discuss it.

* changing the current Processing behaviour, avoiding the import-export
phase when piping consecutive GRASS commands; this makes GRASS modules
slower than the equivalent commands of other backends.

... this would not be a good idea.

While the first issue can be solved easily by a couple of people in a
few days, the second one is more tricky, and requires hard coding skills.
In addition, we'll no longer be able to edit GRASS vectors directly.

In the second case, we'll have more work, and a not-so-nice duplication.

I would like to have an open discussion on this, avoiding things to just
happen, with the possible negative consequences.

I expect that we'll come up with a "Processing" prototype for which
supports GRASS 7 soon.

Best
Markus

3

Il 27/03/2014 12:18, Markus Neteler ha scritto:

* upgrading Processing GRASS modules

I'll do that. I already started with Pirmin and Victor to discuss it.

good news

* changing the current Processing behaviour, avoiding the import-export
phase when piping consecutive GRASS commands; this makes GRASS modules
slower than the equivalent commands of other backends.

... this would not be a good idea.

could you please explain why?

all the best.

--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html

4

On 27 March 2014 11:18, Paolo Cavallini <cavallini@faunalia.it> wrote:

Hi all.

Hi all,

I learned during dinner that GRASS7 RC1 is due very soon. This opens the
issue of its functioning in QGIS. IMHO:

* the qgis-grass-plugin might stop working (this has to be tested)

I test to compile QGIS master with GRASS7 but it doesn't work, the
GRASS7 directory it seems not be recognized bu QGIS.
Can I open a ticket on QGIS bug tracker?

* some of the module options will be different
* new modules will not be available in QGIS.

I think we can deal with this in several ways:

* dropping the plugin, and concentrate the work on Processing
* upgrading both the plugin and Processing.

In the first case, we have two major issues:

* upgrading Processing GRASS modules
* changing the current Processing behaviour, avoiding the import-export
phase when piping consecutive GRASS commands; this makes GRASS modules
slower than the equivalent commands of other backends.

While the first issue can be solved easily by a couple of people in a
few days, the second one is more tricky, and requires hard coding skills.
In addition, we'll no longer be able to edit GRASS vectors directly.

In the second case, we'll have more work, and a not-so-nice duplication.

I'm not using at all GRASS from QGIS, so I think could be useful make
a poll to know what the user need

All the best.

--
ciao
Luca

http://gis.cri.fmach.it/delucchi/
www.lucadelu.org

5

Il 27/03/2014 12:33, Nathan Woodrow ha scritto:

I would vote for dropping the plugin and just updating the processing
plugin. Having both ways is bad for us and bad for users, even worse
when some functions are missing from one but not in the other.

I understand well the point; however, the plugin has additional
functions, e.g.:
* a grass shell
* a grass data browser
* a grass digitizing environment.
Whether these are important or not, it's a matter of users.
All the best.

--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html

6

Dear all,

From my user perspective (I am using both GRASS and QGIS, or the other way around, depends on topic), processing is not really a replacement for the GRASS plugin.

It is handy and probably better for those who do not use GRASS but are interested in single functions / modules.

Yet, once you have a GRASS database with a considerable amount of data, the GRASS plugin is very valuable for accessing those data in QGIS (e.g. for cartography). Also users with no or little GRASS experience benefit from the GRASS plugin in cases where they have access to a GRASS database.
That way they can work on it without acquainting themselves with a "new" GIS in depth...

Concluding, if you have the possibility to maintain it, keep the GRASS plugin in QGIS!

Cheers
Stefan

-----Original Message-----
From: qgis-developer-bounces@lists.osgeo.org [mailto:qgis-developer-bounces@lists.osgeo.org] On Behalf Of Paolo Cavallini
Sent: 27. mars 2014 12:41
To: Nathan Woodrow
Cc: qgis-developer; grass-dev
Subject: Re: [Qgis-developer] GRASS & QGIS: the future

Il 27/03/2014 12:33, Nathan Woodrow ha scritto:

I would vote for dropping the plugin and just updating the processing
plugin. Having both ways is bad for us and bad for users, even worse
when some functions are missing from one but not in the other.

I understand well the point; however, the plugin has additional functions, e.g.:
* a grass shell
* a grass data browser
* a grass digitizing environment.
Whether these are important or not, it's a matter of users.
All the best.

--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
_______________________________________________
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

7

I'm not much of a QGIS-as-GRASS-frontend user, either, but from a general point of view I also think that duplication is a problem and that the current way to go in QGIS is the processing framework. So;

On 27/03/14 12:49, Blumentrath, Stefan wrote:

I understand well the point; however, the plugin has additional functions, e.g.:
* a grass shell

couldn't this be implemented within the processing environment ?

* a grass data browser

If we are talking about accessing GRASS data for loading into QGIS, wouldn't it be a better idea to improve the GDAL/OGR handling in order to be able to load GRASS data just like any other data format ?

* a grass digitizing environment.

Maybe this could be split out into a specific plugin ?

Moritz

8

That sounds very reasonable to me.

Proper GDAL/OGR handling for GRASS 7 would be very nice in any case (see http://trac.osgeo.org/gdal/ticket/2953).

As for a "GRASS data browser", I think, a plugin would be required with regards to user friendliness, because one needs to know what files to access from a GRASS data base when using GDAL.

I also understand that at some point in time one will have to use GRASS directly in order to access full functionality (e.g. ortho-rectification, nviz, mapswipe, animation and stuff), which makes the way Moritz suggests maybe even more reasonable...

Cheers
Stefan

-----Original Message-----
From: Moritz Lennert [mailto:mlennert@club.worldonline.be]
Sent: 27. mars 2014 13:36
To: Blumentrath, Stefan
Cc: Paolo Cavallini; Nathan Woodrow; qgis-developer; grass-dev
Subject: Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

I'm not much of a QGIS-as-GRASS-frontend user, either, but from a general point of view I also think that duplication is a problem and that the current way to go in QGIS is the processing framework. So;

On 27/03/14 12:49, Blumentrath, Stefan wrote:

I understand well the point; however, the plugin has additional functions, e.g.:
* a grass shell

couldn't this be implemented within the processing environment ?

* a grass data browser

If we are talking about accessing GRASS data for loading into QGIS, wouldn't it be a better idea to improve the GDAL/OGR handling in order to be able to load GRASS data just like any other data format ?

* a grass digitizing environment.

Maybe this could be split out into a specific plugin ?

Moritz

9

On Thu, Mar 27, 2014 at 9:38 AM, Blumentrath, Stefan <
Stefan.Blumentrath@nina.no> wrote:

I also understand that at some point in time one will have to use GRASS
directly in order to access full functionality (e.g. ortho-rectification,
nviz, mapswipe, animation and stuff), which makes the way Moritz suggests
maybe even more reasonable...

I hope that we find the way to start these tools from QGIS. Tck/Tk nviz is
starting in this way. I hope that it will be possible to do the same for
g.gui.* modules.

Also, it would be nice if the QGIS GRASS vector digitizer (plugin/tool)
would use the same backend as vector digitizer in GRASS wxGUI (Python
subprocessing exit-safe wrapper). Now the first problem is that there is no
reusable backend in wxGUI.

Vaclav

10

In the gvSIG CE project, we had to make the same
type of decisions and came to our own conclusions.

Allow me to summarize our reasoning, maybe it will be
useful for the QGIS project, as well:

1. The number one cause of irritations among novice
users is having to set up a GRASS mapset and having
to understand how GRASS data management works.

2. The number two cause of irritations are the effects
of importing vector data with bad topology into a
GRASS mapset.

3. The original QGIS plugin does nothing to alleviate (1).
No plug-in, however cleverly designed, can do anything
about (2): garbage in, garbage out.

4. GRASS "power users" gain very little (if anything)
from running GRASS through a host GIS, such as QGIS
or gvSIG. But they do lose functionality, such as the
d.* family of modules.

Therefore, we gave up trying to design a plug-in for
advanced users. We assume that such users will use
GRASS through its native CLI and/or native GUI.

The resulting design of the original SEXTANTE-GRASS
interface (which is now also mirrored in the Python
re-write that became QGIS' Processing) addresses users
that either don't care much for GRASS' CLI capabilities
and internal data management, or are willing to switch to
native GRASS as needed.

If you want to change this and address another type
of user, then you will need to re-examine the entire design
of the current SEXTANTE/Processing approach, which is to use
only temporary mapsets and perform data import/export every
time a GRASS module is run.

You can optimize the I/O performance of Processing by using
r.external to directly access raster input maps. But there
is little you can do about vector data with the current
design, as GRASS needs to build its own topological data
structures (and rebuild them every time you run a GRASS module
on non-topological input!).

In any case, I do not think that it is worth maintaining
the original QGIS plugin, since it is neither very well
suited for novice nor advanced users, IMHO.

Best,

Ben

On 27/03/14 14:38, Blumentrath, Stefan wrote:

That sounds very reasonable to me.

Proper GDAL/OGR handling for GRASS 7 would be very nice in any case (see http://trac.osgeo.org/gdal/ticket/2953).

As for a "GRASS data browser", I think, a plugin would be required with regards to user friendliness, because one needs to know what files to access from a GRASS data base when using GDAL.

I also understand that at some point in time one will have to use GRASS directly in order to access full functionality (e.g. ortho-rectification, nviz, mapswipe, animation and stuff), which makes the way Moritz suggests maybe even more reasonable...

Cheers
Stefan

-----Original Message-----
From: Moritz Lennert [mailto:mlennert@club.worldonline.be]
Sent: 27. mars 2014 13:36
To: Blumentrath, Stefan
Cc: Paolo Cavallini; Nathan Woodrow; qgis-developer; grass-dev
Subject: Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

I'm not much of a QGIS-as-GRASS-frontend user, either, but from a general point of view I also think that duplication is a problem and that the current way to go in QGIS is the processing framework. So;

On 27/03/14 12:49, Blumentrath, Stefan wrote:

I understand well the point; however, the plugin has additional functions, e.g.:
* a grass shell

couldn't this be implemented within the processing environment ?

* a grass data browser

If we are talking about accessing GRASS data for loading into QGIS, wouldn't it be a better idea to improve the GDAL/OGR handling in order to be able to load GRASS data just like any other data format ?

* a grass digitizing environment.

Maybe this could be split out into a specific plugin ?

Moritz
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

--
Dr. Benjamin Ducke, M.A.
{*} Geospatial Consultant
{*} GIS Developer

  benducke@fastmail.fm

11

Hi all,

Here is my wish list:

- To be able to access GRASS data within QGIS. Similar to PostGIS and other
GDAL/OGR data sources, it should be up to users to create GRASS
geodatabase/location/mapsets. QGIS will be able to simply add layers for
viewing and editing
  
- Having access to GRASS shell. It will be a very handy shortcut to run
GRASS modules without the need to open GRASS. GRASS modules which are
available in Processing toolbox can use the same UI. Otherwise, user has to
run it in shell.

- Simple import data to the mapsets...similar approach as drag-n-drop for
PostGIS databases. Right-click and save as... in QGIS does a great work for
Export.

In summary, Processing toolbox should be a good one-stop for one-off use of
a specific GRASS module. But if a user wants to dig more, GRASS already
offers all that. For those who have already got their GRASS set up, they
should be able to view and edit their data in QGIS with the added benefit of
having access to shell.

Some of the stuff can be ported to plugin(s) as mentioned by others: e.g.
creating location/mapsets, setting region from qgis canvas or other qgis
layers, etc.

Cheers,

Saber

-----Original Message-----
From: grass-dev-bounces@lists.osgeo.org
[mailto:grass-dev-bounces@lists.osgeo.org] On Behalf Of Benjamin Ducke
Sent: 27 March 2014 14:31
To: grass-dev
Subject: Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

In the gvSIG CE project, we had to make the same type of decisions and came
to our own conclusions.

Allow me to summarize our reasoning, maybe it will be useful for the QGIS
project, as well:

1. The number one cause of irritations among novice users is having to set
up a GRASS mapset and having to understand how GRASS data management works.

2. The number two cause of irritations are the effects of importing vector
data with bad topology into a GRASS mapset.

3. The original QGIS plugin does nothing to alleviate (1).
No plug-in, however cleverly designed, can do anything about (2): garbage
in, garbage out.

4. GRASS "power users" gain very little (if anything) from running GRASS
through a host GIS, such as QGIS or gvSIG. But they do lose functionality,
such as the
d.* family of modules.

Therefore, we gave up trying to design a plug-in for advanced users. We
assume that such users will use GRASS through its native CLI and/or native
GUI.

The resulting design of the original SEXTANTE-GRASS interface (which is now
also mirrored in the Python re-write that became QGIS' Processing) addresses
users that either don't care much for GRASS' CLI capabilities and internal
data management, or are willing to switch to native GRASS as needed.

If you want to change this and address another type of user, then you will
need to re-examine the entire design of the current SEXTANTE/Processing
approach, which is to use only temporary mapsets and perform data
import/export every time a GRASS module is run.

You can optimize the I/O performance of Processing by using r.external to
directly access raster input maps. But there is little you can do about
vector data with the current design, as GRASS needs to build its own
topological data structures (and rebuild them every time you run a GRASS
module on non-topological input!).

In any case, I do not think that it is worth maintaining the original QGIS
plugin, since it is neither very well suited for novice nor advanced users,
IMHO.

Best,

Ben

On 27/03/14 14:38, Blumentrath, Stefan wrote:

That sounds very reasonable to me.

Proper GDAL/OGR handling for GRASS 7 would be very nice in any case (see

http://trac.osgeo.org/gdal/ticket/2953).

As for a "GRASS data browser", I think, a plugin would be required with

regards to user friendliness, because one needs to know what files to access
from a GRASS data base when using GDAL.

I also understand that at some point in time one will have to use GRASS

directly in order to access full functionality (e.g. ortho-rectification,
nviz, mapswipe, animation and stuff), which makes the way Moritz suggests
maybe even more reasonable...

Cheers
Stefan

-----Original Message-----
From: Moritz Lennert [mailto:mlennert@club.worldonline.be]
Sent: 27. mars 2014 13:36
To: Blumentrath, Stefan
Cc: Paolo Cavallini; Nathan Woodrow; qgis-developer; grass-dev
Subject: Re: [GRASS-dev] [Qgis-developer] GRASS & QGIS: the future

I'm not much of a QGIS-as-GRASS-frontend user, either, but from a
general point of view I also think that duplication is a problem and
that the current way to go in QGIS is the processing framework. So;

On 27/03/14 12:49, Blumentrath, Stefan wrote:

I understand well the point; however, the plugin has additional

functions, e.g.:

* a grass shell

couldn't this be implemented within the processing environment ?

* a grass data browser

If we are talking about accessing GRASS data for loading into QGIS,

wouldn't it be a better idea to improve the GDAL/OGR handling in order to be
able to load GRASS data just like any other data format ?

* a grass digitizing environment.

Maybe this could be split out into a specific plugin ?

Moritz
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

--
Dr. Benjamin Ducke, M.A.
{*} Geospatial Consultant
{*} GIS Developer

  benducke@fastmail.fm
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

--
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the
individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately
by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified
that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

Whilst reasonable care has been taken to avoid virus transmission, no responsibility for viruses is taken and it is your responsibility to carry out
such checks as you feel appropriate.

If this email contains a quote or offer to sell products, carry out work or perform services then our standard terms and conditions (which can be found at http://www.lutraconsulting.co.uk/downloads/Lutra%20Consulting%20Standard%20Terms%20and%20Conditions.pdf shall apply unless explicitly stated otherwise.

Saber Razmjooei and Peter Wells trading as Lutra Consulting.

12

Hi Paolo,

2014-03-27 11:18 GMT+01:00 Paolo Cavallini <cavallini@faunalia.it>:

Hi all.
I learned during dinner that GRASS7 RC1 is due very soon. This opens the
issue of its functioning in QGIS. IMHO:

* the qgis-grass-plugin might stop working (this has to be tested)

Yes, it will stop working, since the API in GRASS7 has changed (Raster
API functions have now "Rast_" as prefix). Besides of that must the
cmake files be modified to detect GRASS7.

* some of the module options will be different
* new modules will not be available in QGIS.

Yes, but this shouldn't be a problem if the module interface
description created by the GRASS modules itself was used to generate
the module interface in QGIS. New and modified modules will not work
if the interfaces are handcrafted.

I think we can deal with this in several ways:

* dropping the plugin, and concentrate the work on Processing

That is IMHO not a good idea. I think to provide the full
functionality of GRASS7 to QGIS user, this plugin should be maintained
and updated to support the new GRASS7 API. Handling and processing of
massive datasets, especially time series, is only meaningful if GRASS
is used as data storage as well. The processing interface will add
massively overhead in data processing. The temporary location/mapset
creation approach is not well suited to process massive data, even
though r.external and v.external are used to link external data
temporary into GRASS.

* upgrading both the plugin and Processing.

Yes, that's the way to go.

In the first case, we have two major issues:

* upgrading Processing GRASS modules
* changing the current Processing behaviour, avoiding the import-export
phase when piping consecutive GRASS commands; this makes GRASS modules
slower than the equivalent commands of other backends.

While the first issue can be solved easily by a couple of people in a
few days, the second one is more tricky, and requires hard coding skills.
In addition, we'll no longer be able to edit GRASS vectors directly.

In the second case, we'll have more work, and a not-so-nice duplication.

I would like to have an open discussion on this, avoiding things to just
happen, with the possible negative consequences.

My suggestion would be:

Full integration of the GRASS7 into QGIS via C++ or Python plugin.
This includes the temporal GIS capabilities as well.
The existing plugin is a very good start point, lots its functionality
can be reused, especially the vector editing, grass shell and map
management.

But there is a major problem with the GRASS QGIS plugin: it links
directly to the grass libraries and calls plenty of functions that can
QGIS cause to crash in case of an error. We face the same problem in
GRASS with the vector editing tools. My solution would be to use a RPC
(Remote Procedure Call) interface to calls GRASS library functions in
a remote process using binary data for inter-process communication.

IMHO the best tool for this is apache thrift[1] which allows us to
implement a RPC interface in GRASS7 to the needed library functions.
IMHO the number of RPC functions is limited since only vector editing,
raster map rendering and some map/stds management functions are needed
for direct access, all other functionality is provided by GRASS
modules.

So the first step is to implement an RPC interface in GRASS7, that
supports C/C++, Java, Python and JavaScript on the client side out of
the box. This interface can be used by the GRASS GUI itself to
implement exit safe vector editing and it can be used by QGIS and
other nice GIS desktop systems to provide GRASS database access, fast
raster rendering and vector edit functionality.

The beauty of this approach is, that the client side (the QGIS plugin
for example) do not need to link against GRASS libraries, since it
will communicate via pipes or sockets with one or several persistent
GRASS7 processes, which can be restarted in case of a fatal error. The
client side do not need to be updated in case the GRASS7 API changes
again, only the server side which will be implemented in GRASS7 must
be updated.

Implementation effort In case of the QGIS plugin:
All direct GRASS dependencies and function calls must be removed and
replaced by the client RPC solution. Hence the provider classes needs
to be rewritten, the C++ plugin code itself needs to be modified to
support the new interface datatypes that are used for inter-process
communication. Access to the temporal GIS functionality must be
implemented to list space time datasets.

What do you think about this approach?

Best regards
Soeren

[1] https://thrift.apache.org/

All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

13

Hi,
This approach is pretty to what the IPython developers implemented with their HTM5+JS interface to the IPython Kernel. (IPython Notebook)

As RPC interface, IPython is using pyzmq (based on zeromq).

IPython has a already a QtConsole with “inline plot” that can easily replace the Qgis python shell.
This will give us inline plot to implement the ,missed d.* capabilities of grass 5.* and 6.*
as well as direct call to grass commands (in IPython any system call can be performed using the prefix “!”)
and using the “!” prefix or the %%BASH magic we can call any grass module, if the GRASS envs are properly exported.

I’m on a mac osx 64 bit, is almost 3 years i can not use grass GUI because of the WxPython interface that doesn’t support 64 bit on osx,
and i found in this approach the only way to let me use GRASS in a productive way(now that tcltk is removed)

Recent development in IPYthon enabled the ability to build complex widget based on query or othe r JS libs thanks to the “interact-widget” API

I was planning to reuse part of this technologies for the WebGRASS interface, mixing IPython and PyWT.

I’m catching a fight right now, this video is almost “old” but shows how the widget interact API works :

http://www.frequency.com/video/ipython-attributes-ofsoftware-how-they/135236136/-/5-1851523

i like the RPC approach, and i guess we can reuse the ipython sw to implement this capabilities.

please apologize me if i went OFF topic …

Massimo.

On Mar 27, 2014, at 11:17 AM, Sören Gebbert <soerengebbert@googlemail.com> wrote:

Hi Paolo,

2014-03-27 11:18 GMT+01:00 Paolo Cavallini <cavallini@faunalia.it>:

Hi all.
I learned during dinner that GRASS7 RC1 is due very soon. This opens the
issue of its functioning in QGIS. IMHO:

  • the qgis-grass-plugin might stop working (this has to be tested)

Yes, it will stop working, since the API in GRASS7 has changed (Raster
API functions have now “Rast_” as prefix). Besides of that must the
cmake files be modified to detect GRASS7.

  • some of the module options will be different
  • new modules will not be available in QGIS.

Yes, but this shouldn’t be a problem if the module interface
description created by the GRASS modules itself was used to generate
the module interface in QGIS. New and modified modules will not work
if the interfaces are handcrafted.

I think we can deal with this in several ways:

  • dropping the plugin, and concentrate the work on Processing

That is IMHO not a good idea. I think to provide the full
functionality of GRASS7 to QGIS user, this plugin should be maintained
and updated to support the new GRASS7 API. Handling and processing of
massive datasets, especially time series, is only meaningful if GRASS
is used as data storage as well. The processing interface will add
massively overhead in data processing. The temporary location/mapset
creation approach is not well suited to process massive data, even
though r.external and v.external are used to link external data
temporary into GRASS.

  • upgrading both the plugin and Processing.

Yes, that’s the way to go.

In the first case, we have two major issues:

  • upgrading Processing GRASS modules
  • changing the current Processing behaviour, avoiding the import-export
    phase when piping consecutive GRASS commands; this makes GRASS modules
    slower than the equivalent commands of other backends.

While the first issue can be solved easily by a couple of people in a
few days, the second one is more tricky, and requires hard coding skills.
In addition, we’ll no longer be able to edit GRASS vectors directly.

In the second case, we’ll have more work, and a not-so-nice duplication.

I would like to have an open discussion on this, avoiding things to just
happen, with the possible negative consequences.

My suggestion would be:

Full integration of the GRASS7 into QGIS via C++ or Python plugin.
This includes the temporal GIS capabilities as well.
The existing plugin is a very good start point, lots its functionality
can be reused, especially the vector editing, grass shell and map
management.

But there is a major problem with the GRASS QGIS plugin: it links
directly to the grass libraries and calls plenty of functions that can
QGIS cause to crash in case of an error. We face the same problem in
GRASS with the vector editing tools. My solution would be to use a RPC
(Remote Procedure Call) interface to calls GRASS library functions in
a remote process using binary data for inter-process communication.

IMHO the best tool for this is apache thrift[1] which allows us to
implement a RPC interface in GRASS7 to the needed library functions.
IMHO the number of RPC functions is limited since only vector editing,
raster map rendering and some map/stds management functions are needed
for direct access, all other functionality is provided by GRASS
modules.

So the first step is to implement an RPC interface in GRASS7, that
supports C/C++, Java, Python and JavaScript on the client side out of
the box. This interface can be used by the GRASS GUI itself to
implement exit safe vector editing and it can be used by QGIS and
other nice GIS desktop systems to provide GRASS database access, fast
raster rendering and vector edit functionality.

The beauty of this approach is, that the client side (the QGIS plugin
for example) do not need to link against GRASS libraries, since it
will communicate via pipes or sockets with one or several persistent
GRASS7 processes, which can be restarted in case of a fatal error. The
client side do not need to be updated in case the GRASS7 API changes
again, only the server side which will be implemented in GRASS7 must
be updated.

Implementation effort In case of the QGIS plugin:
All direct GRASS dependencies and function calls must be removed and
replaced by the client RPC solution. Hence the provider classes needs
to be rewritten, the C++ plugin code itself needs to be modified to
support the new interface datatypes that are used for inter-process
communication. Access to the temporal GIS functionality must be
implemented to list space time datasets.

What do you think about this approach?

Best regards
Soeren

[1] https://thrift.apache.org/

All the best.

Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html

grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

14

Il 28/03/2014 08:34, Paolo Cavallini ha scritto:

Thanks to all for the interesting comments. Obviously we have a varied
and rich ecosystem here, that's why I want to preserve it.
Obviously we all want to have all. IMHO the points are:

* can we (GRASS+QGIS) agree on a specific course of actions?
* can we find the resource to implement it?

Of course we could think of sticking with GRASS6, but I think sooner or
later this will be untenable (e.g. major distros will switch to GRASS7
once out), so better prepare now.

Quick interesting chat with Luca Delucchi and Martin Landa from GRASS
dev team.
One interesting option would be:
* to add GRASS browsing capabilities in QGIS file browser
* to add support to direct GRASS reading (possibly writing) in
Processing, and avoid import/export whenever possible; v.external and
v.out.external could be used for this; this is probably the thoughest part
* of course modules should be upgraded anyway
* GRASS direct digitizing can be skipped; heavy digitizer can use GRASS
directly
* keep a GRASS shell?
* with all the above, upgrading the plugin may probably be skipped.

Any thoughts.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html

15

One last addition to this from my side:

Since, as I mentioned, we have evaluated many of the
issues discussed in this thread in the design of the
GRASS plug-in for SEXTANTE/gvSIG CE, here are some things
you might want to take into consideration:

On 28/03/14 11:48, Paolo Cavallini wrote:

Il 28/03/2014 08:34, Paolo Cavallini ha scritto:

Thanks to all for the interesting comments. Obviously we have a varied
and rich ecosystem here, that's why I want to preserve it.
Obviously we all want to have all. IMHO the points are:

* can we (GRASS+QGIS) agree on a specific course of actions?
* can we find the resource to implement it?

Of course we could think of sticking with GRASS6, but I think sooner or
later this will be untenable (e.g. major distros will switch to GRASS7
once out), so better prepare now.

Quick interesting chat with Luca Delucchi and Martin Landa from GRASS
dev team.
One interesting option would be:
* to add GRASS browsing capabilities in QGIS file browser

Browsing capabilities for existing GRASS mapsets would be
nice, but having the plug-in directly manipulate the data
in there leads down a difficult path (see my notes further
below).

* to add support to direct GRASS reading (possibly writing) in
Processing, and avoid import/export whenever possible; v.external and
v.out.external could be used for this; this is probably the thoughest part

As far as I understand, the v.out.* modules create read-only
datasets. They also create minimal topological data structures
that are not enough for many GRASS modules (correct me if any of
these statements is no longer true). Therefore, we decided against
this path and use v.in/out.ogr instead -- with all known problems.

But r.external seems to work fine, so that at least raster I/O is
efficient enough; with the one limitation being that GRASS does not
support multi-band rasters in single files (we do not have a good
solution for working around that problem yet).

* of course modules should be upgraded anyway
* GRASS direct digitizing can be skipped; heavy digitizer can use GRASS
directly

(see below)

* keep a GRASS shell?

This only works well with the original GRASS plug-in design, not
with the one derived from SEXTANTE:

Any plug-in design that allows direct manipulation of existing
GRASS data and/or runs GRASS modules on existing mapsets is
not very compatible with the SEXTANTE/Processing on-the-fly
"import/process/export/delete temp" approach. As long as the GRASS
mapset is guaranteed to be temporary, the above chain is clean
and simple. But if you give the plug-in access to existing mapset
data, then you need a nice, thick wrapper of additional safety
checks to make sure that the final "delete" step *never* deletes
any pre-existing user data! Not only would you have to keep track
of each and every bit of data that an intermediate processing step
creates (modules that call other modules!), but also make sure
that no stale data is left in the user's mapset if processing does
not complete.

This is all very, very tricky and normally the role of the GRASS user
who knows about GRASS data management and is in full control.
I.e. if you want to make this safe, then you have to expose the GRASS
mapset and management completely to the user --> and this in turn
defeats the idea of radically simplifying the use of GRASS modules.

This is the reason why in gvSIG CE we steered clear of existing
user mapsets; our conviction being that data safety is always more
important than convenience and that advanced users can/will just run
GRASS by themselves.

Best,

Ben

* with all the above, upgrading the plugin may probably be skipped.

Any thoughts.

--
Dr. Benjamin Ducke, M.A.
{*} Geospatial Consultant
{*} GIS Developer

  benducke@fastmail.fm

16

Il 28/03/2014 15:04, Martin Dobias ha scritto:

On Fri, Mar 28, 2014 at 11:48 AM, Paolo Cavallini <cavallini@faunalia.it> wrote:

* to add GRASS browsing capabilities in QGIS file browser

The support for GRASS is already in the browser - if you enter a
directory that is a GRASS database, it will detect it and show
locations/mapsets/maps/layers.

That's great news, never tried. Works beautifully, thanks.
In the GRASS plugin there are a few more features, i.e.:
* showing history (important)
* copy/rename/delete.
If we can add these, I think the corresponding section of the plugin
would be rather useless.
Am I missing something?
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html

17

Il 28/03/2014 15:16, Paolo Cavallini ha scritto:

Il 28/03/2014 15:04, Martin Dobias ha scritto:

The support for GRASS is already in the browser - if you enter a
directory that is a GRASS database, it will detect it and show
locations/mapsets/maps/layers.

In the GRASS plugin there are a few more features, i.e.:
* showing history (important)
* copy/rename/delete.
If we can add these, I think the corresponding section of the plugin
would be rather useless.
Am I missing something?

What about writing down the minumum set of requirements to be preserved?
Please add it to this thread, we can note it on a wiki page once bolied
down.
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html

18

Hi all,

On Thu, Mar 27, 2014 at 12:18 PM, Markus Neteler <neteler@osgeo.org> wrote:

Hi Paolo,

(note: I'm not ready qgis-dev, not sure if this email reaches it)

On Thu, Mar 27, 2014 at 11:18 AM, Paolo Cavallini <cavallini@faunalia.it> wrote:

Hi all.
I learned during dinner that GRASS7 RC1 is due very soon. This opens the
issue of its functioning in QGIS. IMHO:

* the qgis-grass-plugin might stop working (this has to be tested)
* some of the module options will be different

For the time being people can certainly continue to work with GRASS 6.

I think that the plugin needs to be "cloned" for GRASS 7.

I have submitted the "Processing" for GRASS GIS 7 to Pirmin and Victor
for git upload.
Now the user has GRASS 6 and 7 both supported in Processing.

Cheers from the train,

Markus

19

Il 28/03/2014 15:51, Markus Neteler ha scritto:

I have submitted the "Processing" for GRASS GIS 7 to Pirmin and Victor
for git upload.
Now the user has GRASS 6 and 7 both supported in Processing.

good news, thanks. are you referring to the upgraded and new modules?
all the best.

--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html

20

I updated all existing modules incl the newly included r.stream.*.

A few more new modules might be interesting to add. That's easy...

Markus