[GRASS-dev] GSoC Proposal: Python package for topology tools

GRASS GRASS developer
, , ,
1

Hi there!

My name is Facundo Ferrin. I am a nuclear engineer who is taking a master in Computer Vision in Barcelona, and finally I found my opportunity to contribute to OSGeo by applying two things that I really like: Python and Backend development :raised_hands:. I do not know exactly what I should write in this first email, so Iā€™ll start by listing the projects Iā€™m interested in.

Iā€™m working in a company that is developing a platform for precision agriculture called Auravant (https://www.auravant.com/). I work as a backend developer and data analyst and I use daily almost every tool that you post in the ideas: GeoServer, PostGIS, QGis. Iā€™m also porting a tool for polygon simplification called topoJSON (https://github.com/fferrin/topojson).

MY MAIN IDEA is to start porting GRASS tools into a python package that can be used in other projects (beyond the client to use by command line). I donā€™t know if itā€™s something you have in mind but for offline and automated analysis it would be very useful. I particularly had problems when I tried to simplify geometries since the geometry of polygons was not taken into account.

If itā€™s not a viable project at this point, there are two projects that also caught my attention:

I think that maybe I can apply some of the things that I learned from topoJSON for the first project, and things that I learned from the master to the second project. Learn C++ is in my schedule (because for Computer Vision is really important) so it will be really useful for me to program in C++, and in case that the project is hard enough, I could make it in Python (which has support for Qt) and it could be translated into C++ as a GSoC 2020 project.

Hope to hear from you soon!

2

On Tue, 26 Mar 2019 at 03:11, Facundo Ferrin <facundo.ferrin@gmail.com> wrote:

Hi there!

Hi Facundo,

My name is Facundo Ferrin. I am a nuclear engineer who is taking a master in Computer Vision in Barcelona, and finally I found my opportunity to contribute to OSGeo by applying two things that I really like: Python and Backend development . I do not know exactly what I should write in this first email, so I'll start by listing the projects I'm interested in.

I'm working in a company that is developing a platform for precision agriculture called Auravant (https://www.auravant.com/). I work as a backend developer and data analyst and I use daily almost every tool that you post in the ideas: GeoServer, PostGIS, QGis. I'm also porting a tool for polygon simplification called topoJSON (https://github.com/fferrin/topojson).

---
MY MAIN IDEA is to start porting GRASS tools into a python package that can be used in other projects (beyond the client to use by command line). I don't know if it's something you have in mind but for offline and automated analysis it would be very useful. I particularly had problems when I tried to simplify geometries since the geometry of polygons was not taken into account.
---

Your idea is not clear to me, there are already two Python library to
work with GRASS. you can find some ideas in the proposal page
https://trac.osgeo.org/grass/wiki/GSoC/2019 (for example
Neweasy-to-useCLIandAPIforGRASSGIS) and
https://trac.osgeo.org/grass/wiki/GSoC/2018 (Improve GRASS integration
in QGIS 3)

Hope to hear from you soon!

--
ciao
Luca

www.lucadelu.org

3

Hi Luca!

Thanks for replying! In my job, there were things we had to do programmatically. For example, to manipulate geometries that reach the backend from a GeoJSON we use tools like these:

https://pcjericks.github.io/py-gdalogr-cookbook/geometry.html#create-geometry-from-wkt

However, polygon simplification does not work very well because it does not take topology into account. My idea was to port part of the GRASS algorithms to be able to use them without needing the graphical interface or command line, but only importing a library in a Python script.

Is it something that you have in mind to do or that might be useful to you?

El jue., 28 de mar. de 2019 a la(s) 00:32, Luca Delucchi (lucadeluge@gmail.com) escribiĆ³:

On Tue, 26 Mar 2019 at 03:11, Facundo Ferrin <facundo.ferrin@gmail.com> wrote:

Hi there!

Hi Facundo,

My name is Facundo Ferrin. I am a nuclear engineer who is taking a master in Computer Vision in Barcelona, and finally I found my opportunity to contribute to OSGeo by applying two things that I really like: Python and Backend development . I do not know exactly what I should write in this first email, so Iā€™ll start by listing the projects Iā€™m interested in.

Iā€™m working in a company that is developing a platform for precision agriculture called Auravant (https://www.auravant.com/). I work as a backend developer and data analyst and I use daily almost every tool that you post in the ideas: GeoServer, PostGIS, QGis. Iā€™m also porting a tool for polygon simplification called topoJSON (https://github.com/fferrin/topojson).

MY MAIN IDEA is to start porting GRASS tools into a python package that can be used in other projects (beyond the client to use by command line). I donā€™t know if itā€™s something you have in mind but for offline and automated analysis it would be very useful. I particularly had problems when I tried to simplify geometries since the geometry of polygons was not taken into account.

Your idea is not clear to me, there are already two Python library to
work with GRASS. you can find some ideas in the proposal page
https://trac.osgeo.org/grass/wiki/GSoC/2019 (for example
Neweasy-to-useCLIandAPIforGRASSGIS) and
https://trac.osgeo.org/grass/wiki/GSoC/2018 (Improve GRASS integration
in QGIS 3)

Hope to hear from you soon!

ā€“
ciao
Luca

www.lucadelu.org

4

Hello Facundo,
the easiest way would be moving functions of v.generalize into a
library (e.g. grass_generalize) and thus make available for calling
via ctypes.
In the past I have had a good success manipulating GRASS vectors via
ctypes. It takes more skill than a plain Python implementation but it
is easier than a full blown C code and faster than pure Python one.

Māris.

ceturtd., 2019. g. 28. marts, plkst. 03:13 ā€” lietotājs Facundo Ferrin
(<facundo.ferrin@gmail.com>) rakstīja:

Hi Luca!

Thanks for replying! In my job, there were things we had to do programmatically. For example, to manipulate geometries that reach the backend from a GeoJSON we use tools like these:

https://pcjericks.github.io/py-gdalogr-cookbook/geometry.html#create-geometry-from-wkt

However, polygon simplification does not work very well because it does not take topology into account. My idea was to port part of the GRASS algorithms to be able to use them without needing the graphical interface or command line, but only importing a library in a Python script.

Is it something that you have in mind to do or that might be useful to you?

El jue., 28 de mar. de 2019 a la(s) 00:32, Luca Delucchi (lucadeluge@gmail.com) escribiĆ³:

On Tue, 26 Mar 2019 at 03:11, Facundo Ferrin <facundo.ferrin@gmail.com> wrote:
>
> Hi there!

Hi Facundo,
>
>
> My name is Facundo Ferrin. I am a nuclear engineer who is taking a master in Computer Vision in Barcelona, and finally I found my opportunity to contribute to OSGeo by applying two things that I really like: Python and Backend development . I do not know exactly what I should write in this first email, so I'll start by listing the projects I'm interested in.
>
> I'm working in a company that is developing a platform for precision agriculture called Auravant (https://www.auravant.com/). I work as a backend developer and data analyst and I use daily almost every tool that you post in the ideas: GeoServer, PostGIS, QGis. I'm also porting a tool for polygon simplification called topoJSON (https://github.com/fferrin/topojson).
>
> ---
> MY MAIN IDEA is to start porting GRASS tools into a python package that can be used in other projects (beyond the client to use by command line). I don't know if it's something you have in mind but for offline and automated analysis it would be very useful. I particularly had problems when I tried to simplify geometries since the geometry of polygons was not taken into account.
> ---

Your idea is not clear to me, there are already two Python library to
work with GRASS. you can find some ideas in the proposal page
https://trac.osgeo.org/grass/wiki/GSoC/2019 (for example
Neweasy-to-useCLIandAPIforGRASSGIS) and
https://trac.osgeo.org/grass/wiki/GSoC/2018 (Improve GRASS integration
in QGIS 3)

> Hope to hear from you soon!
>

--
ciao
Luca

www.lucadelu.org

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

5

On Thu, Mar 28, 2019 at 9:02 AM Maris Nartiss <maris.gis@gmail.com> wrote:

Hello Facundo,
the easiest way would be moving functions of v.generalize into a
library (e.g. grass_generalize) and thus make available for calling
via ctypes.
In the past I have had a good success manipulating GRASS vectors via
ctypes. It takes more skill than a plain Python implementation but it
is easier than a full blown C code and faster than pure Python one.

Māris.

ceturtd., 2019. g. 28. marts, plkst. 03:13 ā€” lietotājs Facundo Ferrin
(<facundo.ferrin@gmail.com>) rakstīja:

Hi Luca!

Thanks for replying! In my job, there were things we had to do programmatically. For example, to manipulate geometries that reach the backend from a GeoJSON we use tools like these:

https://pcjericks.github.io/py-gdalogr-cookbook/geometry.html#create-geometry-from-wkt

However, polygon simplification does not work very well because it does not take topology into account. My idea was to port part of the GRASS algorithms to be able to use them without needing the graphical interface or command line, but only importing a library in a Python script.

In this particular case, the core of the corresponding python script would be three lines (import, simplify, export):

ā†’

import grass.script as grass

grass.run_command(ā€˜v.in.ogrā€™, ā€¦)

grass.run_command(ā€˜v.generalizeā€™, ā€¦)

grass.run_command(ā€˜v.out.ogrā€™, ā€¦)
ā†

The import step with v.in.ogr is needed because the vector to be simplified must be a native GRASS vector with topology.

How does your proposal differ from the QGIS-GRASS interface?

Markus M

Is it something that you have in mind to do or that might be useful to you?

El jue., 28 de mar. de 2019 a la(s) 00:32, Luca Delucchi (lucadeluge@gmail.com) escribiĆ³:

On Tue, 26 Mar 2019 at 03:11, Facundo Ferrin <facundo.ferrin@gmail.com> wrote:

Hi there!

Hi Facundo,

My name is Facundo Ferrin. I am a nuclear engineer who is taking a master in Computer Vision in Barcelona, and finally I found my opportunity to contribute to OSGeo by applying two things that I really like: Python and Backend development . I do not know exactly what I should write in this first email, so Iā€™ll start by listing the projects Iā€™m interested in.

Iā€™m working in a company that is developing a platform for precision agriculture called Auravant (https://www.auravant.com/). I work as a backend developer and data analyst and I use daily almost every tool that you post in the ideas: GeoServer, PostGIS, QGis. Iā€™m also porting a tool for polygon simplification called topoJSON (https://github.com/fferrin/topojson).

MY MAIN IDEA is to start porting GRASS tools into a python package that can be used in other projects (beyond the client to use by command line). I donā€™t know if itā€™s something you have in mind but for offline and automated analysis it would be very useful. I particularly had problems when I tried to simplify geometries since the geometry of polygons was not taken into account.

Your idea is not clear to me, there are already two Python library to
work with GRASS. you can find some ideas in the proposal page
https://trac.osgeo.org/grass/wiki/GSoC/2019 (for example
Neweasy-to-useCLIandAPIforGRASSGIS) and
https://trac.osgeo.org/grass/wiki/GSoC/2018 (Improve GRASS integration
in QGIS 3)

Hope to hear from you soon!

ā€“
ciao
Luca

www.lucadelu.org

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

6

Hello Facundo,

Using the GRASS GIS algorithms in various settings is certainly something we are aiming for. However, as you probably understood from the conversation here, it is not clear what exactly you are trying to achieve considering the already available ways (APIs if you will). Also, to evaluate your proposal, we need to test your skills for this particular project. Therefore, I suggest you 1) implement a prototype of what you have in mind, 2) compare it with the existing approaches (see e.g. email from Markus M.), and 3) identify what is missing in the existing approaches and what would you need to add in order to make you prototype work.

Alternatively, or ideally in addition to the above, you can implement couple of the test and training tasks from the proposal already linked by Luca:

https://trac.osgeo.org/grass/wiki/GSoC/2019#Neweasy-to-useCLIandAPIforGRASSGIS

Best,
Vaclav

On Thu, Mar 28, 2019 at 4:13 AM Markus Metz <markus.metz.giswork@gmail.com> wrote:

On Thu, Mar 28, 2019 at 9:02 AM Maris Nartiss <maris.gis@gmail.com> wrote:

Hello Facundo,
the easiest way would be moving functions of v.generalize into a
library (e.g. grass_generalize) and thus make available for calling
via ctypes.
In the past I have had a good success manipulating GRASS vectors via
ctypes. It takes more skill than a plain Python implementation but it
is easier than a full blown C code and faster than pure Python one.

Māris.

ceturtd., 2019. g. 28. marts, plkst. 03:13 ā€” lietotājs Facundo Ferrin
(<facundo.ferrin@gmail.com>) rakstīja:

Hi Luca!

Thanks for replying! In my job, there were things we had to do programmatically. For example, to manipulate geometries that reach the backend from a GeoJSON we use tools like these:

https://pcjericks.github.io/py-gdalogr-cookbook/geometry.html#create-geometry-from-wkt

However, polygon simplification does not work very well because it does not take topology into account. My idea was to port part of the GRASS algorithms to be able to use them without needing the graphical interface or command line, but only importing a library in a Python script.

In this particular case, the core of the corresponding python script would be three lines (import, simplify, export):

ā†’

import grass.script as grass

grass.run_command(ā€˜v.in.ogrā€™, ā€¦)

grass.run_command(ā€˜v.generalizeā€™, ā€¦)

grass.run_command(ā€˜v.out.ogrā€™, ā€¦)
ā†

The import step with v.in.ogr is needed because the vector to be simplified must be a native GRASS vector with topology.

How does your proposal differ from the QGIS-GRASS interface?

Markus M

Is it something that you have in mind to do or that might be useful to you?

El jue., 28 de mar. de 2019 a la(s) 00:32, Luca Delucchi (lucadeluge@gmail.com) escribiĆ³:

On Tue, 26 Mar 2019 at 03:11, Facundo Ferrin <facundo.ferrin@gmail.com> wrote:

Hi there!

Hi Facundo,

My name is Facundo Ferrin. I am a nuclear engineer who is taking a master in Computer Vision in Barcelona, and finally I found my opportunity to contribute to OSGeo by applying two things that I really like: Python and Backend development . I do not know exactly what I should write in this first email, so Iā€™ll start by listing the projects Iā€™m interested in.

Iā€™m working in a company that is developing a platform for precision agriculture called Auravant (https://www.auravant.com/). I work as a backend developer and data analyst and I use daily almost every tool that you post in the ideas: GeoServer, PostGIS, QGis. Iā€™m also porting a tool for polygon simplification called topoJSON (https://github.com/fferrin/topojson).

MY MAIN IDEA is to start porting GRASS tools into a python package that can be used in other projects (beyond the client to use by command line). I donā€™t know if itā€™s something you have in mind but for offline and automated analysis it would be very useful. I particularly had problems when I tried to simplify geometries since the geometry of polygons was not taken into account.

Your idea is not clear to me, there are already two Python library to
work with GRASS. you can find some ideas in the proposal page
https://trac.osgeo.org/grass/wiki/GSoC/2019 (for example
Neweasy-to-useCLIandAPIforGRASSGIS) and
https://trac.osgeo.org/grass/wiki/GSoC/2018 (Improve GRASS integration
in QGIS 3)

Hope to hear from you soon!

ā€“
ciao
Luca

www.lucadelu.org

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

7

On Mon, Apr 22, 2019 at 4:45 AM Facundo FerrĆ­n <facundo.ferrin@gmail.com> wrote:

Hello!

I donā€™t know if Iā€™m expressing myself well. The use of GRASS is command-line, in the sense:

rass run --mapset=/some/directory/grassdata/ncspm/practice1 r.lake elevation=some/file.tiff

What I propose is the use of a package to import from any Python script. For example, read a TIFF with GDAL or GeoPandas, perform some operations like removing some data and finally simplify the polygon.

GRASS is essentially a toolset consisting of several hundred modules which are called on the command line or in scripts. If you want to use this functionality, you need to call GRASS modules as done in e.g.

https://trac.osgeo.org/grass/browser/grass/trunk/scripts

GRASS is something like GDALā€™s option to create polygons from images:

https://gdal.org/gdal_polygonize.html

The GRASS equivalent is ā€˜r.to.vect type=areaā€™

My idea, or my need, is to use GRASS functions programmatically, as in this case.

https://pcjericks.github.io/py-gdalogr-cookbook/raster_layers.html#polygonize-a-raster-band

Convert an OGR File to a Raster
more generic: convert a vector to a raster

The GRASS equivalent is ā€˜v.to.rastā€™

Clip a GeoTiff with Shapefile
more generic: clip a raster to vector polygons
In GRASS: convert to vector polygons to raster with ā€˜v.to.rastā€™, then create a MASK with ā€˜r.maskā€™

Calculate zonal statistics
more precisely here: get raster statistics for each vector polygon

The GRASS equivalent is ā€˜v.rast.statsā€™

Raster to vector line
In GRASS: use ā€˜r.reclassā€™ or ā€˜r.mapcalcā€™ to select pixel values that need to be converted to vector lines, then use r.to.vect type=line. BTW, the green lines in the example are wrong because they don#t go through the pixel centers.

Create raster from array

The GRASS equivalent is ā€˜r.in.asciiā€™

Create least cost path

The GRASS equivalent is ā€˜r.costā€™ + ā€˜r.pathā€™

Replace No Data Value of Raster with new value

The GRASS equivalent is ā€˜r.mapcalcā€™ using ā€œif(isnull(rastermap), <new_value>, rastermap)ā€

Is that among your ideas for the future? If thatā€™s the case and youā€™re interested, let me know. If so, you could make some prototype use to start migrating the functionalities.

All this can be easily implemented as a GRASS GIS addon in a Python script with much less code than in your example. Please have a look at
https://trac.osgeo.org/grass/browser/grass/trunk/scripts
to get started.

Alternatively, a QGIS plugin using GRASS GIS functionality would be an option (QGIS has a GRASS GIS interface that can be used for inspiration).

my2c,
Markus M

Cheers!

On 10/04/2019 16:47, Vaclav Petras wrote:

Hello Facundo,

Using the GRASS GIS algorithms in various settings is certainly something we are aiming for. However, as you probably understood from the conversation here, it is not clear what exactly you are trying to achieve considering the already available ways (APIs if you will). Also, to evaluate your proposal, we need to test your skills for this particular project. Therefore, I suggest you 1) implement a prototype of what you have in mind, 2) compare it with the existing approaches (see e.g. email from Markus M.), and 3) identify what is missing in the existing approaches and what would you need to add in order to make you prototype work.

Alternatively, or ideally in addition to the above, you can implement couple of the test and training tasks from the proposal already linked by Luca:

https://trac.osgeo.org/grass/wiki/GSoC/2019#Neweasy-to-useCLIandAPIforGRASSGIS

Best,
Vaclav

On Thu, Mar 28, 2019 at 4:13 AM Markus Metz <markus.metz.giswork@gmail.com> wrote:

On Thu, Mar 28, 2019 at 9:02 AM Maris Nartiss <maris.gis@gmail.com> wrote:

Hello Facundo,
the easiest way would be moving functions of v.generalize into a
library (e.g. grass_generalize) and thus make available for calling
via ctypes.
In the past I have had a good success manipulating GRASS vectors via
ctypes. It takes more skill than a plain Python implementation but it
is easier than a full blown C code and faster than pure Python one.

Māris.

ceturtd., 2019. g. 28. marts, plkst. 03:13 ā€” lietotājs Facundo Ferrin
(<facundo.ferrin@gmail.com>) rakstīja:

Hi Luca!

Thanks for replying! In my job, there were things we had to do programmatically. For example, to manipulate geometries that reach the backend from a GeoJSON we use tools like these:

https://pcjericks.github.io/py-gdalogr-cookbook/geometry.html#create-geometry-from-wkt

However, polygon simplification does not work very well because it does not take topology into account. My idea was to port part of the GRASS algorithms to be able to use them without needing the graphical interface or command line, but only importing a library in a Python script.

In this particular case, the core of the corresponding python script would be three lines (import, simplify, export):

ā†’
import grass.script as grass

grass.run_command(ā€˜v.in.ogrā€™, ā€¦)
grass.run_command(ā€˜v.generalizeā€™, ā€¦)
grass.run_command(ā€˜v.out.ogrā€™, ā€¦)
ā†

The import step with v.in.ogr is needed because the vector to be simplified must be a native GRASS vector with topology.

How does your proposal differ from the QGIS-GRASS interface?

Markus M

Is it something that you have in mind to do or that might be useful to you?

El jue., 28 de mar. de 2019 a la(s) 00:32, Luca Delucchi (lucadeluge@gmail.com) escribiĆ³:

On Tue, 26 Mar 2019 at 03:11, Facundo Ferrin <facundo.ferrin@gmail.com> wrote:

Hi there!

Hi Facundo,

My name is Facundo Ferrin. I am a nuclear engineer who is taking a master in Computer Vision in Barcelona, and finally I found my opportunity to contribute to OSGeo by applying two things that I really like: Python and Backend development . I do not know exactly what I should write in this first email, so Iā€™ll start by listing the projects Iā€™m interested in.

Iā€™m working in a company that is developing a platform for precision agriculture called Auravant (https://www.auravant.com/). I work as a backend developer and data analyst and I use daily almost every tool that you post in the ideas: GeoServer, PostGIS, QGis. Iā€™m also porting a tool for polygon simplification called topoJSON (https://github.com/fferrin/topojson).

MY MAIN IDEA is to start porting GRASS tools into a python package that can be used in other projects (beyond the client to use by command line). I donā€™t know if itā€™s something you have in mind but for offline and automated analysis it would be very useful. I particularly had problems when I tried to simplify geometries since the geometry of polygons was not taken into account.

Your idea is not clear to me, there are already two Python library to
work with GRASS. you can find some ideas in the proposal page
https://trac.osgeo.org/grass/wiki/GSoC/2019 (for example
Neweasy-to-useCLIandAPIforGRASSGIS) and
https://trac.osgeo.org/grass/wiki/GSoC/2018 (Improve GRASS integration
in QGIS 3)

Hope to hear from you soon!

ā€“
ciao
Luca

www.lucadelu.org

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev