[GRASS-dev] access to grass (trunk 7.0.svn ) vector map from python

I’m trying to access a grass vector map with python (2.6). I’m using the svn version (revision 42 565) but keep getting “Segmentation fault” when trying to open the file with Vect_open_old (map, input, mpaset). The code I’m using can be found here: http://osgeo.pastebin.com/VLGunxyq
Can you please tell me I’m doing wrong (the same code works well with python libraries generated with SWIG)?. Greetings. Thank you very much.

Ing. Fernando Pacheco M.S.L.

Ingesur srl
Dirección: Requena 1062 ap.302, Montevideo, Uruguay.
Teléfonos: (+598 2) 408 90 48, (+598 9) 962 79 32
Web: http://ingesur.com.uy/

Fernando Pacheco wrote:

I'm trying to access a grass vector map with python (2.6). I'm using the svn
version (revision 42 565) but keep getting "Segmentation fault" when trying
to open the file with Vect_open_old (map, input, mpaset). The code I'm using
can be found here: http://osgeo.pastebin.com/VLGunxyq
Can you please tell me I'm doing wrong (the same code works well with python
libraries generated with SWIG)?.

There are two problems, one in your code, one in the ctypes wrappers.

First, Vect_open_old needs a pointer to a a Map_info structure, not
the structure itself, so you need to use byref(), e.g.:

  from ctypes import *

  ...

  vector.Vect_open_old(byref(map), input, mapset)

Second, if LFS is enabled, any files which use the vector structures
are compiled with -D_FILE_OFFSET_BITS=64. This wasn't being passed to
the preprocessor when the ctypes wrappers were generated, resulting in
incorrect wrappers being generated for various structures, including
the Map_info structure. This should be fixed by r42566.

--
Glynn Clements <glynn@gclements.plus.com>

Woks!. Thank you. Fernando.

Ing. Fernando Pacheco M.S.L.

Ingesur srl
Dirección: Requena 1062 ap.302, Montevideo, Uruguay.
Teléfonos: (+598 2) 408 90 48, (+598 9) 962 79 32
Web: http://ingesur.com.uy/

On Tue, Jun 15, 2010 at 1:56 PM, Glynn Clements <glynn@gclements.plus.com> wrote:

Fernando Pacheco wrote:

I’m trying to access a grass vector map with python (2.6). I’m using the svn
version (revision 42 565) but keep getting “Segmentation fault” when trying
to open the file with Vect_open_old (map, input, mpaset). The code I’m using
can be found here: http://osgeo.pastebin.com/VLGunxyq
Can you please tell me I’m doing wrong (the same code works well with python
libraries generated with SWIG)?.

There are two problems, one in your code, one in the ctypes wrappers.

First, Vect_open_old needs a pointer to a a Map_info structure, not
the structure itself, so you need to use byref(), e.g.:

from ctypes import *

vector.Vect_open_old(byref(map), input, mapset)

Second, if LFS is enabled, any files which use the vector structures
are compiled with -D_FILE_OFFSET_BITS=64. This wasn’t being passed to
the preprocessor when the ctypes wrappers were generated, resulting in
incorrect wrappers being generated for various structures, including
the Map_info structure. This should be fixed by r42566.


Glynn Clements <glynn@gclements.plus.com>

I modified the code as you have suggested. But I encounter a new problem. If I run a simple program from the command line like this: http://osgeo.pastebin.com/pPqkcvk3 get the following error:
“python: malloc.c:4628: _int_malloc: Assertion `(unsigned long)(size) >= (unsigned long)(nb)’ failed.”
I updated the code to the 42566 version but I still have the same problem. Any suggestions?. I will keep trying!. Thank you. Fernando.

Ing. Fernando Pacheco M.S.L.

Ingesur srl
Dirección: Requena 1062 ap.302, Montevideo, Uruguay.
Teléfonos: (+598 2) 408 90 48, (+598 9) 962 79 32
Web: http://ingesur.com.uy/

On Tue, Jun 15, 2010 at 2:10 PM, Fernando Pacheco <fernando.pacheco@ingesur.com.uy> wrote:

Woks!. Thank you. Fernando.

Ing. Fernando Pacheco M.S.L.

Ingesur srl
Dirección: Requena 1062 ap.302, Montevideo, Uruguay.
Teléfonos: (+598 2) 408 90 48, (+598 9) 962 79 32
Web: http://ingesur.com.uy/

On Tue, Jun 15, 2010 at 1:56 PM, Glynn Clements <glynn@gclements.plus.com> wrote:

Fernando Pacheco wrote:

I’m trying to access a grass vector map with python (2.6). I’m using the svn
version (revision 42 565) but keep getting “Segmentation fault” when trying
to open the file with Vect_open_old (map, input, mpaset). The code I’m using
can be found here: http://osgeo.pastebin.com/VLGunxyq
Can you please tell me I’m doing wrong (the same code works well with python
libraries generated with SWIG)?.

There are two problems, one in your code, one in the ctypes wrappers.

First, Vect_open_old needs a pointer to a a Map_info structure, not
the structure itself, so you need to use byref(), e.g.:

from ctypes import *

vector.Vect_open_old(byref(map), input, mapset)

Second, if LFS is enabled, any files which use the vector structures
are compiled with -D_FILE_OFFSET_BITS=64. This wasn’t being passed to
the preprocessor when the ctypes wrappers were generated, resulting in
incorrect wrappers being generated for various structures, including
the Map_info structure. This should be fixed by r42566.


Glynn Clements <glynn@gclements.plus.com>

Fernando Pacheco wrote:

I modified the code as you have suggested. But I encounter a new problem. If
I run a simple program from the command line like this:
http://osgeo.pastebin.com/pPqkcvk3 get the following error:

"python: malloc.c:4628: _int_malloc: Assertion `(unsigned long)(size) >=
(unsigned long)(nb)' failed."

That's the error which I saw before fixing the wrappers.

I updated the code to the 42566 version but I still have the same problem. Any
suggestions?. I will keep trying!. Thank you. Fernando.

Did you run "make clean" after updating? Makefiles aren't treated as
prerequisites, so an incremental build won't cause the wrappers to be
regenerated.

It should suffice to run:

  make -C lib/python/ctypes clean
  make -C lib/python/ctypes

--
Glynn Clements <glynn@gclements.plus.com>

Yes of course, after updating the source code to the 42 566 version I ran “make clean”, “./configure …”, “make”, "rm-rf / usr/local/grass-7.0.svn " and “make install”. In this situation is that I get the error mentioned in my previous email.
If you have suggestions or an example that works please send me. I need to make a decision about the language used to develop the application (I want to use Python!). Thank you very much. Greetings. Fernando.

Ing. Fernando Pacheco M.S.L.

Ingesur srl
Dirección: Requena 1062 ap.302, Montevideo, Uruguay.
Teléfonos: (+598 2) 408 90 48, (+598 9) 962 79 32
Web: http://ingesur.com.uy/

On Tue, Jun 15, 2010 at 7:45 PM, Glynn Clements <glynn@gclements.plus.com> wrote:

Fernando Pacheco wrote:

I modified the code as you have suggested. But I encounter a new problem. If
I run a simple program from the command line like this:
http://osgeo.pastebin.com/pPqkcvk3 get the following error:

“python: malloc.c:4628: _int_malloc: Assertion `(unsigned long)(size) >=
(unsigned long)(nb)’ failed.”

That’s the error which I saw before fixing the wrappers.

I updated the code to the 42566 version but I still have the same problem. Any
suggestions?. I will keep trying!. Thank you. Fernando.

Did you run “make clean” after updating? Makefiles aren’t treated as
prerequisites, so an incremental build won’t cause the wrappers to be
regenerated.

It should suffice to run:

make -C lib/python/ctypes clean
make -C lib/python/ctypes

Glynn Clements <glynn@gclements.plus.com>

I think there is an error in the generation process with ctypesgen for vector.py. The problem is that “dig_estructs.h” needs the heading “ogr_api.h” which is not found in the path included. The problem is solved if you add the path to that file (in my case -I/usr/include/gdal). Thank you. Greetings. Fernando.

Ing. Fernando Pacheco M.S.L.

Ingesur srl
Dirección: Requena 1062 ap.302, Montevideo, Uruguay.
Teléfonos: (+598 2) 408 90 48, (+598 9) 962 79 32
Web: http://ingesur.com.uy/

On Wed, Jun 16, 2010 at 1:12 AM, Fernando Pacheco <fernando.pacheco@ingesur.com.uy> wrote:

Yes of course, after updating the source code to the 42 566 version I ran “make clean”, “./configure …”, “make”, "rm-rf / usr/local/grass-7.0.svn " and “make install”. In this situation is that I get the error mentioned in my previous email.
If you have suggestions or an example that works please send me. I need to make a decision about the language used to develop the application (I want to use Python!). Thank you very much. Greetings. Fernando.

Ing. Fernando Pacheco M.S.L.

Ingesur srl
Dirección: Requena 1062 ap.302, Montevideo, Uruguay.
Teléfonos: (+598 2) 408 90 48, (+598 9) 962 79 32
Web: http://ingesur.com.uy/

On Tue, Jun 15, 2010 at 7:45 PM, Glynn Clements <glynn@gclements.plus.com> wrote:

Fernando Pacheco wrote:

I modified the code as you have suggested. But I encounter a new problem. If
I run a simple program from the command line like this:
http://osgeo.pastebin.com/pPqkcvk3 get the following error:

“python: malloc.c:4628: _int_malloc: Assertion `(unsigned long)(size) >=
(unsigned long)(nb)’ failed.”

That’s the error which I saw before fixing the wrappers.

I updated the code to the 42566 version but I still have the same problem. Any
suggestions?. I will keep trying!. Thank you. Fernando.

Did you run “make clean” after updating? Makefiles aren’t treated as
prerequisites, so an incremental build won’t cause the wrappers to be
regenerated.

It should suffice to run:

make -C lib/python/ctypes clean
make -C lib/python/ctypes

Glynn Clements <glynn@gclements.plus.com>

Fernando Pacheco wrote:

I think there is an error in the generation process with ctypesgen for
vector.py. The problem is that "dig_estructs.h" needs the heading
"ogr_api.h" which is not found in the path included. The problem is solved
if you add the path to that file (in my case -I/usr/include/gdal).

Try r42567.

--
Glynn Clements <glynn@gclements.plus.com>

Everything works correctly now, the problem has been fixed.
I think the interface generated automatically with ctypesgen (like that of SWIG) isn’t attractive for inexperienced developers. It works very well and is fast, but the calls to the functions are complex (logically identical to those of C) and is not what an average developer would expect in Python.
In the coming months we will be developing an interface for finite element modeling (Fenics project=Dolfin, Tritetmesh, etc.). We will start by making access to vector layers with Python more friendly, taking as a starting point what has been generated with ctypesgen. Thank you. Greetings. Fernando.

Ing. Fernando Pacheco M.S.L.

Ingesur srl
Dirección: Requena 1062 ap.302, Montevideo, Uruguay.
Teléfonos: (+598 2) 408 90 48, (+598 9) 962 79 32
Web: http://ingesur.com.uy/

On Wed, Jun 16, 2010 at 12:13 PM, Glynn Clements <glynn@gclements.plus.com> wrote:

Fernando Pacheco wrote:

I think there is an error in the generation process with ctypesgen for
vector.py. The problem is that “dig_estructs.h” needs the heading
“ogr_api.h” which is not found in the path included. The problem is solved
if you add the path to that file (in my case -I/usr/include/gdal).

Try r42567.

Glynn Clements <glynn@gclements.plus.com>

Fernando Pacheco wrote:

Everything works correctly now, the problem has been fixed.
I think the interface generated automatically with ctypesgen (like that of
SWIG) isn't attractive for inexperienced developers. It works very well and
is fast, but the calls to the functions are complex (logically identical to
those of C) and is not what an average developer would expect in Python.

I don't think that there is a realistic alternative.

If you modify the wrappers, you then have to update the modified
wrappers as the underlying libraries change. I just can't see this
happening.

In the coming months we will be developing an interface for finite element
modeling (Fenics project=Dolfin, Tritetmesh, etc.). We will start by making
access to vector layers with Python more friendly, taking as a starting
point what has been generated with ctypesgen.

Realistically, any higher-level interface is going to have to be built
on top of whatever is generated by ctypesgen, not replace it.

I can't see anyone being willing to specifically maintain
hand-generated wrappers, and we aren't likely to impose a requirement
that anyone modifying a header file must manually modify the
associated Python wrapper to match.

In fact, the lowest levels must be auto-generated to allow for
platform-specific details, e.g. the _FILE_OFFSET_BITS issue.

--
Glynn Clements <glynn@gclements.plus.com>