Martin Landa wrote:
D1/5: Vect__open_old(): name = (null) mapset = (null) layer= (null) update = 0
Segmentation fault
"(null)" is what libc's printf("%s") prints for a null pointer.
This is a different problem to your original report.
--
Glynn Clements <glynn@gclements.plus.com>
Hi,
2011/7/23 Glynn Clements <glynn@gclements.plus.com>:
D1/5: Vect__open_old(): name = (null) mapset = (null) layer= (null) update = 0
Segmentation fault"(null)" is what libc's printf("%s") prints for a null pointer.
This is a different problem to your original report.
sorry, small bug in my sample test. Here is a new attempt - sample script:
"""
#!/usr/bin/python
from grass.lib.gis import *
from grass.lib.vector import *
map_obj = Map_info()
map = pointer(map_obj)
print os.environ['PYTHONPATH']
print sizeof(off_t)
Vect_open_update(map, "a", "")
p = Vect_new_line_struct()
c = Vect_new_cats_struct()
Vect_append_point(p, 1, 1, 0)
Vect_cat_set(c, 1, 1)
print p.contents.n_points
Vect_write_line(map, GV_POINT, p, c)
Vect_build(map)
Vect_close(map)
"""
My system is Debian GNU/Linux unstable 32bit, the script works when
GRASS compiled without LFS, with LFS it fails.
/usr/local/src/grass_trunk/dist.i686-pc-linux-gnu/etc/python
8
/usr/local/src/grass_trunk/dist.i686-pc-linux-gnu/etc/python
8
D1/5: Vect__open_old(): name = a mapset = layer= (null) update = 1
D1/5: Vect_set_thresh(): thresh = 0.000000
D3/5: dig_init_plus()
D3/5: dig_cidx_init()
D1/5: G_find_vector2(): name=a mapset=
D1/5: Map name: a
D1/5: Map mapset: pg
D1/5: open format file: 'pg/vector/a/frmt'
D1/5: Vector format: 0 (native)
D2/5: G__read_Cell_head
D2/5: G__read_Cell_head_array
....
D1/5: Vect_open_sidx(): name = a mapset= pg mode = update
D1/5: get coor info: /home/martin/grassdata/nc_spm_08/pg/vector/a/coor
D1/5: Info->size = 134, Info->mtime = 1311408989
D2/5: Spidx header: file version 5.1 , supported from GRASS version 5.1
D2/5: byte order 0
D2/5: header size 113
D2/5: with_z 0
D2/5: coor size 134
D1/5: Sidx head: coor size = 134, coor mtime = 0
D1/5: Memory based spatial index
D1/5: dig_read_spindx()
D2/5: Spidx header: file version 5.1 , supported from GRASS version 5.1
D2/5: byte order 0
D2/5: header size 113
D2/5: with_z 0
D2/5: coor size 134
*** glibc detected *** /usr/bin/python: malloc(): memory corruption:
0x09ba9fe8 ***
It indicates other problem than reported originally.
Martin
--
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa
Martin Landa wrote:
sorry, small bug in my sample test. Here is a new attempt - sample script:
#!/usr/bin/python
from grass.lib.gis import *
from grass.lib.vector import *map_obj = Map_info()
map = pointer(map_obj)print os.environ['PYTHONPATH']
print sizeof(off_t)
Vect_open_update(map, "a", "")
...
*** glibc detected *** /usr/bin/python: malloc(): memory corruption:
0x09ba9fe8 ***
Does calling G_gisinit() first change anything?
--
Glynn Clements <glynn@gclements.plus.com>
Hi,
2011/7/23 Glynn Clements <glynn@gclements.plus.com>:
[...]
*** glibc detected *** /usr/bin/python: malloc(): memory corruption:
0x09ba9fe8 ***Does calling G_gisinit() first change anything?
unfortunately it didn't help, still corrupted memory, now
D1/5: Vect__open_old(): name = a mapset = layer= (null) update = 1
...
D1/5: Vect_Rewind(): name = a
D3/5: Vect_write_line(): name = �$��`�/`�, format = 155237412, level = 1
Martin
--
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa
Martin Landa wrote:
Hi,
2011/7/23 Glynn Clements <glynn@gclements.plus.com>:
[...]
>> *** glibc detected *** /usr/bin/python: malloc(): memory corruption:
>> 0x09ba9fe8 ***
>
> Does calling G_gisinit() first change anything?unfortunately it didn't help, still corrupted memory, now
I think I've found it. Here, vector.py has:
# /usr/include/bits/types.h: 62
class struct_anon_1(Structure):
pass
struct_anon_1.__slots__ = [
'__val',
]
struct_anon_1._fields_ = [
('__val', c_long * 2),
]
__quad_t = struct_anon_1 # /usr/include/bits/types.h: 62
...
__off64_t = __quad_t # /usr/include/bits/types.h: 142
...
off_t = __off64_t # /usr/include/stdio.h: 93
Vect_write_line() returns an off_t. With the above definitions, this
is a structure containing a pair of 32-bit integers.
On Linux-x86, structure return values are implemented by having the
caller pass a pointer to where to store the return value as the first
(zeroth?) argument.
If I add:
Vect_write_line.restype = c_longlong
then the script works.
The <bits/types.h> header file defines __quad_t as:
/* quad_t is also 64 bits. */
#if __WORDSIZE == 64
typedef long int __quad_t;
typedef unsigned long int __u_quad_t;
#elif defined __GLIBC_HAVE_LONG_LONG
__extension__ typedef long long int __quad_t;
__extension__ typedef unsigned long long int __u_quad_t;
#else
typedef struct
{
long __val[2];
} __quad_t;
typedef struct
{
__u_long __val[2];
} __u_quad_t;
#endif
I.e. as "long int" if that type is 64-bit, else "long long int" if
__GLIBC_HAVE_LONG_LONG is defined, else the structure.
__GLIBC_HAVE_LONG_LONG is defined in <features.h> as:
/* Decide whether a compiler supports the long long datatypes. */
#if defined __GNUC__ \
|| (defined __PGI && defined __i386__ ) \
|| (defined __INTEL_COMPILER && (defined __i386__ || defined __ia64__)) \
|| (defined __STDC_VERSION__ && __STDC_VERSION__ >= 199901L)
# define __GLIBC_HAVE_LONG_LONG 1
#endif
And ctypesgen.py specifically undefines __GNUC__ when running the
preprocessor. (__STDC_VERSION__ is only defined if you use e.g.
-std=c99 or -std=gnu99, not for C89)
This causes ctypes to pass an incorrect argument list for any function
which returns a 64-bit off_t, and Vect_write_line() is just such a
function.
The simplest fix is probably to add -D__GLIBC_HAVE_LONG_LONG to
CTYPESFLAGS in lib/python/ctypes/Makefile. That macro is unlikely to
be understood by anything other than glibc, and any system using glibc
is almost certain to be using a compiler which supports "long long".
However, I wouldn't be surprised if there are other situations where
undefining __GNUC__ has consequences for the ABI, but we'll just have
to deal with those as we find them.
--
Glynn Clements <glynn@gclements.plus.com>
Hi,
2011/7/28 Glynn Clements <glynn@gclements.plus.com>:
[...]
The simplest fix is probably to add -D__GLIBC_HAVE_LONG_LONG to
CTYPESFLAGS in lib/python/ctypes/Makefile. That macro is unlikely to
be understood by anything other than glibc, and any system using glibc
is almost certain to be using a compiler which supports "long long".
I modified Makefile
CTYPESFLAGS = --cpp "$(CC) -E $(CPPFLAGS) $(LFS_CFLAGS)
$(EXTRA_CFLAGS) $(NLS_CFLAGS) $(DEFS) $(EXTRA_INC) $(INC)
-D__GLIBC_HAVE_LONG_LONG"
after recompilation I can see in vector.py
__quad_t = c_longlong # /usr/include/i386-linux-gnu/bits/types.h: 56
__off_t = c_long # /usr/include/i386-linux-gnu/bits/types.h: 141
__off64_t = __quad_t # /usr/include/i386-linux-gnu/bits/types.h: 142
Unfortunately the sample script still fails
...
D1/5: Vect_open_sidx(): name = a mapset= landa mode = update
D1/5: get coor info: /home/martin/grassdata/nc_spm_08/landa/vector/a/coor
D1/5: Info->size = 76, Info->mtime = 1311887704
D2/5: Spidx header: file version 5.1 , supported from GRASS version 5.1
D2/5: byte order 0
D2/5: header size 113
D2/5: with_z 0
D2/5: coor size 76
D1/5: Sidx head: coor size = 76, coor mtime = 0
D1/5: Memory based spatial index
D1/5: dig_read_spindx()
D2/5: Spidx header: file version 5.1 , supported from GRASS version 5.1
D2/5: byte order 0
D2/5: header size 113
D2/5: with_z 0
D2/5: coor size 76
*** glibc detected *** python: malloc(): memory corruption: 0x086b5908 ***
Any idea where could be a problem?
Thanks, Martin
--
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa
Martin Landa wrote:
Unfortunately the sample script still fails
I can't reproduce this. Running it under GDB might help, or it might
not; memory corruption is like that.
--
Glynn Clements <glynn@gclements.plus.com>
Same here. Provided sample script runs just fine. Does it need some
special settings (Location, DB connection)?
Gentoo ~AMD64
gcc version 4.5.2 (Gentoo 4.5.2 p1.0, pie-0.4.5)
Python 2.7.2
Maris.
2011/7/29 Glynn Clements <glynn@gclements.plus.com>:
Martin Landa wrote:
Unfortunately the sample script still fails
I can't reproduce this. Running it under GDB might help, or it might
not; memory corruption is like that.--
Glynn Clements <glynn@gclements.plus.com>
Hi,
2011/7/29 Glynn Clements <glynn@gclements.plus.com>:
I can't reproduce this. Running it under GDB might help, or it might
not; memory corruption is like that.
after pressing Ctrl+C
Program received signal SIGINT, Interrupt.
0xb7fe2424 in __kernel_vsyscall ()
bt
#0 0xb7fe2424 in __kernel_vsyscall ()
#1 0xb7ceddb1 in __lll_lock_wait_private () at
../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/lowlevellock.S:95
#2 0xb7c84ba7 in _L_lock_9661 () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
#3 0xb7c833d6 in __libc_free (mem=0x8311ac0) at malloc.c:3736
#4 0xb7ff444f in ?? () from /lib/ld-linux.so.2
#5 0xb7fefab1 in ?? () from /lib/ld-linux.so.2
#6 0xb7ff4e21 in ?? () from /lib/ld-linux.so.2
#7 0xb7ff0b46 in ?? () from /lib/ld-linux.so.2
#8 0xb7ff4866 in ?? () from /lib/ld-linux.so.2
#9 0xb7d17972 in do_dlopen (ptr=0xbfff52c8) at dl-libc.c:86
#10 0xb7ff0b46 in ?? () from /lib/ld-linux.so.2
#11 0xb7d17a71 in dlerror_run (operate=<value optimized out>,
args=0x839d4b0) at dl-libc.c:47
#12 0xb7d17b8b in __libc_dlopen_mode (name=0xb7d44c3d "libgcc_s.so.1",
mode=-2147483647) at dl-libc.c:160
#13 0xb7cf55b8 in init () at ../sysdeps/i386/backtrace.c:44
#14 0xb7fb885e in pthread_once () at
../nptl/sysdeps/unix/sysv/linux/i386/pthread_once.S:122
#15 0xb7cf57ad in __backtrace (array=0xbfff589c, size=64) at
../sysdeps/i386/backtrace.c:121
#16 0xb7c749eb in __libc_message (do_abort=2, fmt=0xb7d49a90 "***
glibc detected *** %s: %s: 0x%s ***\n")
at ../sysdeps/unix/sysv/linux/libc_fatal.c:168
#17 0xb7c7eac1 in malloc_printerr (action=<value optimized out>,
str=0x8311ab8 "", ptr=0x842b918) at malloc.c:6283
#18 0xb7c818a4 in _int_malloc (av=<value optimized out>, bytes=<value
optimized out>) at malloc.c:4396
#19 0xb7c834ac in __libc_malloc (bytes=476) at malloc.c:3660
#20 0xb5fc92b6 in RTreeNewNode (t=0x8474ba0, level=0) at node.c:76
#21 0xb5fe3ecd in rtree_load_to_memory (fp=0x842dcf4, rootpos=589,
t=0x8474ba0, off_t_size=4) at spindex_rw.c:965
#22 0xb5fe481d in rtree_load_from_sidx (fp=0x842dcf4, rootpos=589,
t=0x8474ba0, off_t_size=4) at spindex_rw.c:1123
#23 0xb5fe4aac in dig_Rd_spidx (fp=0x842dcf4, Plus=0x842dafc) at
spindex_rw.c:1201
#24 0xb60520a1 in Vect_open_sidx (Map=0x842daf0, mode=1) at open.c:1104
#25 0xb6050830 in Vect__open_old (Map=0x842daf0, name=0xb7b95b54 "a",
mapset=0xb7bd30c4 "", layer=0x0, update=1, head_only=0) at open.c:312
#26 0xb60510c2 in Vect_open_update (Map=0x842daf0, name=0xb7b95b54
"a", mapset=0xb7bd30c4 "") at open.c:558
#27 0xb7868baf in ffi_call_SYSV () from
/usr/lib/python2.6/lib-dynload/_ctypes.so
#28 0xb786894e in ffi_call () from /usr/lib/python2.6/lib-dynload/_ctypes.so
#29 0xb7863622 in _CallProc () from /usr/lib/python2.6/lib-dynload/_ctypes.so
#30 0xb785abdd in ?? () from /usr/lib/python2.6/lib-dynload/_ctypes.so
#31 0x08066efd in PyObject_Call ()
#32 0x080d85e9 in PyEval_EvalFrameEx ()
#33 0x080dbb27 in PyEval_EvalCodeEx ()
#34 0x080dbc37 in PyEval_EvalCode ()
#35 0x080fb026 in PyRun_FileExFlags ()
#36 0x080fbdc6 in PyRun_SimpleFileExFlags ()
#37 0x0805e4e2 in Py_Main ()
#38 0x0805d6bb in main ()
It seems to crash at [1]. In summary, the sample script works on
* my laptop (Debian GNU/Linux unstable 32bit, gcc 4.6.1, python 2.6.7)
- GRASS compiled *without* LFS
and fails on
* testing server (Debian GNU/Linux stable 64bit, gcc 4.4.5, python
2.6.6) - GRASS compiled *with* LFS
...
D1/5: Vect_Rewind(): name = a
D3/5: Vect_write_line(): name = (null), format = 152120884, level = 0
ERROR: Unable to write feature, vector map is not opened
* my computer (Debian GNU/Linux unstable 32bit, gcc 4.6.1, python
2.6.7) - GRASS compiled *with* LFS
1/5: Vect_open_sidx(): name = a mapset= landa mode = update
D1/5: get coor info: /home/martin/grassdata/nc_spm_08/landa/vector/a/coor
D1/5: Info->size = 105, Info->mtime = 1312193976
D2/5: Spidx header: file version 5.1 , supported from G4.4.5RASS version 5.1
D2/5: byte order 0
D2/5: header size 113
D2/5: with_z 0
D2/5: coor size 105
D1/5: Sidx head: coor size = 105, coor mtime = 0
D1/5: Memory based spatial index
D1/5: dig_read_spindx()
D2/5: Spidx header: file version 5.1 , supported from GRASS version 5.1
D2/5: byte order 0
D2/5: header size 113
D2/5: with_z 0
D2/5: coor size 105
*** glibc detected *** /usr/bin/python: malloc(): memory corruption:
0x0976e950 ***
Martin
[1] http://trac.osgeo.org/grass/browser/grass/trunk/lib/vector/rtree/node.c#L76
--
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa
Hi,
2011/7/30 Maris Nartiss <maris.gis@gmail.com>:
Same here. Provided sample script runs just fine. Does it need some
special settings (Location, DB connection)?Gentoo ~AMD64
gcc version 4.5.2 (Gentoo 4.5.2 p1.0, pie-0.4.5)
Python 2.7.2
compiled with LFS?
Martin
--
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa
Martin Landa wrote:
> I can't reproduce this. Running it under GDB might help, or it might
> not; memory corruption is like that.
It seems to crash at [1].
That's where it trips over the memory corruption, but it doesn't tell
us where it happened. Printing the contents of the various structures
(Map_info, Plus_head, etc) might provide some clues.
In summary, the sample script works on
* my laptop (Debian GNU/Linux unstable 32bit, gcc 4.6.1, python 2.6.7)
- GRASS compiled *without* LFSand fails on
* testing server (Debian GNU/Linux stable 64bit, gcc 4.4.5, python
2.6.6) - GRASS compiled *with* LFS
LFS should be a no-op on a 64-bit platform, i.e. off_t should always
be 64-bit.
FWIW, it works for me on a 32-bit platform with LFS enabled.
Have you tried writing an equivalent C program and seeing whether that
works? We might be looking at a bug which has nothing to do with
Python or ctypes. As the script is much simpler than a typical v.* C
program, it's possible that some initialisation is being skipped.
--
Glynn Clements <glynn@gclements.plus.com>
Here's my configuration call:
./configure --enable-debug --with-nls --with-wxwidgets --with-python
--with-sqlite --with-readline --with-geos --with-freetype
--with-freetype-includes=/usr/include/freetype2 --with-cxx
--with-pthread --with-proj-share=/usr/share/proj/
Maris.
2011/8/1 Martin Landa <landa.martin@gmail.com>:
Hi,
2011/7/30 Maris Nartiss <maris.gis@gmail.com>:
Same here. Provided sample script runs just fine. Does it need some
special settings (Location, DB connection)?Gentoo ~AMD64
gcc version 4.5.2 (Gentoo 4.5.2 p1.0, pie-0.4.5)
Python 2.7.2compiled with LFS?
Martin
--
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa
Hi,
2011/8/1 Martin Landa <landa.martin@gmail.com>:
* my computer (Debian GNU/Linux unstable 32bit, gcc 4.6.1, python
2.6.7) - GRASS compiled *with* LFS
strangely the script works on
(Debian GNU/Linux stable 32bit, gcc 4.4.5, python 2.6.6) - GRASS
compiled *with* LFS
Can someone with Debian unstable test it?
Martin
--
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa
Martin wrote:
Can someone with Debian unstable test it?
sorry, I am away from my dev machine right now, but fyi if
someone wants to go to the trouble it is not too hard to set
up a virtual machine instance to do it. instructions for qemu-
kvm follow: (kvm requires newish cpu + linux kernel; mac and
ms windows users could use Sun's VirtualBox)
# '-m' is memory to use in MB
# '-boot d' tells it to boot from the virtual-CD drive first (the ISO)
ISO=debian-6.0.1a-amd64-netinst.iso
IMG=debian_sid_minimal_VM-amd64.img
## create and install to a disk image
#kvm-img create -f qcow2 $IMG 20G
#kvm -hda $IMG -cdrom $ISO -m 1024 -boot d
kvm -hda ~/iso/VMs/$IMG -m 1024 \
-cpu phenom -smp 2 -redir 'tcp:2222::22' \
-nographic -daemonize
# (first time probably skip the '-nographic -daemonize')
## modify /etc/apt/sources.list to point to sid
#apt-get update && apt-get upgrade
# at this point I usually copy + bzip2 the fresh image.
#ssh -p 2222 user@localhost
#scp -P 2222 file user@localhost:
hope it helps,
Hamish