[GRASS-user] v.label.sa out of memory on small vector map.

GRASS GRASS user
1

I have just started playing with v.label.sa in GRASS 6.5-svn (last updated
yesterday).

I had success in getting it to process one fairly large map of US Forest Service
trail data, but had it run out of memory on a similar map from the same source
of road data. Even trying to run it on a very small subset (40 lines) of the
road data in my area of interest gets an out-of-memory error as the size of
v.label.sa grows to more than 1GB.

Here's what I've done:

Download "Current Trails Inventory" and "Current Roads Inventory" from
http://www.fs.fed.us/r3/cibola/plan-revision/gis_data/data.shtml

These are Zipped shapefile sets. I imported into a NAD83, UTM zone 13
region (EPSG:26913) using v.in.ogr without problem. I imported Trail_Route.shp
as "trails2008" and Road_Route.shp as usfs_roads2008.

This works:

v.label.sa map=trails2008 column=TRL_NO labels=trails2008 font=VeraSe

It doesn't produce the lovliest of labels, but runs in small memory and in
reasonable time.

But this fails with a memory allocation error, after chewing a long time:
v.label.sa map=usfs_roads2008 column=RTE_NO labels=roads2008 font=Vera

usfs_roads2008 contains 6203 lines, so I tried to restrict my consideration
to the area I actually want to make a paper map of.

g.region -p

projection: 1 (UTM)
zone: 13
datum: nad83
ellipsoid: grs80
north: 3883362.61697134
south: 3877081.62443534
west: 373147.02714574
east: 380581.87668613
nsres: 2.4382735
ewres: 2.43845508
rows: 2576
cols: 3049
cells: 7854224

v.in.region output=tmpreg

v.select ainput=usfs_roads2008 binput=tmpreg output=trnngroads operator=overlap

v.info trnngroads

+----------------------------------------------------------------------------+
| Layer: trnngroads |
| Mapset: PERMANENT |
| Location: NewMexico |
| Database: /users/russo/GIS |
| Title: |
| Map scale: 1:1 |
| Map format: native |
| Name of creator: russo |
| Organization: |
| Source date: Tue May 26 10:30:59 2009 |
|----------------------------------------------------------------------------|
| Type of Map: vector (level: 2) |
| |
| Number of points: 0 Number of areas: 0 |
| Number of lines: 40 Number of islands: 0 |
| Number of boundaries: 0 Number of faces: 0 |
| Number of centroids: 0 Number of kernels: 0 |
| |
| Map is 3D: No |
| Number of dblinks: 1 |
| |
| Projection: x,y (zone 0) |
| N: 3940710.738 S: 3846467.585 |
| E: 678499.548 W: 132270.347 |
| |
| Digitization threshold: 0 |
| Comments: |
| |
+----------------------------------------------------------------------------+

But v.label.sa still gets a memory allocation error on this:
Initialising labels... 100%
Generating label candidates: ... 100%
51%

At this point, "top" shows v.label.sa starting to grow tremendously, rapidly
getting to 200M, gradually growing to 1000M.

This only happens with this particular data set. What could make v.label.sa
grow so much on a map with only 40 lines?

--
Tom Russo KM5VY SAR502 DM64ux http://www.swcp.com/~russo/
Tijeras, NM QRPL#1592 K2#398 SOC#236 http://kevan.org/brain.cgi?DDTNM
  In some cultures what I do would be considered normal.
                                  -- Ineffective daily affirmation

2

On Tue, May 26, 2009 at 10:44 PM, Tom Russo <russo@bogodyn.org> wrote:

I have just started playing with v.label.sa in GRASS 6.5-svn (last updated
yesterday).

I had success in getting it to process one fairly large map of US Forest Service
trail data, but had it run out of memory on a similar map from the same source
of road data. Even trying to run it on a very small subset (40 lines) of the
road data in my area of interest gets an out-of-memory error as the size of
v.label.sa grows to more than 1GB.

Can you please use "valgrind" to check for a memory leak?

http://grass.osgeo.org/wiki/GRASS_Debugging#Using_Valgrind

CMD="v.label.sa map=usfs_roads2008 column=RTE_NO labels=roads2008 font=Vera"

valgrind --tool=memcheck --leak-check=yes --show-reachable=yes $CMD --o

The result will be long - you may send it offlist to me and I'll
extract the relevant part for the list.

Markus

3

Hi,

Thanks for the report. I'll have to test it. So far I have not run into
anything like that. Very interesting indeed. Are the roads very long? Or
is there perhaps one very long road? When it generates label candidates
it will first create many and select the 32 "best" candidates. This
could make it grow quite a lot especially if the label is short compared
to the road. Another possibility would be a bug of some sort.

--Wolf

On 05/26/2009 11:44 PM, Tom Russo wrote:

I have just started playing with v.label.sa in GRASS 6.5-svn (last updated
yesterday).

I had success in getting it to process one fairly large map of US Forest Service
trail data, but had it run out of memory on a similar map from the same source
of road data. Even trying to run it on a very small subset (40 lines) of the
road data in my area of interest gets an out-of-memory error as the size of
v.label.sa grows to more than 1GB.

Here's what I've done:

Download "Current Trails Inventory" and "Current Roads Inventory" from
http://www.fs.fed.us/r3/cibola/plan-revision/gis_data/data.shtml

These are Zipped shapefile sets. I imported into a NAD83, UTM zone 13
region (EPSG:26913) using v.in.ogr without problem. I imported Trail_Route.shp
as "trails2008" and Road_Route.shp as usfs_roads2008.

This works:

v.label.sa map=trails2008 column=TRL_NO labels=trails2008 font=VeraSe

It doesn't produce the lovliest of labels, but runs in small memory and in
reasonable time.

But this fails with a memory allocation error, after chewing a long time:
v.label.sa map=usfs_roads2008 column=RTE_NO labels=roads2008 font=Vera

usfs_roads2008 contains 6203 lines, so I tried to restrict my consideration
to the area I actually want to make a paper map of.

g.region -p

projection: 1 (UTM)
zone: 13
datum: nad83
ellipsoid: grs80
north: 3883362.61697134
south: 3877081.62443534
west: 373147.02714574
east: 380581.87668613
nsres: 2.4382735
ewres: 2.43845508
rows: 2576
cols: 3049
cells: 7854224

v.in.region output=tmpreg

v.select ainput=usfs_roads2008 binput=tmpreg output=trnngroads operator=overlap

v.info trnngroads

+----------------------------------------------------------------------------+
| Layer: trnngroads |
| Mapset: PERMANENT |
| Location: NewMexico |
| Database: /users/russo/GIS |
| Title: |
| Map scale: 1:1 |
| Map format: native |
| Name of creator: russo |
| Organization: |
| Source date: Tue May 26 10:30:59 2009 |
|----------------------------------------------------------------------------|
| Type of Map: vector (level: 2) |
| |
| Number of points: 0 Number of areas: 0 |
| Number of lines: 40 Number of islands: 0 |
| Number of boundaries: 0 Number of faces: 0 |
| Number of centroids: 0 Number of kernels: 0 |
| |
| Map is 3D: No |
| Number of dblinks: 1 |
| |
| Projection: x,y (zone 0) |
| N: 3940710.738 S: 3846467.585 |
| E: 678499.548 W: 132270.347 |
| |
| Digitization threshold: 0 |
| Comments: |
| |
+----------------------------------------------------------------------------+

But v.label.sa still gets a memory allocation error on this:
Initialising labels... 100%
Generating label candidates: ... 100%
51%

At this point, "top" shows v.label.sa starting to grow tremendously, rapidly
getting to 200M, gradually growing to 1000M.

This only happens with this particular data set. What could make v.label.sa
grow so much on a map with only 40 lines?

--

Wolf Bergenheim (wolf+grass@bergenheim.net)

4

Ok I have now tested a bit, and I can repeat Tom's problem.

The cause is the double roads I-40E and I-40W. It is extremely long and
the label is very short. Try with a larger label size or try to split it
into two or more parts. But I'm calling this defect. The label
candidate positions should maybe not be based on the size of the label
so strongly. I'll try to think of something...

As for the quality of label placement itself. I know. There is some bug
there or else the algorithm is not as good as it is supposed to be,
probably just the implementation is buggy or bad. I'm continuing to work
on improving it.

--Wolf

On 05/27/2009 09:16 AM, Wolf Bergenheim wrote:

Hi,

Thanks for the report. I'll have to test it. So far I have not run into
anything like that. Very interesting indeed. Are the roads very long? Or
is there perhaps one very long road? When it generates label candidates
it will first create many and select the 32 "best" candidates. This
could make it grow quite a lot especially if the label is short compared
to the road. Another possibility would be a bug of some sort.

--Wolf

On 05/26/2009 11:44 PM, Tom Russo wrote:

I have just started playing with v.label.sa in GRASS 6.5-svn (last updated
yesterday).

I had success in getting it to process one fairly large map of US Forest Service
trail data, but had it run out of memory on a similar map from the same source
of road data. Even trying to run it on a very small subset (40 lines) of the
road data in my area of interest gets an out-of-memory error as the size of
v.label.sa grows to more than 1GB.

Here's what I've done:

Download "Current Trails Inventory" and "Current Roads Inventory" from
http://www.fs.fed.us/r3/cibola/plan-revision/gis_data/data.shtml

These are Zipped shapefile sets. I imported into a NAD83, UTM zone 13
region (EPSG:26913) using v.in.ogr without problem. I imported Trail_Route.shp
as "trails2008" and Road_Route.shp as usfs_roads2008.

This works:

v.label.sa map=trails2008 column=TRL_NO labels=trails2008 font=VeraSe

It doesn't produce the lovliest of labels, but runs in small memory and in
reasonable time.

But this fails with a memory allocation error, after chewing a long time:
v.label.sa map=usfs_roads2008 column=RTE_NO labels=roads2008 font=Vera

usfs_roads2008 contains 6203 lines, so I tried to restrict my consideration
to the area I actually want to make a paper map of.

g.region -p

projection: 1 (UTM)
zone: 13
datum: nad83
ellipsoid: grs80
north: 3883362.61697134
south: 3877081.62443534
west: 373147.02714574
east: 380581.87668613
nsres: 2.4382735
ewres: 2.43845508
rows: 2576
cols: 3049
cells: 7854224

v.in.region output=tmpreg
v.select ainput=usfs_roads2008 binput=tmpreg output=trnngroads operator=overlap
v.info trnngroads

+----------------------------------------------------------------------------+
| Layer: trnngroads |
| Mapset: PERMANENT |
| Location: NewMexico |
| Database: /users/russo/GIS |
| Title: |
| Map scale: 1:1 |
| Map format: native |
| Name of creator: russo |
| Organization: |
| Source date: Tue May 26 10:30:59 2009 |
|----------------------------------------------------------------------------|
| Type of Map: vector (level: 2) |
| |
| Number of points: 0 Number of areas: 0 |
| Number of lines: 40 Number of islands: 0 |
| Number of boundaries: 0 Number of faces: 0 |
| Number of centroids: 0 Number of kernels: 0 |
| |
| Map is 3D: No |
| Number of dblinks: 1 |
| |
| Projection: x,y (zone 0) |
| N: 3940710.738 S: 3846467.585 |
| E: 678499.548 W: 132270.347 |
| |
| Digitization threshold: 0 |
| Comments: |
| |
+----------------------------------------------------------------------------+

But v.label.sa still gets a memory allocation error on this:
Initialising labels... 100%
Generating label candidates: ... 100%
51%

At this point, "top" shows v.label.sa starting to grow tremendously, rapidly
getting to 200M, gradually growing to 1000M.

This only happens with this particular data set. What could make v.label.sa
grow so much on a map with only 40 lines?

--

Wolf Bergenheim (wolf+grass@bergenheim.net)

5

On Wed, May 27, 2009 at 10:33:04AM +0300, we recorded a bogon-computron collision of the <wolf+grass@bergenheim.net> flavor, containing:

Ok I have now tested a bit, and I can repeat Tom's problem.

The cause is the double roads I-40E and I-40W. It is extremely long and
the label is very short. Try with a larger label size or try to split it
into two or more parts. But I'm calling this defect. The label
candidate positions should maybe not be based on the size of the label
so strongly. I'll try to think of something...

AH! I had thought that I'd taken care of that with the v.select, but I realize
now I should have used v.overlay instead to chop out the tiny segment of
I-40 that intersects my region of interest.

As for the quality of label placement itself. I know. There is some bug
there or else the algorithm is not as good as it is supposed to be,
probably just the implementation is buggy or bad. I'm continuing to work
on improving it.

I've seen one or two labels for trails2008 that are not at all close to their
associated feature. These are mostly cases where the line is almost as long
as its label.

--Wolf

On 05/27/2009 09:16 AM, Wolf Bergenheim wrote:
> Hi,
>
> Thanks for the report. I'll have to test it. So far I have not run into
> anything like that. Very interesting indeed. Are the roads very long? Or
> is there perhaps one very long road? When it generates label candidates
> it will first create many and select the 32 "best" candidates. This
> could make it grow quite a lot especially if the label is short compared
> to the road. Another possibility would be a bug of some sort.
>
> --Wolf
>
> On 05/26/2009 11:44 PM, Tom Russo wrote:
>> I have just started playing with v.label.sa in GRASS 6.5-svn (last updated
>> yesterday).
>>
>> I had success in getting it to process one fairly large map of US Forest Service
>> trail data, but had it run out of memory on a similar map from the same source
>> of road data. Even trying to run it on a very small subset (40 lines) of the
>> road data in my area of interest gets an out-of-memory error as the size of
>> v.label.sa grows to more than 1GB.
>>
>> Here's what I've done:
>>
>> Download "Current Trails Inventory" and "Current Roads Inventory" from
>> http://www.fs.fed.us/r3/cibola/plan-revision/gis_data/data.shtml
>>
>> These are Zipped shapefile sets. I imported into a NAD83, UTM zone 13
>> region (EPSG:26913) using v.in.ogr without problem. I imported Trail_Route.shp
>> as "trails2008" and Road_Route.shp as usfs_roads2008.
>>
>> This works:
>>
>> v.label.sa map=trails2008 column=TRL_NO labels=trails2008 font=VeraSe
>>
>> It doesn't produce the lovliest of labels, but runs in small memory and in
>> reasonable time.
>>
>> But this fails with a memory allocation error, after chewing a long time:
>> v.label.sa map=usfs_roads2008 column=RTE_NO labels=roads2008 font=Vera
>>
>> usfs_roads2008 contains 6203 lines, so I tried to restrict my consideration
>> to the area I actually want to make a paper map of.
>>
>>> g.region -p
>> projection: 1 (UTM)
>> zone: 13
>> datum: nad83
>> ellipsoid: grs80
>> north: 3883362.61697134
>> south: 3877081.62443534
>> west: 373147.02714574
>> east: 380581.87668613
>> nsres: 2.4382735
>> ewres: 2.43845508
>> rows: 2576
>> cols: 3049
>> cells: 7854224
>>
>>
>>> v.in.region output=tmpreg
>>> v.select ainput=usfs_roads2008 binput=tmpreg output=trnngroads operator=overlap
>>> v.info trnngroads
>> +----------------------------------------------------------------------------+
>> | Layer: trnngroads |
>> | Mapset: PERMANENT |
>> | Location: NewMexico |
>> | Database: /users/russo/GIS |
>> | Title: |
>> | Map scale: 1:1 |
>> | Map format: native |
>> | Name of creator: russo |
>> | Organization: |
>> | Source date: Tue May 26 10:30:59 2009 |
>> |----------------------------------------------------------------------------|
>> | Type of Map: vector (level: 2) |
>> | |
>> | Number of points: 0 Number of areas: 0 |
>> | Number of lines: 40 Number of islands: 0 |
>> | Number of boundaries: 0 Number of faces: 0 |
>> | Number of centroids: 0 Number of kernels: 0 |
>> | |
>> | Map is 3D: No |
>> | Number of dblinks: 1 |
>> | |
>> | Projection: x,y (zone 0) |
>> | N: 3940710.738 S: 3846467.585 |
>> | E: 678499.548 W: 132270.347 |
>> | |
>> | Digitization threshold: 0 |
>> | Comments: |
>> | |
>> +----------------------------------------------------------------------------+
>>
>> But v.label.sa still gets a memory allocation error on this:
>> Initialising labels... 100%
>> Generating label candidates: ... 100%
>> 51%
>>
>> At this point, "top" shows v.label.sa starting to grow tremendously, rapidly
>> getting to 200M, gradually growing to 1000M.
>>
>> This only happens with this particular data set. What could make v.label.sa
>> grow so much on a map with only 40 lines?
>>
>

--

Wolf Bergenheim (wolf+grass@bergenheim.net)

--
Tom Russo KM5VY SAR502 DM64ux http://www.swcp.com/~russo/
Tijeras, NM QRPL#1592 K2#398 SOC#236 http://kevan.org/brain.cgi?DDTNM
  In some cultures what I do would be considered normal.
                                  -- Ineffective daily affirmation