[GRASSLIST:4650] vector database stuff and gshhs

Hi all, so I have changed tactics and tried to import the gshhs data into 5.7 a different way (since 24 hours of v.convert only put me at 6% finished for a ~180 MB file). I first tried to export it from 5.3 as an ascii file. I could import the file into 5.7 with v.in.ascii (and display it), however I don't understand how to get the attributes from the old dig_cats file to connect with the imported file. I know this is all part of the new database management of vectors in 5.7, but after reading everything I could on the help pages, I think I am more confused. I don't really know anything about dbf files, ODBC, SQL, or other database formats. Do I need to learn how to use a database program to use vectors in GRASS? Isn't there a simple way to convert a dig_cats file into a .dbf file and connect that to my vector file?

Thanks,

-Ian

PS mac os 10.3, grass 5.7 from ~may, 2004

On Wednesday 27 October 2004 03:37, Ian MacMillan wrote:

Hi all, so I have changed tactics and tried to import the gshhs data
into 5.7 a different way (since 24 hours of v.convert only put me at 6%
finished for a ~180 MB file). I first tried to export it from 5.3 as
an ascii file. I could import the file into 5.7 with v.in.ascii (and
display it), however I don't understand how to get the attributes from
the old dig_cats file to connect with the imported file. I know this
is all part of the new database management of vectors in 5.7, but after
reading everything I could on the help pages, I think I am more
confused. I don't really know anything about dbf files, ODBC, SQL, or
other database formats. Do I need to learn how to use a database
program to use vectors in GRASS? Isn't there a simple way to convert a
dig_cats file into a .dbf file and connect that to my vector file?

The problem is in grass 5.3 vector format. Categories are stored as
points and then attached to lines, that takes long time for
large files. Try to avoid 5.3, for example try to find shapefile.

Radim

On Thu, 28 Oct 2004, Radim Blazek wrote:

On Wednesday 27 October 2004 03:37, Ian MacMillan wrote:
> Hi all, so I have changed tactics and tried to import the gshhs data
> into 5.7 a different way (since 24 hours of v.convert only put me at 6%
> finished for a ~180 MB file). I first tried to export it from 5.3 as
> an ascii file. I could import the file into 5.7 with v.in.ascii (and
> display it), however I don't understand how to get the attributes from
> the old dig_cats file to connect with the imported file. I know this
> is all part of the new database management of vectors in 5.7, but after
> reading everything I could on the help pages, I think I am more
> confused. I don't really know anything about dbf files, ODBC, SQL, or
> other database formats. Do I need to learn how to use a database
> program to use vectors in GRASS? Isn't there a simple way to convert a
> dig_cats file into a .dbf file and connect that to my vector file?

The problem is in grass 5.3 vector format. Categories are stored as
points and then attached to lines, that takes long time for
large files. Try to avoid 5.3, for example try to find shapefile.

Has anyone tried:

http://www.ngdc.noaa.gov/mgg/shorelines/data/gshhs/gshhs_shp/

Roger

Radim

--
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Breiviksveien 40, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 93 93
e-mail: Roger.Bivand@nhh.no

Hi,

I'm trying to create a vector file through v.in.ascii, following the
template files in Example 1 from the v.in.ascii manual page:

http://grass.itc.it/grass57/manuals/html57_user/v.in.ascii.html

How does the category input from dig_cats work? There was a similar
thread last week started by Ian MacMillan but I didn't see a solution to
getting a dig_cats file working. I've appended head output for the
ascii, att and cats files that I have created, in case there is problem
there.

(Recent build of Grass 5.7 running on RHEL 3).

Many thanks,

David

[dorme@diomedea dorme]$ head -n 20 dig_ascii/EBCC_Grid
ORGANIZATION: ADHoC
DIGIT DATE: 26/10/2004
DIGIT NAME: -
MAP NAME: EBCC_Grid
MAP DATE: 2004
MAP SCALE: 50000
OTHER INFO: Test polygons
ZONE: 0
MAP THRESH: 0.500000
VERTI:
A 7
3000000 202512
3000000 250000
3000000 300000
3050000 300000
3050000 250000
3050000 203715
3000000 202512
A 5
3000000 300000
[dorme@diomedea dorme]$ head -n 20 dig_att/EBCC_Grid
A 250000 3025000 1
A 3e+05 3025000 2
A 350000 3025000 3
A 4e+05 3025000 4
A 450000 3025000 5
A 5e+05 3025000 6
A 550000 3025000 7
A 6e+05 3025000 8
A 650000 3025000 9
A 7e+05 3025000 10
A 250000 3075000 11
A 3e+05 3075000 12
A 350000 3075000 13
A 4e+05 3075000 14
A 450000 3075000 15
A 5e+05 3075000 16
A 550000 3075000 17
A 6e+05 3075000 18
A 650000 3075000 19
A 7e+05 3075000 20
[dorme@diomedea dorme]$ head -n 20 dig_cats/EBCC_Grid
# 904 categories
Vector map: EBCC_Grid

0.00 0.00 0.00 0.00
1:31SBL4
2:31SCL2
3:31SCL4
4:31SDL2
5:31SDL4
6:31SEL2
7:31SEL4
8:31SFL2
9:31SFL4
10:31SGL2
11:31SBL3
12:31SCL1
13:31SCL3
14:31SDL1
15:31SDL3
16:31SEL1
--
Dr. David Orme

Department of Biological Sciences
Imperial College London
Silwood Park
Ascot
Berkshire
SL5 7PY

Tel (020) 759 42358

dig_cats are ignored by v.in.ascii, you have to create
db table directly and link it to vector:
db.execute (create table ..., insert into ....)
v.db.connect

Radim

On Monday 01 November 2004 19:14, David Orme wrote:

Hi,

I'm trying to create a vector file through v.in.ascii, following the
template files in Example 1 from the v.in.ascii manual page:

http://grass.itc.it/grass57/manuals/html57_user/v.in.ascii.html

How does the category input from dig_cats work? There was a similar
thread last week started by Ian MacMillan but I didn't see a solution to
getting a dig_cats file working. I've appended head output for the
ascii, att and cats files that I have created, in case there is problem
there.

(Recent build of Grass 5.7 running on RHEL 3).

Many thanks,

David

[dorme@diomedea dorme]$ head -n 20 dig_ascii/EBCC_Grid
ORGANIZATION: ADHoC
DIGIT DATE: 26/10/2004
DIGIT NAME: -
MAP NAME: EBCC_Grid
MAP DATE: 2004
MAP SCALE: 50000
OTHER INFO: Test polygons
ZONE: 0
MAP THRESH: 0.500000
VERTI:
A 7
3000000 202512
3000000 250000
3000000 300000
3050000 300000
3050000 250000
3050000 203715
3000000 202512
A 5
3000000 300000
[dorme@diomedea dorme]$ head -n 20 dig_att/EBCC_Grid
A 250000 3025000 1
A 3e+05 3025000 2
A 350000 3025000 3
A 4e+05 3025000 4
A 450000 3025000 5
A 5e+05 3025000 6
A 550000 3025000 7
A 6e+05 3025000 8
A 650000 3025000 9
A 7e+05 3025000 10
A 250000 3075000 11
A 3e+05 3075000 12
A 350000 3075000 13
A 4e+05 3075000 14
A 450000 3075000 15
A 5e+05 3075000 16
A 550000 3075000 17
A 6e+05 3075000 18
A 650000 3075000 19
A 7e+05 3075000 20
[dorme@diomedea dorme]$ head -n 20 dig_cats/EBCC_Grid
# 904 categories
Vector map: EBCC_Grid

0.00 0.00 0.00 0.00
1:31SBL4
2:31SCL2
3:31SCL4
4:31SDL2
5:31SDL4
6:31SEL2
7:31SEL4
8:31SFL2
9:31SFL4
10:31SGL2
11:31SBL3
12:31SCL1
13:31SCL3
14:31SDL1
15:31SDL3
16:31SEL1

Thanks Radim - just as a follow up question on v.in.ascii - is the
dig_att file used? I followed up the v.in.ascii with 'v.category
op=print' and got a list of -1 values. Does this mean that neither
dig_vats nor dig_att is used?

In which case, just to check, the correct use of v.in.ascii in grass 5.7
should be something like:

v.in.ascii EBCC_Grid output=EBCC_Grid
v.category EBCC_Grid output=EBCC_Grid_C op=add
db.execute # to create attribute table
v.db.connect # to link the geometry in to the attributes

Thanks,
David

On Tue, 2004-11-02 at 09:25, Radim Blazek wrote:

dig_cats are ignored by v.in.ascii, you have to create
db table directly and link it to vector:
db.execute (create table ..., insert into ....)
v.db.connect

Radim

On Monday 01 November 2004 19:14, David Orme wrote:
> Hi,
>
> I'm trying to create a vector file through v.in.ascii, following the
> template files in Example 1 from the v.in.ascii manual page:
>
> http://grass.itc.it/grass57/manuals/html57_user/v.in.ascii.html
>
> How does the category input from dig_cats work? There was a similar
> thread last week started by Ian MacMillan but I didn't see a solution to
> getting a dig_cats file working. I've appended head output for the
> ascii, att and cats files that I have created, in case there is problem
> there.
>
> (Recent build of Grass 5.7 running on RHEL 3).
>
> Many thanks,
>
> David
>
>
> [dorme@diomedea dorme]$ head -n 20 dig_ascii/EBCC_Grid
> ORGANIZATION: ADHoC
> DIGIT DATE: 26/10/2004
> DIGIT NAME: -
> MAP NAME: EBCC_Grid
> MAP DATE: 2004
> MAP SCALE: 50000
> OTHER INFO: Test polygons
> ZONE: 0
> MAP THRESH: 0.500000
> VERTI:
> A 7
> 3000000 202512
> 3000000 250000
> 3000000 300000
> 3050000 300000
> 3050000 250000
> 3050000 203715
> 3000000 202512
> A 5
> 3000000 300000
> [dorme@diomedea dorme]$ head -n 20 dig_att/EBCC_Grid
> A 250000 3025000 1
> A 3e+05 3025000 2
> A 350000 3025000 3
> A 4e+05 3025000 4
> A 450000 3025000 5
> A 5e+05 3025000 6
> A 550000 3025000 7
> A 6e+05 3025000 8
> A 650000 3025000 9
> A 7e+05 3025000 10
> A 250000 3075000 11
> A 3e+05 3075000 12
> A 350000 3075000 13
> A 4e+05 3075000 14
> A 450000 3075000 15
> A 5e+05 3075000 16
> A 550000 3075000 17
> A 6e+05 3075000 18
> A 650000 3075000 19
> A 7e+05 3075000 20
> [dorme@diomedea dorme]$ head -n 20 dig_cats/EBCC_Grid
> # 904 categories
> Vector map: EBCC_Grid
>
> 0.00 0.00 0.00 0.00
> 1:31SBL4
> 2:31SCL2
> 3:31SCL4
> 4:31SDL2
> 5:31SDL4
> 6:31SEL2
> 7:31SEL4
> 8:31SFL2
> 9:31SFL4
> 10:31SGL2
> 11:31SBL3
> 12:31SCL1
> 13:31SCL3
> 14:31SDL1
> 15:31SDL3
> 16:31SEL1

I'm trying to create a vector file through v.in.ascii, following the
template files in Example 1 from the v.in.ascii manual page:

http://grass.itc.it/grass57/manuals/html57_user/v.in.ascii.html

How does the category input from dig_cats work? There was a similar
thread last week started by Ian MacMillan but I didn't see a solution
to getting a dig_cats file working. I've appended head output for the
ascii, att and cats files that I have created, in case there is
problem there.

(Recent build of Grass 5.7 running on RHEL 3).

see also
http://freegis.org/cgi-bin/viewcvs.cgi/~checkout~/grass51/doc/vector/vector.html#ascii

namely, you can assign category values on the feature line, e.g.:

VERTI:
A 7 1
3000000 202512
3000000 250000
3000000 300000
3050000 300000
3050000 250000
3050000 203715
3000000 202512
A 5 2
3000000 300000

[...]

assigns the first area to cat 1, the second to cat 2, etc.
You might also have to assign centroids for areas?

I don't know if the dig_cats and dig_att support work(?). If you are
importing fresh data, it is probably better to avoid the old 5.3 support
files altogether and import category attribues into a 5.7 table directly
to give you access to multiple attributes per category.

For points, you can just use v.in.ascii's points mode to load attribues.
For lines, areas, etc., you need to make a table. Hopefully soon there
will be db.in.ascii, but for now you can use db.execute to create and
populate, and v.db.connect to connect the cat table to the new vector file.
It is a bit complicated, but gets the job done.

for example:

NAME=mapname
ATTR_COLS='cat int, route_id int, name varchar(20), start_wpt varchar(10)'
ATTR_FILE=datafile_atts.txt

#input file must be in $MAPSET/dig_ascii/
v.in.ascii in=datafile.txt out=mapname

# check that a default databse connection exists first
db.connect -p

# Create new table
echo "CREATE TABLE $NAME ($ATTR_COLS)" | db.execute

# Populate table with attributes
##following doesn't work for DBF:
##echo "COPY $NAME FROM '$ATTR_FILE'" | db.execute
#
# so we use the SQL 'INSERT' command instead, 1 record at a time (make a loop):
#
# echo "INSERT INTO $NAME VALUES (2, 1, 'AB', 'B')" | db.execute
#===========================================
# you can do the above in a loop or use the trick below:
cat "$ATTR_FILE" | \
  awk -F'\t' '{ printf("echo \"INSERT INTO $NAME VALUES (%d, %d, ^%s^, ^%s^)\" | db.execute\n", $1, $2, $3,
$4) }' \
    | tr '^' \' > "$ATTR_FILE"_exe

. "$ATTR_FILE"_exe

(depends on $ATTR_FILE format being stable)

#===========================================
#

#Connect attribute table to vector file
v.db.connect map="$NAME" table="$NAME"

Look at the latest 5.7 v.in.garmin for a (complicated) example.

good luck,
Hamish