[GRASS-dev] v.in.gpsbabel nearly fixed: awk magic needed (RT bug #5478)

HI,

I think I got v.in.gpsbabel for tracks and routes (did that
ever work?). However, I have a final problem which requires
a bit of awk magic:

The GPX tracks file extracted from my Garmin contains a track
which isn't attributed (I dunno why but that's life). The
v.in.gpsbabel scripts successfully creates the map, then fails
on creating the attrib table. Here the problem:

# note: files contains tab delimiters
head -2 12021.0.track_atts
1
2 45.417394638 10.847196579 45.359029770 10.166580677

# make tab delim visible (just for illustration):
head -2 12021.0.track_atts | tr -s '\t' '|'
1 | | | | | |
2 | |45.417394638 |10.847196579 | |45.359029770 |10.166580677

# this is what happens in v.in.gpsbabel (note: it does cat instead of head):
head -2 12021.0.track_atts | awk -F'\t' '{ printf("echo \"INSERT INTO $NAME VALUES (%d, ^%s^, %s, %s, ^%s^, %s, %s)\" | db.execute\n", $1, $2, $3, $4, $5, $6, $7) }' | tr '^' \'
echo "INSERT INTO $NAME VALUES (1, ' ', , , ' ', , )" | db.execute
echo "INSERT INTO $NAME VALUES (2, ' ', 45.417394638 , 10.847196579 , ' ', 45.359029770 , 10.166580677)" | db.execute

We see that the first line isn't valid SQL. The first line should be:
echo "INSERT INTO $NAME VALUES (1, ' ', NULL , NULL , ' ', NULL , NULL)" | db.execute

Question: what to ask in the awk line to autoinsert NULL if 0 length is
found in the numeric columns 3, 4, 6, 7?

thanks
Markus

How about this?

head -2 Markus.txt | awk -F'\t' '{
if($3 == ""){$3 = "NULL"}
if($4 == ""){$4 = "NULL"}
if($6 == ""){$6 = "NULL"}
if($7 == ""}{$7 = "NULL"}

printf("echo \"INSERT INTO $NAME VALUES (%d, ^%s^, %s, %s, ^%s^, %s, %s)\" | db.execute\n", $1, $2, $3, $4, $5, $6, $7) }' | tr '^' \'

:wink:

~ Eric.

-----Original Message-----
From: grass-dev-bounces@grass.itc.it on behalf of Markus Neteler
Sent: Thu 5/10/2007 6:00 PM
To: GRASS developers list
Cc: grass-bugs@intevation.de
Subject: [GRASS-dev] v.in.gpsbabel nearly fixed: awk magic needed (RT bug#5478)

HI,

I think I got v.in.gpsbabel for tracks and routes (did that
ever work?). However, I have a final problem which requires
a bit of awk magic:

The GPX tracks file extracted from my Garmin contains a track
which isn't attributed (I dunno why but that's life). The
v.in.gpsbabel scripts successfully creates the map, then fails
on creating the attrib table. Here the problem:

# note: files contains tab delimiters
head -2 12021.0.track_atts
1
2 45.417394638 10.847196579 45.359029770 10.166580677

# make tab delim visible (just for illustration):
head -2 12021.0.track_atts | tr -s '\t' '|'
1 | | | | | |
2 | |45.417394638 |10.847196579 | |45.359029770 |10.166580677

# this is what happens in v.in.gpsbabel (note: it does cat instead of head):
head -2 12021.0.track_atts | awk -F'\t' '{ printf("echo \"INSERT INTO $NAME VALUES (%d, ^%s^, %s, %s, ^%s^, %s, %s)\" | db.execute\n", $1, $2, $3, $4, $5, $6, $7) }' | tr '^' \'
echo "INSERT INTO $NAME VALUES (1, ' ', , , ' ', , )" | db.execute
echo "INSERT INTO $NAME VALUES (2, ' ', 45.417394638 , 10.847196579 , ' ', 45.359029770 , 10.166580677)" | db.execute

We see that the first line isn't valid SQL. The first line should be:
echo "INSERT INTO $NAME VALUES (1, ' ', NULL , NULL , ' ', NULL , NULL)" | db.execute

Question: what to ask in the awk line to autoinsert NULL if 0 length is
found in the numeric columns 3, 4, 6, 7?

thanks
Markus

_______________________________________________
grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

Markus Neteler wrote:

I think I got v.in.gpsbabel for tracks and routes (did that
ever work?). However, I have a final problem which requires
a bit of awk magic:

The GPX tracks file extracted from my Garmin contains a track
which isn't attributed (I dunno why but that's life). The
v.in.gpsbabel scripts successfully creates the map, then fails
on creating the attrib table. Here the problem:

# note: files contains tab delimiters
head -2 12021.0.track_atts
1
2 45.417394638 10.847196579 45.359029770 10.166580677

# make tab delim visible (just for illustration):
head -2 12021.0.track_atts | tr -s '\t' '|'
1 | | | | | |
2 | |45.417394638 |10.847196579 | |45.359029770 |10.166580677

I see you too have figured out that ,style="$STYLE" wasn't needed for
gpx format. I was just about to commit that! :wink:

I think the extra ,style= only triggerd a harmless warning, which is why
we didn't pick it up before. If gpsbabel has an exit code of 1 the
script should quit right there, but in the bug report it continues on.

# make tab delim visible (just for illustration):
head -2 12021.0.track_atts | tr -s '\t' '|'
1 | | | | | |
2 | |45.417394638 |10.847196579 | |45.359029770 |10.166580677

this MAY may may be related to some change I May have made to work
around an off-by-one bug I was struggling with. It turns out that there
is a bug in the Garmin 45,48,12XL firmware* where if the track fills
memory and loops the timestamp record can be shifted by one spot vs.
x,y,z record. Before I figured that out I was having all sorts of
trouble getting v.in.gpsbabel and v.in.garmin to produce the same
output- always atts off by one! I had asked Davide and Claudio to fix
this bug in their script before submission, but they never could! oops.

[*] reported to Garmin, but I never heard back from them

I will request one of our field GPSs back here for more testing.

summary: compare track and timestamps vs GPX download file, I suspect
the track_atts are shifted down one line, and worrying about sed -e
's/| |/|NULL|/' only masks the real bug.

Hamish

Hi,

in similar form I have submitted this now. v.in.gpsbabel no longer crashes!
To figure out: are the attributes shifted by one row or not?

Markus

Patton, Eric wrote:

How about this?

head -2 Markus.txt | awk -F'\t' '{
if($3 == ""){$3 = "NULL"}
if($4 == ""){$4 = "NULL"}
if($6 == ""){$6 = "NULL"}
if($7 == ""}{$7 = "NULL"}

printf("echo \"INSERT INTO $NAME VALUES (%d, ^%s^, %s, %s, ^%s^, %s, %s)\"
| db.execute\n", $1, $2, $3, $4, $5, $6, $7) }' | tr '^' \'

:wink:

~ Eric.

-----Original Message-----
From: grass-dev-bounces@grass.itc.it on behalf of Markus Neteler
Sent: Thu 5/10/2007 6:00 PM
To: GRASS developers list
Cc: grass-bugs@intevation.de
Subject: [GRASS-dev] v.in.gpsbabel nearly fixed: awk magic needed (RT
bug#5478)

HI,

I think I got v.in.gpsbabel for tracks and routes (did that
ever work?). However, I have a final problem which requires
a bit of awk magic:

The GPX tracks file extracted from my Garmin contains a track
which isn't attributed (I dunno why but that's life). The
v.in.gpsbabel scripts successfully creates the map, then fails
on creating the attrib table. Here the problem:

# note: files contains tab delimiters
head -2 12021.0.track_atts
1
2 45.417394638 10.847196579 45.359029770
10.166580677

# make tab delim visible (just for illustration):
head -2 12021.0.track_atts | tr -s '\t' '|'
1 | | | | | |
2 | |45.417394638 |10.847196579 | |45.359029770 |10.166580677

# this is what happens in v.in.gpsbabel (note: it does cat instead of
head):
head -2 12021.0.track_atts | awk -F'\t' '{ printf("echo \"INSERT INTO
$NAME VALUES (%d, ^%s^, %s, %s, ^%s^, %s, %s)\" | db.execute\n", $1, $2,
$3, $4, $5, $6, $7) }' | tr '^' \'
echo "INSERT INTO $NAME VALUES (1, ' ', , , ' ', , )" | db.execute
echo "INSERT INTO $NAME VALUES (2, ' ', 45.417394638 , 10.847196579 , ' ',
45.359029770 , 10.166580677)" | db.execute

We see that the first line isn't valid SQL. The first line should be:
echo "INSERT INTO $NAME VALUES (1, ' ', NULL , NULL , ' ', NULL , NULL)"
| db.execute

Question: what to ask in the awk line to autoinsert NULL if 0 length is
found in the numeric columns 3, 4, 6, 7?

thanks
Markus

_______________________________________________
grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

_______________________________________________
grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

--
View this message in context: http://www.nabble.com/v.in.gpsbabel-nearly-fixed%3A-awk-magic-needed-(RT-bug--5478)-tf3724138.html#a10435856
Sent from the Grass - Dev mailing list archive at Nabble.com.

Markus Neteler wrote:

Hi,

in similar form I have submitted this now. v.in.gpsbabel no longer
crashes!
To figure out: are the attributes shifted by one row or not?

I checked the timestamps of my tracks and it looks to be NOT shifted!
Very nice. I'll backport the fix to 6.2-CVS.

Markus
--
View this message in context: http://www.nabble.com/v.in.gpsbabel-nearly-fixed%3A-awk-magic-needed-(RT-bug--5478)-tf3724138.html#a10437213
Sent from the Grass - Dev mailing list archive at Nabble.com.

Markus Neteler wrote:

> in similar form I have submitted this now. v.in.gpsbabel no longer
> crashes!
> To figure out: are the attributes shifted by one row or not?
>

I checked the timestamps of my tracks and it looks to be NOT shifted!
Very nice. I'll backport the fix to 6.2-CVS.

any idea where the extra line comes from? I would like to understand /
solve that before forgetting about the bug.

(no test gps here for a few more days)

Hamish