[GRASS-dev] Request to add a GCP file in vector directories

I’m integrating vector georectification (v.transform) into the GIS Manager georectifier. It would be nice if a GCP file could be saved in a vector folder so that it could be called up, edited, and reused and a user doesn’t have to re-enter GCP’s every time they start the georectifier. This is the case now for rasters, because a GCP file (POINTS) is saved inside the group folder for a raster group.

I’m proposing to have the georectifier (at the user’s option) save a file named “gcp” (points might be confusing in a vector folder) in the appropriate vector folder to be rectified. It would be a simple ascii text file and take the format of the standard file needed by v.transform, with the addition of a header indicating the target location and mapset it is aimed at. For example…

target: /Users/shared/grassdata/spearfish60/newvector

-584 585 598000 4920770
580 585 598020 4920770
580 -600 598020 4920750
-584 -600 598000 4920750

Does anyone foresee a problem with this?

Michael


Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

Michael Barton wrote:

I'm integrating vector georectification (v.transform) into the GIS Manager
georectifier. It would be nice if a GCP file could be saved in a vector
folder so that it could be called up, edited, and reused and a user doesn't
have to re-enter GCP's every time they start the georectifier. This is the
case now for rasters, because a GCP file (POINTS) is saved inside the group
folder for a raster group.

I'm proposing to have the georectifier (at the user's option) save a file
named ³gcp² (points might be confusing in a vector folder) in the
appropriate vector folder to be rectified. It would be a simple ascii text
file and take the format of the standard file needed by v.transform, with
the addition of a header indicating the target location and mapset it is
aimed at. For example...

# target: /Users/shared/grassdata/spearfish60/newvector
-584 585 598000 4920770
580 585 598020 4920770
580 -600 598020 4920750
-584 -600 598000 4920750

Does anyone foresee a problem with this?

Unlike i.rectify, v.transform doesn't operate upon maps from a
different location. You have to get the source map into the current
location first.

[On the last point, g.copy should probably have an option to allow
data to be copied from different locations or even from different
database directories.]

Also, transformations are arguably a property of a location rather
than a specific map. If you had a number of maps, all in the same
(unspecified) coordinate system, you would probably want to use the
same transformation for all of them.

AFAICT, the POINTS file is just an artifact of the i.points/i.rectify
workflow. The data just needed to be stored somewhere, and not much
thought was really given to the details.

It's arguable that GCP files should be a separate entity, similar to
e.g. named regions.

Also, it might be useful to allow global GCP files for an X-Y location
which specify the transformations to/from lat-lon. Adding support for
arbitrary polynomial transformations to libproj would allow you to use
*.proj to project to/from X-Y locations.

--
Glynn Clements <glynn@gclements.plus.com>

From: Glynn Clements <glynn@gclements.plus.com>
Date: Wed, 7 Jun 2006 07:33:32 +0100
To: Michael Barton <michael.barton@asu.edu>
Cc: grass5 <grass-dev@grass.itc.it>
Subject: Re: [GRASS-dev] Request to add a GCP file in vector directories

Unlike i.rectify, v.transform doesn't operate upon maps from a
different location. You have to get the source map into the current
location first.

True. I.target can also be set to the current location/mapset. For
consistency, the georectifier can operate on maps (raster and vector) from
other locations and the same location (I'm adding this part now).

[On the last point, g.copy should probably have an option to allow
data to be copied from different locations or even from different
database directories.]

This would be nice as long it wasn't misused (e.g., copying a raster from a
latlon location to a utm location)

Also, transformations are arguably a property of a location rather
than a specific map. If you had a number of maps, all in the same
(unspecified) coordinate system, you would probably want to use the
same transformation for all of them.

This is a good point. Should a generic gcp file just be stored in PERMANENT?
What about maps with different extents--especially non-overlapping maps--in
the same location?

Michael

__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

Michael Barton wrote:

I¹m integrating vector georectification (v.transform) into the GIS
Manager georectifier. It would be nice if a GCP file could be saved in
a vector folder so that it could be called up, edited, and reused and
a user doesn¹t have to re-enter GCP¹s every time they start the
georectifier. This is the case now for rasters, because a GCP file
(POINTS) is saved inside the group folder for a raster group.

I¹m proposing to have the georectifier (at the user¹s option) save a
file named ³gcp² (points might be confusing in a vector folder) in the
appropriate vector folder to be rectified. It would be a simple ascii
text file and take the format of the standard file needed by
v.transform, with the addition of a header indicating the target
location and mapset it is aimed at. For example...

# target: /Users/shared/grassdata/spearfish60/newvector
-584 585 598000 4920770
580 585 598020 4920770
580 -600 598020 4920750
-584 -600 598000 4920750

a couple of points:

* "group" isn't (or shouldn't be) limited to raster maps.

It's a common thing to have a bunch of maps in one projection and need
them in another. (e.g. multiple satellite channels)
I have a dozen XY vectors here which all exist in the same CAD space, it
would be grand if I could transform them all without diving into the
MAPSET dir & copying the file by hand..

I suggest you reuse group/ folder & structure.

* reuse the POINTS file format. It's the same thing with different
whitespace & comments. It's not so bluky that there's a real need to
simplify it. The "status" column gets set to 0 when you disable a point
in i.rectify, this is a good thing.

The less divergence the better until a full rewrite.

mind that gdal uses different row convention to GRASS. (flipped)
See my i.warp script,

http://bambi.otago.ac.nz/hamish/grass/gdal/sidescan_warp.html
  (also a demo of the broken CRS order=3 in GDAL there)

note 6.1/6.0 difference, IIRC it comes from r.in.gdal in 6.0 importing
the entire thing in negative coordinates. The difference is on the
GDAL_GCP= line?? (-r reverse sort). It's a bit of a mess .. was trying
to debug a GDAL memory problem without a clue. [in the end running
gdalwarp -tps via electric fence and removing some GCPs helped the most;
run in two passes and r.patch the results together rather that fix the
breakage for the full scene]

+______________

no need to reinvent the wheel.

Hamish

This is a good point. Should a generic gcp file just be stored in
PERMANENT? What about maps with different extents--especially
non-overlapping maps--in the same location?

I'm not a fan.

e.g. I've got a mapset:

grassdata/simple_xy/charts/

where I have scanned versions of nautical charts imported.

For each new chart I have it's own group with its own GCP info.

I don't want a new mapset, let alone location, for each new scan.

The group thing is good logically & works well in practice.
Again I'd suggest we keep using the same method, just make a nice
front end for it which merges and hides the gritty details.
(I can picture a tabbed GUI with target in one, registed maps in
another, etc..)

Hamish

Hamish wrote:

> This is a good point. Should a generic gcp file just be stored in
> PERMANENT? What about maps with different extents--especially
> non-overlapping maps--in the same location?

I'm not a fan.

e.g. I've got a mapset:

grassdata/simple_xy/charts/

where I have scanned versions of nautical charts imported.

For each new chart I have it's own group with its own GCP info.

I don't want a new mapset, let alone location, for each new scan.

Right. It depends largely on whether the location is being used as a
general purpose container or like a normal (georeferenced) location
except that it has a non-standard projection.

Even in the latter case, you might need multiple transformations, as
one which is a good fit for one map might not be a good fit for
another. Producing a transformation which is a good fit for the entire
location would be difficult if you can only set reference points over
a single map.

Support for locations with user-defined (polynomial) projections would
need to be in addition to the existing transformation tools, not a
replacement. But it may be worth considering at this point.

The group thing is good logically & works well in practice.
Again I'd suggest we keep using the same method, just make a nice
front end for it which merges and hides the gritty details.
(I can picture a tabbed GUI with target in one, registed maps in
another, etc..)

Command-line front-ends for some of the tools would also be nice.

[By "command-line", I mean input via command-line options and/or
files/stdin, rather than via curses forms.]

--
Glynn Clements <glynn@gclements.plus.com>

Hamish wrote:

* reuse the POINTS file format. It's the same thing with different
whitespace & comments. It's not so bluky that there's a real need to
simplify it. The "status" column gets set to 0 when you disable a point
in i.rectify, this is a good thing.

v.transform already has a defined format for GCPs, and that format is
exposed to the user (i.e. the user is supposed to create the data in
that format). A common format would need to remain compatible (e.g.
change v.transform to allow either 4 or 5 columns).

--
Glynn Clements <glynn@gclements.plus.com>

Hamish,

Haven't read through all my mail from overnight yet, so there might be
comments by others. But I'll start with my own.

See below.

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics and Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: Hamish <hamish_nospam@yahoo.com>
Date: Wed, 7 Jun 2006 22:34:00 +1200
To: Michael Barton <michael.barton@asu.edu>
Cc: <grass-dev@grass.itc.it>
Subject: Re: [GRASS-dev] Request to add a GCP file in vector directories

Michael Barton wrote:

I¹m integrating vector georectification (v.transform) into the GIS
Manager georectifier. It would be nice if a GCP file could be saved in
a vector folder so that it could be called up, edited, and reused and
a user doesn¹t have to re-enter GCP¹s every time they start the
georectifier. This is the case now for rasters, because a GCP file
(POINTS) is saved inside the group folder for a raster group.

I¹m proposing to have the georectifier (at the user¹s option) save a
file named ³gcp² (points might be confusing in a vector folder) in the
appropriate vector folder to be rectified. It would be a simple ascii
text file and take the format of the standard file needed by
v.transform, with the addition of a header indicating the target
location and mapset it is aimed at. For example...

# target: /Users/shared/grassdata/spearfish60/newvector
-584 585 598000 4920770
580 585 598020 4920770
580 -600 598020 4920750
-584 -600 598000 4920750

a couple of points:

* "group" isn't (or shouldn't be) limited to raster maps.

It's a common thing to have a bunch of maps in one projection and need
them in another. (e.g. multiple satellite channels)
I have a dozen XY vectors here which all exist in the same CAD space, it
would be grand if I could transform them all without diving into the
MAPSET dir & copying the file by hand..

I suggest you reuse group/ folder & structure.

This seems fine and is the reason I wanted to save a vector gcp file.
However, v.transform won't read a 'group'. I could emulate it in TclTk, but
if this is what we want to do, it would be better to have v.transform simply
work like i.rectify.

* reuse the POINTS file format. It's the same thing with different
whitespace & comments. It's not so bluky that there's a real need to
simplify it. The "status" column gets set to 0 when you disable a point
in i.rectify, this is a good thing.

The less divergence the better until a full rewrite.

Again, v.transform might have problems with the 1 or 0 at the end of each
GCP line. Otherwise, the formats are the same. Again, altering v.transform
to read a standard POINTS file makes for nice consistency.

mind that gdal uses different row convention to GRASS. (flipped)
See my i.warp script,

http://bambi.otago.ac.nz/hamish/grass/gdal/sidescan_warp.html
  (also a demo of the broken CRS order=3 in GDAL there)

note 6.1/6.0 difference, IIRC it comes from r.in.gdal in 6.0 importing
the entire thing in negative coordinates. The difference is on the
GDAL_GCP= line?? (-r reverse sort). It's a bit of a mess .. was trying
to debug a GDAL memory problem without a clue. [in the end running
gdalwarp -tps via electric fence and removing some GCPs helped the most;
run in two passes and r.patch the results together rather that fix the
breakage for the full scene]

This and other issues make me shy away from trying to add in the abilty to
use gdalwarp right now. It needs very different parameters, requires more
file manipulation, and doesn't do vectors (AFAIK). I kind of like your idea
better to use gdal libraries.

+______________

no need to reinvent the wheel.

Rather, lets have the same wheels all around the vehicle.

Michael

Hamish

Currently, I've got the gcp file stored in $GISDBASE/vector/[vector name].
It is no problem to put into $GISBASE/group/[vector group].

But to do this, it would be better if...

--i.group (or a new v.group) could create the group structure and the group
file

--v.transform could read a group file and act on the vectors listed

--v.transform could read a standard POINTS file and use or ignore the "use
gcp" column.

I could emulate steps 1 & 2 in TclTk (and probably hack step 3), but it
would be better in the long run if the GRASS C modules acted in a consistent
way for vectors and rasters.

I'll keep the 'test version' of the rectifier as is for the moment, but can
change it to use a group structure if that's what we want and can make the
changes to i.group/v.group and v.transform.

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics and Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: Glynn Clements <glynn@gclements.plus.com>
Date: Wed, 7 Jun 2006 12:15:14 +0100
To: Hamish <hamish_nospam@yahoo.com>
Cc: Michael Barton <michael.barton@asu.edu>, <grass-dev@grass.itc.it>
Subject: Re: [GRASS-dev] Request to add a GCP file in vector directories

Hamish wrote:

This is a good point. Should a generic gcp file just be stored in
PERMANENT? What about maps with different extents--especially
non-overlapping maps--in the same location?

I'm not a fan.

e.g. I've got a mapset:

grassdata/simple_xy/charts/

where I have scanned versions of nautical charts imported.

For each new chart I have it's own group with its own GCP info.

I don't want a new mapset, let alone location, for each new scan.

Right. It depends largely on whether the location is being used as a
general purpose container or like a normal (georeferenced) location
except that it has a non-standard projection.

Even in the latter case, you might need multiple transformations, as
one which is a good fit for one map might not be a good fit for
another. Producing a transformation which is a good fit for the entire
location would be difficult if you can only set reference points over
a single map.

Support for locations with user-defined (polynomial) projections would
need to be in addition to the existing transformation tools, not a
replacement. But it may be worth considering at this point.

The group thing is good logically & works well in practice.
Again I'd suggest we keep using the same method, just make a nice
front end for it which merges and hides the gritty details.
(I can picture a tabbed GUI with target in one, registed maps in
another, etc..)

Command-line front-ends for some of the tools would also be nice.

[By "command-line", I mean input via command-line options and/or
files/stdin, rather than via curses forms.]

--
Glynn Clements <glynn@gclements.plus.com>