[GRASS-dev] v.in.gshhs - bogus horizontal lines

Maciej_Sieczka · March 16, 2009, 8:18pm

Markus,

Thanks for your v.in.gshhs GRASS 6 port.

I have a problem: when importing full GSHHS extent, strange horizontal
lines through the whole longitudal extent are present in the output
GRASS vector map, which are *visible only in v.digit or QGIS*, but never
on the regular wxGUI map display or X monitors.

To reproduce please import e.g. gshhs_c.b, zoom to it in wxGUI, then
open in digitizer and compare. Also please notice that *in QGIS those
bogus lines are always visible*.

I suppose this is related to the GSSHS data extent 0 - 360 vs. GRASS
-180 - 180.

Best,
Maciek

--
Maciej Sieczka
www.sieczka.org

Markus_GRASS · March 16, 2009, 9:44pm

Maciej Sieczka wrote:

Markus,

Thanks for your v.in.gshhs GRASS 6 port.

I have a problem: when importing full GSHHS extent, strange horizontal
lines through the whole longitudal extent are present in the output
GRASS vector map, which are *visible only in v.digit or QGIS*, but never
on the regular wxGUI map display or X monitors.

To reproduce please import e.g. gshhs_c.b, zoom to it in wxGUI, then
open in digitizer and compare. Also please notice that *in QGIS those
bogus lines are always visible*.

I suppose this is related to the GSSHS data extent 0 - 360 vs. GRASS
-180 - 180.

Strange. v.in.gshhs converts all lon coordinates to the range of -180 - 180. The only possible problem is Eurasiafrica (also converted to -180 - 180), crossing the datum border, all other shorelines are not crossing the datum border. I wrote the new v.in.gshhs port so that it displays properly, but GRASS latlon handling is not fully consistent, e.g. bounding boxes in vector topology ignore the datum border problems whereas (most?) lib/gis functions take care of that. My guess is that neither the wxGUI vdigit nor QGIS do this on-the-fly wraparound like the GRASS display. No idea how to solve this. Glynn once remarked that he tried to support -360 - 360 in latlon, and said that this opens a problem box of Pandora. I tried myself, and found the same problem box

Best regards,

Markus

Maciej_Sieczka · March 17, 2009, 7:38pm

Markus Metz pisze:

Maciej Sieczka wrote:

Thanks for your v.in.gshhs GRASS 6 port.

I have a problem: when importing full GSHHS extent, strange
horizontal lines through the whole longitudal extent are present in
the output GRASS vector map, which are *visible only in v.digit or
QGIS*, but never on the regular wxGUI map display or X monitors.

To reproduce please import e.g. gshhs_c.b, zoom to it in wxGUI,
then open in digitizer and compare. Also please notice that *in
QGIS those bogus lines are always visible*.

I suppose this is related to the GSSHS data extent 0 - 360 vs.
GRASS -180 - 180.

Strange. v.in.gshhs converts all lon coordinates to the range of -180
- 180. The only possible problem is Eurasiafrica (also converted to
-180 - 180), crossing the datum border, all other shorelines are not
crossing the datum border. I wrote the new v.in.gshhs port so that it
displays properly, but GRASS latlon handling is not fully consistent,
e.g. bounding boxes in vector topology ignore the datum border
problems whereas (most?) lib/gis functions take care of that. My
guess is that neither the wxGUI vdigit nor QGIS do this on-the-fly
wraparound like the GRASS display. No idea how to solve this. Glynn
once remarked that he tried to support -360 - 360 in latlon, and said
that this opens a problem box of Pandora. I tried myself, and found
the same problem box

I personally like the way QGIS or MapServer handles it - they just don't
care and don't try to treat lat-long data as connected at the datum
border.

This has the major plus that in those applications there is no problem
with manual browsing/editing world-wide lat-long data. One can pan and
zoom just like in any other coordinate system and the display coordinate
extent is not restricted.

GRASS automatic wrapparound is a feature that brings more troubles than
benefits to developers and users IMHO.

Maciek

--
Maciej Sieczka
http://www.sieczka.org

Markus_GRASS · March 18, 2009, 8:26am

Maciej Sieczka wrote:

I personally like the way QGIS or MapServer handles it - they just don't
care and don't try to treat lat-long data as connected at the datum
border.

So they stop displaying maps beyond -180 or 180? West of Alaska is nothing? If Asia is displayed west of Alaska and Alaska is displayed east of Asia, the spatial data viewer used (QGIS or MapServer) does wrap around.

This has the major plus that in those applications there is no problem
with manual browsing/editing world-wide lat-long data. One can pan and
zoom just like in any other coordinate system and the display coordinate
extent is not restricted.

I thought this is true for GRASS. Generally, if you zoom/pan beyond the extend of the feature (raster or vector), nothing is displayed. This should not happen in latlon, features must be wrapped around so that you can pan beyond -180 or 180 and everything that is supposed to be there is displayed, as in GRASS. Spatial data viewers/editors can not treat latlon like any other (like a proper) projection.

GRASS automatic wrapparound is a feature that brings more troubles than
benefits to developers and users IMHO.

There are two distinct issues, once the wrap around and once the restriction to -180 - 180 (causing wrap around of parts of a geometry feature only). They are connected but not identical. The horizontal lines are caused by the -180 - 180 restriction, not the wrap around. BTW, the restriction does not cause problems with rasters, in GRASS I can create a raster with West = 170 and East = -170, i.e. West > East, this is displayed properly as one block. It becomes difficult with vector features, because many operations would need to handle latlon different from proper projections, i.e. in this case wrap around, make sure lon differences are not > 180, run select by bbox twice etc. Currently the vector libs treat latlon pretty much like a proper projection, IMHO not a perfect solution. I'm not going to change current GRASS behaviour for vector handling in latlon on my own, this needs to be discussed by some more people before such drastic changes (from a user perspective) are introduced because it is about the general policy on how to cope with latlon.

Markus M

Maciej_Sieczka · March 18, 2009, 8:26pm

Markus Metz pisze:

Maciej Sieczka wrote:

I personally like the way QGIS or MapServer handles it - they just
don't care and don't try to treat lat-long data as connected at
the datum border.

So they stop displaying maps beyond -180 or 180? West of Alaska is nothing?

Yes.

This has the major plus that in those applications there is no problem with manual browsing/editing world-wide lat-long data. One
can pan and zoom just like in any other coordinate system and the
display coordinate extent is not restricted.

I thought this is true for GRASS. Generally, if you zoom/pan beyond the extend of the feature (raster or vector), nothing is displayed. This should not happen in latlon,

Why not?

features must be wrapped around

Why?

so that you can pan beyond -180 or 180 and everything that is supposed to be there is displayed, as in GRASS.

What about other world-wide CRSs, like those based on sinusoidal,
Goodge, Mercator projections - does GRASS wraps world edges for them as
well?

Spatial data viewers/editors can not treat latlon like any other (like a proper) projection.

GRASS automatic wrapparound is a feature that brings more troubles
than benefits to developers and users IMHO.

There are two distinct issues, once the wrap around and once the restriction to -180 - 180 (causing wrap around of parts of a geometry
feature only). They are connected but not identical. The horizontal
lines are caused by the -180 - 180 restriction, not the wrap around.
BTW, the restriction does not cause problems with rasters, in GRASS
I can create a raster with West = 170 and East = -170, i.e. West > East, this is displayed properly as one block. It becomes difficult with vector features, because many operations would need to handle latlon different from proper projections, i.e. in this case wrap around, make sure lon differences are not > 180, run select by bbox twice etc. Currently the vector libs treat latlon pretty much like a
proper projection, IMHO not a perfect solution. I'm not going to change current GRASS behaviour for vector handling in latlon on my own, this needs to be discussed by some more people before such drastic changes (from a user perspective) are introduced because it is about the general policy on how to cope with latlon.

Is there anything that could be done for v.in.gshs in GRASS 6.5 so that
it's output (when the input is a full GSHHS extent) would be free of the
horizontal longitudal extent-wide artifacts in case of:

1. editing in v.digit
2. exporting with v.out.ogr (that's another issue I haven't mentioned yet)
3. displaying and editing in QGIS

?

Maciek

--
Maciej Sieczka
http://www.sieczka.org

Markus_GRASS · March 20, 2009, 9:54am

Maciej Sieczka wrote:

Markus Metz pisze:

I thought this is true for GRASS. Generally, if you zoom/pan beyond the extend of the feature (raster or vector), nothing is displayed. This should not happen in latlon,

Why not?

Because it does not happen in GRASS:-) and because latlon is not a true projection and because IMHO this is a nice feature and because other GRASS developers have thought long about how to handle latlon in lib/gis and I see no reason to question the current solution.

What about other world-wide CRSs, like those based on sinusoidal,
Goodge, Mercator projections - does GRASS wraps world edges for them as
well?

No idea, you can check.

Is there anything that could be done for v.in.gshs in GRASS 6.5 so that
it's output (when the input is a full GSHHS extent) would be free of the
horizontal longitudal extent-wide artifacts in case of:

1. editing in v.digit
2. exporting with v.out.ogr (that's another issue I haven't mentioned yet)
3. displaying and editing in QGIS

Yes for 1, can't say for 2, No for 3. AFAICT, v.out.ogr itself does not produce these horizontal lines, they are produced by the application rendering the vector. Different applications have different, often incompatible policies, therefore it is not possible to accommodate every application. Either the GRASS display is 100% ok or digitizing is 100% ok, both is not possible with the full dataset. Actually, the mere display is always ok, but zooming to the selected map may or may not work. It does not work when the longitude map extends are > 360, but then the digitizer works... QGIS insists on straight lines also without wrap around in v.in.gshhs. It seems that QGIS wraps coordinates beyond -180 and 180 around into the -180, 180 range, then may draw these horizontal lines, and restricts the display area to -180, 180.
I want v.in.gshhs to produce output that is compatible with GRASS, and there I stop because it is not possible to accommodate every application (see also r.out.gdal discussion).
FWIW, I have submitted changes to make the digitizer work, no more horizontal lines there, tested with grass-6.5.

Markus

Maciej_Sieczka · March 22, 2009, 7:25pm

Markus Metz pisze:

Maciej Sieczka wrote:

Markus Metz pisze:

I thought this is true for GRASS. Generally, if you zoom/pan beyond the extend of the feature (raster or vector), nothing is displayed. This should not happen in latlon,

Why not?

Because it does not happen in GRASS:-) and because latlon is not a true projection and because IMHO this is a nice feature and because other GRASS developers have thought long about how to handle latlon in lib/gis and I see no reason to question the current solution.

Yet this nice feature makes it problematic to use v.in.gshhs output
outside GRASS, no matter if "GRASS approach" is better or not.

I have checked in OpenJump 1.2F, uDig 1.1.1, QGIS SVN trunk, gvSIG
1.1.2, UMN MapServer 5.2.0. All these apps render the full-extent GSHHS
data imported with v.in.gshhs and exported with v.out.ogr with those
horizontal artifacts.

In order to have the best of the two worlds could v.in.gshhs provide a
switch that would force the imported data to be contained exactly within
-180 - 180 longitudal extent? I.e. trim the borders exactly at -180 and 180?

What about other world-wide CRSs, like those based on sinusoidal, Goodge, Mercator projections - does GRASS wraps world edges for them as well?

No idea, you can check.

I'll try when I find time.

Is there anything that could be done for v.in.gshs in GRASS 6.5 so
that it's output (when the input is a full GSHHS extent) would be
free of the horizontal longitudal extent-wide artifacts in case
of:

1. editing in v.digit 2. exporting with v.out.ogr (that's another issue I haven't mentioned yet) 3. displaying and editing in QGIS

Yes for 1, can't say for 2, No for 3. AFAICT, v.out.ogr itself does not produce these horizontal lines, they are produced by the application rendering the vector. Different applications have different, often incompatible policies, therefore it is not possible
to accommodate every application. Either the GRASS display is 100%
ok or digitizing is 100% ok, both is not possible with the full dataset. Actually, the mere display is always ok, but zooming to the selected map may or may not work. It does not work when the longitude
map extends are > 360, but then the digitizer works... QGIS insists on straight lines also without wrap around in v.in.gshhs. It seems that QGIS wraps coordinates beyond -180 and 180 around into the -180,
180 range, then may draw these horizontal lines, and restricts the display area to -180, 180. I want v.in.gshhs to produce output that is compatible with GRASS, and there I stop because it is not possible
to accommodate every application (see also r.out.gdal discussion).

It's not that every application is different, but that GRASS is
different than all the rest in this regard - it seems. Again, I'm not
saying GRASS or you are wrong, but could there be an optional switch to
constrain GSHHS geometry exactly to -180 - 180 in v.in.gshhs to improve
GRASS interoperability regarding this? Or is that just too much work or
not interesting to you?

FWIW, I have submitted changes to make the digitizer work, no more horizontal lines there, tested with grass-6.5.

Thanks for looking into this. I tried it and no more horizontal
artifacts indeed, but with this modification the data imported with
v.in.gshhs and exported with v.out.ogr now extend over -180 - 180
slightly. Please see the attached screendump - violet is a
-180,180,-90,90 box, green is "crude" GSHHS. Same happens in OpenJump,
uDig, gvSIG 1.1.2, MapServer.

Maciek

--
Maciej Sieczka
http://www.sieczka.org

(attachments)

Markus_GRASS · March 22, 2009, 8:06pm

Maciej Sieczka wrote:

It's not that every application is different, but that GRASS is
different than all the rest in this regard - it seems. Again, I'm not
saying GRASS or you are wrong, but could there be an optional switch to
constrain GSHHS geometry exactly to -180 - 180 in v.in.gshhs to improve
GRASS interoperability regarding this? Or is that just too much work or
not interesting to you?

Well, the previous version restricted all coordinates to -180, 180, and GRASS displayed it all right, so I thought that's all right.

FWIW, I have submitted changes to make the digitizer work, no more horizontal lines there, tested with grass-6.5.

Thanks for looking into this. I tried it and no more horizontal
artifacts indeed, but with this modification the data imported with
v.in.gshhs and exported with v.out.ogr now extend over -180 - 180
slightly. Please see the attached screendump - violet is a
-180,180,-90,90 box, green is "crude" GSHHS. Same happens in OpenJump,
uDig, gvSIG 1.1.2, MapServer.

Yes, that was my workaround to avoid horizontal lines, now the map extends are -179, 190, roughly. You can check with v.info. BTW, that's why I don't understand why the screenshot extends beyond -180, it should extend only beyond 180, i.e. on the right side only.
I looked at other worldwide vector data like world boundaries and found that a common solution is to e.g. have two polygons instead of one for Eurasiafrica as in the GSHHS dataset. Eurasiafrica would stop at 180E with a straight vertical line, and the missing bit would start as a separate polygon with a straight vertical line at 180W, then going East. That would be the fool proof solution to avoid horizontal lines. But then you have vertical lines... Of course everything is technically possible, and I can modify v.in.gshhs so that it produces straight vertical lines instead of straight horizontal lines. I'm not really convinced, however, but don't mind if someone else wants to change this in v.in.gshhs. As long as the -r flag keeps its current behaviour and on-the-fly reprojection still works.

Markus M

Maciej_Sieczka · March 22, 2009, 8:41pm

Markus Metz pisze:

Maciej Sieczka wrote:

It's not that every application is different, but that GRASS is different than all the rest in this regard - it seems. Again, I'm not saying GRASS or you are wrong, but could there be an optional switch to constrain GSHHS geometry exactly to -180 - 180 in v.in.gshhs to improve GRASS interoperability regarding this? Or is
that just too much work or not interesting to you?

Well, the previous version restricted all coordinates to -180, 180, and GRASS displayed it all right, so I thought that's all right.

We seem to be using same terms for different things. By "restricted",
"constrained", "limited" to -180 - 180 I mean no feature extends over
-180 or 180 longitude vertical line. I.e that e.g. Alaska is split into
2 pieces.

FWIW, I have submitted changes to make the digitizer work, no more horizontal lines there, tested with grass-6.5.

Thanks for looking into this. I tried it and no more horizontal artifacts indeed, but with this modification the data imported with
v.in.gshhs and exported with v.out.ogr now extend over -180 - 180 slightly. Please see the attached screendump - violet is a -180,180,-90,90 box, green is "crude" GSHHS. Same happens in OpenJump, uDig, gvSIG 1.1.2, MapServer.

Yes, that was my workaround to avoid horizontal lines, now the map extends are -179, 190, roughly. You can check with v.info. BTW, that's why I don't understand why the screenshot extends beyond -180,
it should extend only beyond 180, i.e. on the right side only. I looked at other worldwide vector data like world boundaries and found
that a common solution is to e.g. have two polygons instead of one for Eurasiafrica as in the GSHHS dataset. Eurasiafrica would stop at
180E with a straight vertical line, and the missing bit would start
as a separate polygon with a straight vertical line at 180W, then going East.

That's exactly what I meant from the beginning :). Till now I've been
achieving this using a combination of GRASS vector modules, awk and
manual editing, but that's pretty tedious and error prone.

That would be the fool proof solution to avoid horizontal lines. But
then you have vertical lines...

If you decide it's worth your time to provide that *as an option* in
v.in.gshhs it'd be cool.

Of course everything is technically possible, and I can modify v.in.gshhs so that it produces straight vertical lines instead of straight horizontal lines. I'm not really convinced, however, but don't mind if someone else wants to change this in v.in.gshhs. As long as the -r flag keeps its current behaviour and on-the-fly reprojection still works.

Maciek

--
Maciej Sieczka
http://www.sieczka.org

Maciej_Sieczka · March 23, 2009, 8:02pm

Maciej Sieczka pisze:

I.e that e.g. Alaska is split into 2 pieces.

Me so dumb! That's Chukchi Peninsula (and also some of the Fiji Islands,
Wrangel Island).

Maciek

--
Maciej Sieczka
http://www.sieczka.org

Markus_GRASS · March 24, 2009, 6:20am

Maciej Sieczka wrote:

Markus Metz pisze:

That would be the fool proof solution to avoid horizontal lines. But
then you have vertical lines...

If you decide it's worth your time to provide that *as an option* in
v.in.gshhs it'd be cool.

Not right now, first I want to get grass7 topology substantially faster than grass6 topology, then I want to check an idea on how to make r.watershed in segmented mode faster (than r.terraflow), and then I could look at grass-addons modules. v.in.gshhs is currently very low on my priority list, and this suggested change is time consuming to implement.

Markus M

Maciej_Sieczka · March 24, 2009, 7:38pm

Markus Metz pisze:

Maciej Sieczka wrote:

Markus Metz pisze:

That would be the fool proof solution to avoid horizontal lines. But
then you have vertical lines...

If you decide it's worth your time to provide that *as an option* in
v.in.gshhs it'd be cool.

Not right now, first I want to get grass7 topology substantially faster than grass6 topology, then I want to check an idea on how to make r.watershed in segmented mode faster (than r.terraflow), and then I could look at grass-addons modules. v.in.gshhs is currently very low on my priority list, and this suggested change is time consuming to implement.

I'm really excited about your present and upcoming vector-related improvements. You are doing an extremely important job.

Regards,
Maciek

--
Maciej Sieczka
http://www.sieczka.org