[GRASS-dev] [GRASS GIS] #2010: r.in.wms2 fails to install on 6.x

GDAL · June 17, 2013, 4:15am

#2010: r.in.wms2 fails to install on 6.x
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: defect | Status: new
Priority: major | Milestone: 6.4.4
Component: Addons | Version: svn-releasebranch64
Keywords: r.in.wms2 | Platform: Linux
Cpu: All |
-----------------------+----------------------------------------------------
Hi,

the r.in.wms2 addon fails to build on 6.4 and devbr6 from g.extension(s).

The trouble seems to be in the Makefile, the helper scripts don't get
installed. It relies on Python.make which is badly out of date in 6.x
(still refers to SWIG), which may not be the main trouble, but it
certainly isn't helping. Once that is fixed there may be more grass7->6
porting needed.

It is still somewhat unknown to me which python scripts need .bat wrappers
& presence in the PATH to work in grass6.

currently (amazingly?) r.in.wms[.sh] run from the command line is the only
working option on wingrass. see #820, #1163.

thanks,
Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2010>
GRASS GIS <http://grass.osgeo.org>

GDAL · June 25, 2013, 4:03am

#2010: r.in.wms2 fails to install on 6.x
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: defect | Status: new
Priority: major | Milestone: 6.4.4
Component: Addons | Version: svn-releasebranch64
Keywords: r.in.wms2 | Platform: Linux
Cpu: All |
-----------------------+----------------------------------------------------

Comment(by hamish):

see also #1555, #1196 and #1768.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2010#comment:1>
GRASS GIS <http://grass.osgeo.org>

GDAL · July 2, 2013, 9:38pm

#2010: r.in.wms2 fails to install on 6.x
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: defect | Status: new
Priority: major | Milestone: 6.4.4
Component: Addons | Version: svn-releasebranch64
Keywords: r.in.wms2 | Platform: Linux
Cpu: All |
-----------------------+----------------------------------------------------

Comment(by hamish):

Hi,

I've now done a bit more on this in grass6 addons svn. It now installs
with 'make make MODULE_TOPDIR=$GISBASE' or similar from the source dir; I
haven't tried either of the g.extensions yet though.

Also I did a bit more on the port to G6, the module runs now using the
default settings, but I didn't test it heavily, just Richard's neowms
example using the default download method
{{{
r.in.wms2.py url="http://neowms.sci.gsfc.nasa.gov/wms/wms?" \
out=solar.8day layer=CERES_INSOL_E format=png
}}}

Probably a number of fixes there should be considered for porting back to
trunk.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2010#comment:2>
GRASS GIS <http://grass.osgeo.org>

GDAL · July 3, 2013, 10:01am

#2010: r.in.wms2 fails to install on 6.x
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: defect | Status: new
Priority: major | Milestone: 6.4.4
Component: Addons | Version: svn-releasebranch64
Keywords: r.in.wms2 | Platform: Linux
Cpu: All |
-----------------------+----------------------------------------------------

Comment(by turek):

Hi Hamish,

thanks for the fixes. I have ported back to G7 most of them (r56991,
r56992 and r56993).

So far the name of the o flag remains.
In description of s flag there is written:
Skip requests for fully transparent data

I am not sure if it is correct description. If this flag is not present,
the data are fetched also with transparent layer (if available). Otherwise
you can use bgcolor param with this flag and specify color for these
transparent areas. The o was chosen as abbreviation for opaque.

Spaces between options are still kept, it seems more clear to me than one
block of ~20 options. But it also can be changed.

Stepan

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2010#comment:3>
GRASS GIS <http://grass.osgeo.org>

GDAL · July 3, 2013, 11:05am

#2010: r.in.wms2 fails to install on 6.x
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: defect | Status: new
Priority: major | Milestone: 6.4.4
Component: Addons | Version: svn-releasebranch64
Keywords: r.in.wms2 | Platform: Linux
Cpu: All |
-----------------------+----------------------------------------------------

Comment(by hamish):

Replying to [comment:3 turek]:
> Hi Hamish,
>
> thanks for the fixes. I have ported back to G7 most of them

I hope you found them useful, if they are bad you don't have to listen to
me, it's your module after all.

> So far the name of the o flag remains.
> In description of s flag there is written:
> Skip requests for fully transparent data
>
> I am not sure if it is correct description. If this flag is not
> present, the data are fetched also with transparent layer (if
> available). Otherwise you can use bgcolor param with this flag
> and specify color for these transparent areas.

I misunderstood it then, in one of the TWMS/r.out.kml script I have
somewhere it checks if the returned tile is all-white and skips
warping/mosaicking that so that part of the background becomes NULL not
255. Is the idea to make a complete rectangle instead of missing tiles in
the corners? I was wondering how the bgcolor option fit in. If that
supports standard GRASS colors you might consider adding:
{{{
#% gisprompt: old,color_none,color
}}}
so from the module GUI the user gets the color-picking tool.

> The o was chosen as abbreviation for opaque.

I prefer to avoid -o,-q,-v as option letters if there are others
available, since it is easy for users to confuse them with --o,--q,--v.
fwiw I also like to avoid -l, -1, and -I, but for certain-font reasons.

> Spaces between options are still kept, it seems more clear to me
> than one block of ~20 options. But it also can be changed.

shrug, author's choice; the convention elsewhere is compressed but it
doesn't change the running of the script at all so just cosmetic, and I
agree it gets a bit hard to read when there are many options. Don't fear
the whitespace.

a couple observations/wishes:
  * wx: switching from 1.1.1 to 1.3.0 clears selection and resizes window
height?
  * wx: how about being able to filter the list like the location wizard
has for projection type Search? (often the servers have 50+ layers to
search)
  * support for png8 image format? (like qgis has)
  * I get an error when BGCOLOR is unset, does the spec require it or just
a picky WMS server?
{{{ <ServiceException>
BGCOLOR incorrectly specified (0xRRGGBB format expected)
</ServiceException></ServiceExceptionReport>
}}}
  * module gui: Interpolation type isn't part of the server request. ->
Optional gui tab?

Also, I'm seeing a unicode error vs. layer Title string in the 6.4.3svn's
r.in.wms1's wxGUI wrapper. It's not to do with r.in.wms2.py, so I'll post
that to the -dev ML.

regards,
Hamish

ps- the WMS/WFS I'm using to test with:
http://wiki.openstreetmap.org/wiki/New_Zealand/Imagery

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2010#comment:4>
GRASS GIS <http://grass.osgeo.org>

Hamish · July 3, 2013, 11:13am

I wrote:

Also, I'm seeing a unicode error vs. layer Title string in the 6.4.3svn's
r.in.wms1's wxGUI wrapper. It's not to do with r.in.wms2.py, so I'll
post that to the -dev ML.

details on..

the WMS/WFS I'm using to test with:
http://wiki.openstreetmap.org/wiki/New_Zealand/Imagery

In 6.4.3svn wxGUI File -> import raster -> r.in.wms (wrapper) I'm getting the following error when the parsed list of WMS layer Titles contains a non-ascii char, in this case on of them contains a long -- hyphen.

Any ideas on a quick fix? It didn't like ", unicode = True" where I put it.
All's well in trunk.

Traceback (most recent call last):
File "/home/hamish/src/grass/svn/grass65/dist.x86_64
-unknown-linux-gnu/etc/wxpython/modules/ogc_services.py",
line 216, in OnConnect

self.list.LoadData(layers)
File "/home/hamish/src/grass/svn/grass65/dist.x86_64
-unknown-linux-gnu/etc/wxpython/modules/ogc_services.py",
line 273, in LoadData

self.SetItemText(lchild, title, 1)
File "/usr/lib64/python2.6/dist-
packages/wx-2.8-gtk2-unicode/wx/gizmos.py", line 647, in
SetItemText

return _gizmos.TreeListCtrl_SetItemText(*args, **kwargs)
UnicodeDecodeError
:
'ascii' codec can't decode byte 0xe2 in position 36: ordinal
not in range(128)

thanks,
Hamish

Hamish · July 4, 2013, 10:01pm

to add .decode('utf8') somewhere?

http://washort.twistedmatrix.com/2010/11/unicode-in-python-and-how-to-prevent-it.html

thanks,
Hamish

I wrote:

Also, I'm seeing a unicode error vs. layer Title string in the

6.4.3svn's

r.in.wms1's wxGUI wrapper. It's not to do with r.in.wms2.py, so

I'll

post that to the -dev ML.

details on..

the WMS/WFS I'm using to test with:
http://wiki.openstreetmap.org/wiki/New_Zealand/Imagery

In 6.4.3svn wxGUI File -> import raster -> r.in.wms (wrapper) I'm
getting the following error when the parsed list of WMS layer Titles contains a
non-ascii char, in this case on of them contains a long -- hyphen.

Any ideas on a quick fix? It didn't like ", unicode = True" where
I put it.
All's well in trunk.

Traceback (most recent call last):
File "/home/hamish/src/grass/svn/grass65/dist.x86_64
-unknown-linux-gnu/etc/wxpython/modules/ogc_services.py",
line 216, in OnConnect

self.list.LoadData(layers)
File "/home/hamish/src/grass/svn/grass65/dist.x86_64
-unknown-linux-gnu/etc/wxpython/modules/ogc_services.py",
line 273, in LoadData

self.SetItemText(lchild, title, 1)
File "/usr/lib64/python2.6/dist-
packages/wx-2.8-gtk2-unicode/wx/gizmos.py", line 647, in
SetItemText

return _gizmos.TreeListCtrl_SetItemText(*args, **kwargs)
UnicodeDecodeError
:
'ascii' codec can't decode byte 0xe2 in position 36: ordinal
not in range(128)

GDAL · July 6, 2013, 10:31am

#2010: r.in.wms2 fails to install on 6.x
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: defect | Status: new
Priority: major | Milestone: 6.4.4
Component: Addons | Version: svn-releasebranch64
Keywords: r.in.wms2 | Platform: Linux
Cpu: All |
-----------------------+----------------------------------------------------

Comment(by turek):

Replying to [comment:4 hamish]:

Hi Hamish,
I am sorry for late response.

> Replying to [comment:3 turek]:

> > thanks for the fixes. I have ported back to G7 most of them
>
> I hope you found them useful, if they are bad you don't have to listen
to me, it's your module after all.

Yes, they are useful!

> I prefer to avoid -o,-q,-v as option letters if there are others
available, since it is easy for users to confuse them with --o,--q,--v.
fwiw I also like to avoid -l, -1, and -I, but for certain-font reasons.

That is good point. Do you have any other idea for name of this flag or
just changed it to -s?

>
> a couple observations/wishes:
> * wx: switching from 1.1.1 to 1.3.0 clears selection and resizes window
height?

The selection is not cleared, there exists independent panels for every
web service because it is not guaranteed that capabilities file will be
same for WMS 1.3.0 and 1.1.1 (if you select something it stays selected in
the web service panel). Ideal solution would be existence of one panel,
which would adapt according to set web service. But still it should
probably behave same way. The resize problem should be fixed in r57025.

> * wx: how about being able to filter the list like the location wizard
has for projection type Search? (often the servers have 50+ layers to
search)

It is good idea.
Please add it to: https://trac.osgeo.org/grass/wiki/WebServices. There it
should not be forgotten. Also you can add there the #% gisprompt:
old,color_none,color idea.

> * support for png8 image format? (like qgis has)
Done in r57024, r57023. It need to be tested.

> * I get an error when BGCOLOR is unset, does the spec require it or
just a picky WMS server?

Should be fixed in r57022.

> * module gui: Interpolation type isn't part of the server request. ->
Optional gui tab?
Good idea! Done in r57027 and r57026.

Best,
Stepan

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2010#comment:5>
GRASS GIS <http://grass.osgeo.org>

GDAL · July 6, 2013, 10:55am

#2010: r.in.wms2 fails to install on 6.x
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: defect | Status: new
Priority: major | Milestone: 6.4.4
Component: Addons | Version: svn-releasebranch64
Keywords: r.in.wms2 | Platform: Linux
Cpu: All |
-----------------------+----------------------------------------------------

Comment(by hamish):

Hi,

I will try to test the new changes tomorrow, some quick comments though:

opaque ~ solid, so -s isn't too far away. I'm still trying to put what
that flag does into better words, but not having much luck.

in devbr6 I just added r57028 I added ".decode('utf8')" to the WMS layer
title-reading code to avoid a traceback*. I'm not sure if it will fill the
gui window with ugly \u1234 text or if it will display them ok. do you
know a utf8 enabled non-english wms server to test with? (works ok in
trunk)

[*] traceback:
{{{
Re: [GRASS-dev] [GRASS GIS] #2010: r.in.wms2 fails to install on 6.x
grass-dev ML 2013/7/5
...
> return _gizmos.TreeListCtrl_SetItemText(*args, **kwargs)
> UnicodeDecodeError
> :
> 'ascii' codec can't decode byte 0xe2 in position 36: ordinal
> not in range(128)
}}}

thanks,
Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2010#comment:6>
GRASS GIS <http://grass.osgeo.org>

GDAL · July 7, 2013, 10:36am

#2010: r.in.wms2 fails to install on 6.x
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: defect | Status: new
Priority: major | Milestone: 6.4.4
Component: Addons | Version: svn-releasebranch64
Keywords: r.in.wms2 | Platform: Linux
Cpu: All |
-----------------------+----------------------------------------------------

Comment(by hamish):

Hi,

some notes & observations:

* png8 support missing for WMS 1.3.0

* color picker greyed out in the "Add web service layer" WMS gui.

* loss of region precision in the saved map,

computational region:
{{{
g.region -p
north: 4941600
south: 4848400
west: 1343900
east: 1436600
rows: 1864
cols: 1854
}}}

right click on WMS layer, save from computational region. new raster:
{{{
r.info
  | Rows: 1861
  | Columns: 1852
  | N: 4941600.00001201 S: 4848400.00000474 Res: 50 |
  | E: 1436600.00000344 W: 1343899.99999621 Res: 50 |
}}}

* WFS mixes in WMS layers, and has lot of traceback parse errors, but
realistic data plots; can't query it in the map display window though. is
WFS supposed to work yet or is it still a work in progress?

thanks,
Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2010#comment:7>
GRASS GIS <http://grass.osgeo.org>

GDAL · July 8, 2013, 1:09pm

#2010: r.in.wms2 fails to install on 6.x
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: defect | Status: new
Priority: major | Milestone: 6.4.4
Component: Addons | Version: svn-releasebranch64
Keywords: r.in.wms2 | Platform: Linux
Cpu: All |
-----------------------+----------------------------------------------------

Comment(by turek):

Hi,

Replying to [comment:7 hamish]:
>
> * png8 support missing for WMS 1.3.0
In capabilties file of WMS 1.1.1 there are two formats: image/png;
mode=8bit and image/png8. In WMS 1.3.0 there is only image/png; mode=8bit
format. I am not sure which one to use. Currently is used image/png8
therefore you can not see png8 in WMS 1.3.0. Maybe it should be used
image/png; mode=8bit, because WMS 1.3.0 is more restrictive regarding
formats. It requires only MIME types formats. WMS 1.1.1. is more
benevolent. It could mean that image/png; mode=8bit is MIME type and the
other is not. Or it just changes randomly from server to server?

> * color picker greyed out in the "Add web service layer" WMS gui.

That is not very user friendly. If you check do not request transparent
data, the button will be activated. Should be the tooltip added there or
the button should be hidden if the choice is not checked? Or just let it
activated also when it is unchecked?

> * loss of region precision in the saved map,
>

Probably it is caused by reprojection (using gdalwarp).

> * WFS mixes in WMS layers, and has lot of traceback parse errors, but
realistic data plots; can't query it in the map display window though. is
WFS supposed to work yet or is it still a work in progress?

WFS is not supported, parser just tries interpret capabilities file
somehow. The parser is written in tolerant way, so it does not enforce
standards strictly, instead of this it tries to interpret everything what
it can understood.

Best
Stepan

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2010#comment:8>
GRASS GIS <http://grass.osgeo.org>