[GRASS-dev] r.unpack: unhelpful error message when projection info does not match

Hi,

the error (example UTM map import into LAEA location):

r.unpack landcover_1m.pack
ERROR: Projection information does not match. Aborting.

is rather unhelpful. The code is:

    # check projection compatibility in a rather crappy way
    if not grass.compare_key_value_text_files('PROJ_INFO',
os.path.join(mset_dir, '..', 'PERMANENT', 'PROJ_INFO')) or \
            not grass.compare_key_value_text_files('PROJ_UNITS',
os.path.join(mset_dir, '..', 'PERMANENT', 'PROJ_UNITS')):
        if flags['o']:
            grass.warning(_("Projection information does not match.
Proceeding..."))
        else:
            grass.fatal(_("Projection information does not match. Aborting."))

I would suggest to print a "diff -u file1 file2" rather than nothing
as now (perhaps Python offers a nice way to show the wrong keys to the
user).
Or are there other options?

Markus

Hi,

please test r57969. You should see a diff if the proj_info files don’t match.

Anna

···

On Wed, Oct 9, 2013 at 6:08 PM, Markus Neteler <neteler@osgeo.org> wrote:

Hi,

the error (example UTM map import into LAEA location):

r.unpack landcover_1m.pack
ERROR: Projection information does not match. Aborting.

is rather unhelpful. The code is:

check projection compatibility in a rather crappy way

if not grass.compare_key_value_text_files(‘PROJ_INFO’,
os.path.join(mset_dir, ‘…’, ‘PERMANENT’, ‘PROJ_INFO’)) or
not grass.compare_key_value_text_files(‘PROJ_UNITS’,
os.path.join(mset_dir, ‘…’, ‘PERMANENT’, ‘PROJ_UNITS’)):
if flags[‘o’]:
grass.warning((“Projection information does not match.
Proceeding…”))
else:
grass.fatal(
(“Projection information does not match. Aborting.”))

I would suggest to print a “diff -u file1 file2” rather than nothing
as now (perhaps Python offers a nice way to show the wrong keys to the
user).
Or are there other options?

Markus


grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Dear Anna,

thank you, a lot of work for you!

Here my test case result (certainly a bit extreme but illustrating the case):

GRASS 7.0.svn (europe_laea):~ > r.unpack landcover_1m.pack
WARNING: Difference between PROJ_INFO file of packed map and of current
         location:
         - name: Lambert Conformal Conic
         + name: Lambert Azimuthal Equal Area
         - proj: lcc
         ? ^^
         + proj: laea
         ? ^^^
         + datum: etrs89
         + ellps: grs80
         - datum: nad83
         - a: 6378137.0
         - es: 0.006694380022900787
         - lat_1: 36.16666666666666
         - lat_2: 34.33333333333334
         - lat_0: 33.75
         ? ----
         + lat_0: 52
         ? +
         - lon_0: -79
         ? ^^^
         + lon_0: 10
         ? ^^
         - x_0: 609601.22
         - y_0: 0
         + x_0: 4321000
         + y_0: 3210000
         no_defs: defined
WARNING: Difference between PROJ_UNITS file of packed map and of current
         location:
         - unit: Meter
         ? ^ -
         + unit: metre
         ? ^ +
         - units: Meters
         ? ^ -
         + units: metres
         ? ^ +
         meters: 1
ERROR: Projection information does not match. Aborting.

For other users: Anna improved v.unpack likewise.

thanks again,
Markus

On Thu, Oct 10, 2013 at 8:09 AM, Markus Neteler <neteler@osgeo.org> wrote:

Dear Anna,

thank you, a lot of work for you!

... back to this topic, I would be happy to find a solution for US vs
UK English:

GRASS 7.0.svn (patUTM32):~ > r.unpack x60030_2000.green.histo2000_g.pack
WARNING: Difference between PROJ_INFO file of packed map and of current
         location:
         - name: Universal Transverse Mercator
         ? ^^
         + name: Universe Transverse Mercator
         ? ^
         proj: utm
         datum: wgs84
         ellps: wgs84
         zone: 32
         no_defs: defined
WARNING: Difference between PROJ_UNITS file of packed map and of current
         location:
         - unit: metre
         ? -
         + unit: meter
         ? +
         - units: metres
         ? -
         + units: meters
         ? +
         meters: 1
ERROR: Projection information does not match. Aborting.

r.unpack calls the Python core function compare_key_value_text_files()
http://grass.osgeo.org/programming7/namespacepython_1_1script_1_1core.html#a3ad1670d5f7b4759935d4b38292fa7af
and then does the comparison key by key (so, it fails here).

I think that we need to polish the language differences in r.unpack
prior to the comparison. Maybe via lookup dictionary which can be
expanded in future when more cases come up?

Universe Transverse Mercator --> Universal Transverse Mercator
metre --> meter
metres --> meters
kilometre --> kilometre
kilometres --> kilometres

See also the corresponding C functions:
http://trac.osgeo.org/grass/browser/grass/trunk/lib/gis/proj3.c?rev=58633#L48

The same fix would then be needed in v.unpack.

?
Markus

On 26 February 2014 18:04, Markus Neteler <neteler@osgeo.org> wrote:

Hi all

... back to this topic, I would be happy to find a solution for US vs
UK English:

GRASS 7.0.svn (patUTM32):~ > r.unpack x60030_2000.green.histo2000_g.pack
WARNING: Difference between PROJ_INFO file of packed map and of current
         location:
         - name: Universal Transverse Mercator
         ? ^^
         + name: Universe Transverse Mercator
         ? ^
         proj: utm
         datum: wgs84
         ellps: wgs84
         zone: 32
         no_defs: defined
WARNING: Difference between PROJ_UNITS file of packed map and of current
         location:
         - unit: metre
         ? -
         + unit: meter
         ? +
         - units: metres
         ? -
         + units: meters
         ? +
         meters: 1
ERROR: Projection information does not match. Aborting.

r.unpack calls the Python core function compare_key_value_text_files()
http://grass.osgeo.org/programming7/namespacepython_1_1script_1_1core.html#a3ad1670d5f7b4759935d4b38292fa7af
and then does the comparison key by key (so, it fails here).

I think that we need to polish the language differences in r.unpack
prior to the comparison. Maybe via lookup dictionary which can be
expanded in future when more cases come up?

Universe Transverse Mercator --> Universal Transverse Mercator
metre --> meter
metres --> meters
kilometre --> kilometre
kilometres --> kilometres

See also the corresponding C functions:
http://trac.osgeo.org/grass/browser/grass/trunk/lib/gis/proj3.c?rev=58633#L48

The same fix would then be needed in v.unpack.

?

I solved the problem for units and projection system in r59147.
The is stil a bug in compare_key_value_text_files because if second
file has more keys this is not recognize.
I would like to add at line 913 this:

if dict_a.keys() != dict_b.keys():
    return False

but this could broke all the system of compare_key_value_text_files function
what do you think?

Markus

--
ciao
Luca

http://gis.cri.fmach.it/delucchi/
www.lucadelu.org

On Thu, Feb 27, 2014 at 11:32 AM, Luca Delucchi <lucadeluge@gmail.com> wrote:

On 26 February 2014 18:04, Markus Neteler <neteler@osgeo.org> wrote:

...

I solved the problem for units and projection system in r59147.

fine, thanks.

The is stil a bug in compare_key_value_text_files because if second
file has more keys this is not recognize.
I would like to add at line 913 this:

if dict_a.keys() != dict_b.keys():
    return False

but this could broke all the system of compare_key_value_text_files function
what do you think?

Good (Python) question.

To illustrate the issue to all:

# being in a UTM South location, I try to import an UTM North map:
GRASS 7.0.svn (utm32s):/incoming/proj_mess > r.unpack
x60030_2000.green.histo2000_g.pack
Raster map <x60030_2000.green.histo2000_g> unpacked

--> should have been rejected

GRASS 7.0.svn (utm32s):/incoming/proj_mess > g.proj -p
-PROJ_INFO-------------------------------------------------
name : Universal Transverse Mercator
proj : utm
datum : wgs84
ellps : wgs84
zone : 32
south : defined <<---- not tested properly
no_defs : defined
-PROJ_UNITS------------------------------------------------
unit : metre
units : metres
meters : 1

Ideas? thanks

Markus

On 28 February 2014 15:38, Markus Neteler <neteler@osgeo.org> wrote:

Ideas? thanks

I think the attached patch should fix the problem (I have no time to
test it), but I don't know if it broke something else.
I saw the function compare_key_value_text_files is used only in
r/v.unpack and temporal/stds_import.py, Soeren the attached patch
could create problem in this file?

Markus

--
ciao
Luca

http://gis.cri.fmach.it/delucchi/
www.lucadelu.org

(attachments)

python_core.diff (2.82 KB)

Luca Delucchi wrote:

I would like to add at line 913 this:

if dict_a.keys() != dict_b.keys():
    return False

The ordering of the elements in the list returned by .keys() isn't
defined. E.g. for some dictionary d,

  d.keys() == d.copy().keys()

can be (and typically is) False.

Convert the keys to sets, i.e.

  if set(dict_a.iterkeys()) == set(dict_b.iterkeys()):
      return False

Or sort them; I'm not sure which is faster overall.

Aside from that, I don't think that compare_key_value_text_files()
(and the functions it uses) belongs in grass.script.core; they're too
specialised. Likewise for create_location().

--
Glynn Clements <glynn@gclements.plus.com>

On Fri, Feb 28, 2014 at 5:14 PM, Luca Delucchi <lucadeluge@gmail.com> wrote:

On 28 February 2014 15:38, Markus Neteler <neteler@osgeo.org> wrote:

>
> Ideas? thanks
>

I think the attached patch should fix the problem (I have no time to
test it), but I don't know if it broke something else.
I saw the function compare_key_value_text_files is used only in
r/v.unpack and temporal/stds_import.py, Soeren the attached patch
could create problem in this file?

The diff looks reasonable. Since it is used in the temporal framework for
the same purpose, it should be fine I think. As Glynn mentioned, this
function shouldn't be in core, so how about creating
python/scripts/utils.py? Some of the functions in gui could be moved there
too.

Anna

> Markus
>

--
ciao
Luca

http://gis.cri.fmach.it/delucchi/
www.lucadelu.org

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

On 2 March 2014 04:23, Anna Petrášová <kratochanna@gmail.com> wrote:

On Fri, Feb 28, 2014 at 5:14 PM, Luca Delucchi <lucadeluge@gmail.com> wrote:

On 28 February 2014 15:38, Markus Neteler <neteler@osgeo.org> wrote:

>
> Ideas? thanks
>

I think the attached patch should fix the problem (I have no time to
test it), but I don't know if it broke something else.
I saw the function compare_key_value_text_files is used only in
r/v.unpack and temporal/stds_import.py, Soeren the attached patch
could create problem in this file?

The diff looks reasonable. Since it is used in the temporal framework for
the same purpose, it should be fine I think.

committed in r59173.

As Glynn mentioned, this
function shouldn't be in core, so how about creating
python/scripts/utils.py? Some of the functions in gui could be moved there
too.

I would like to call it python/scripts/functions.py because we already
have python/pygrass/functions.py, what do you think?

Anna

--
ciao
Luca

http://gis.cri.fmach.it/delucchi/
www.lucadelu.org