[GRASS-dev] installing metadata addon fails

ecodiv · January 8, 2015, 3:37pm

The installation of the metadata addon on GRASS 7 (trunk) fails, with the message below. Any ideas?

Fetching <wx.metadata> from GRASS-Addons SVN repository (be patient)…
Compiling…
Makefile:19: warning: overriding commands for target
/tmp/tmpF3gAGc/wx.metadata/etc' /usr/local/grass7/grass-7.1.svn/include/Make/Rules.make:16: warning: ignoring old commands for target /tmp/tmpF3gAGc/wx.metadata/etc’
Makefile:18: warning: overriding commands for target
/tmp/tmpF3gAGc/wx.metadata/etc' /usr/local/grass7/grass-7.1.svn/include/Make/Rules.make:16: warning: ignoring old commands for target /tmp/tmpF3gAGc/wx.metadata/etc’
Traceback (most recent call last):
File “/tmp/tmpF3gAGc/wx.metadata/scripts/g.gui.metadata”,
line 46, in
import mdgrass
ImportError: No module named mdgrass
make[2]: *** [g.gui.metadata.tmp.html] Error 1
Installing…
Makefile:19: warning: overriding commands for target
/tmp/tmpF3gAGc/wx.metadata/etc' /usr/local/grass7/grass-7.1.svn/include/Make/Rules.make:16: warning: ignoring old commands for target /tmp/tmpF3gAGc/wx.metadata/etc’
Makefile:18: warning: overriding commands for target
/tmp/tmpF3gAGc/wx.metadata/etc' /usr/local/grass7/grass-7.1.svn/include/Make/Rules.make:16: warning: ignoring old commands for target /tmp/tmpF3gAGc/wx.metadata/etc’
/usr/bin/install: cannot stat ‘/tmp/tmpF3gAGc/wx.metadata/
docs/html/g.gui.metadata.html’: No such file or directory
make[1]: *** [install] Error 1
make: *** [install] Error 2
WARNING: Installation failed, sorry. Please check above error messages.

martinl · January 8, 2015, 5:44pm

Hi,

2015-01-08 16:37 GMT+01:00 Paulo van Breugel <p.vanbreugel@gmail.com>:

The installation of the metadata addon on GRASS 7 (trunk) fails, with the
message below. Any ideas?

I have fixed installation in r63999-64001. But `r.info.iso` fails

r.info.iso dmt
WARNING: date of creation: unknown date format
Traceback (most recent call last):
  File "/home/martin/.grass7/addons/scripts/r.info.iso", line 83, in <module>
    main()
  File "/home/martin/.grass7/addons/scripts/r.info.iso", line 75, in main
    overwrite=os.getenv('GRASS_OVERWRITE', False))
  File "/home/martin/.grass7/addons/etc/mdlib/mdgrass.py", line 346, in saveXML
    template = env.get_template(self.template)
  File "/usr/local/lib/python2.7/dist-packages/Jinja2-2.8_devdev_20140629-py2.7.egg/jinja2/environment.py",
line 801, in get_template
    return self._load_template(name, self.make_globals(globals))
  File "/usr/local/lib/python2.7/dist-packages/Jinja2-2.8_devdev_20140629-py2.7.egg/jinja2/environment.py",
line 763, in _load_template
    cache_key = self.loader.get_source(self, name)[1]
  File "/usr/local/lib/python2.7/dist-packages/Jinja2-2.8_devdev_20140629-py2.7.egg/jinja2/loaders.py",
line 187, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound:
/home/martin/.grass7/addons/etc/templates/basicTemplate.xml

The mystery is that the file
(/home/martin/.grass7/addons/etc/templates/basicTemplate.xml) exists:

$ file /home/martin/.grass7/addons/etc/templates/basicTemplate.xml
/home/martin/.grass7/addons/etc/templates/basicTemplate.xml: XML document text

I hope that the original author (Matej in cc) will know more. Martin

--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.eu/mentors/landa

ecodiv · January 9, 2015, 10:12am

Installation went fine… but in g.gui.metadata when trying to edit /create metadata for a layer, I get the following error (not sure if this is the same as the one you report)

GRASS 7.1.svn (LatLongTest):~ > g.gui.metadata
Traceback (most recent call last):
File “/home/paulo/.grass7/addons/scripts/g.gui.metadata”, line 1031, in onEdit
ok = self.GetParent().onEditMapMetadata()
File “/home/paulo/.grass7/addons/scripts/g.gui.metadata”, line 257, in onEditMapMetadata
self.mdCreator.createGrassBasicISO()
File “/home/paulo/.grass7/addons/etc/mdlib/mdgrass.py”, line 186, in createGrassBasicISO
self.md.identification.title = mdutil.replaceXMLReservedChar(self.md_grass[‘title’])
KeyError: ‘title’

···

On Thu, Jan 8, 2015 at 6:44 PM, Martin Landa <landa.martin@gmail.com> wrote:

Hi,

2015-01-08 16:37 GMT+01:00 Paulo van Breugel <p.vanbreugel@gmail.com>:

The installation of the metadata addon on GRASS 7 (trunk) fails, with the
message below. Any ideas?

I have fixed installation in r63999-64001. But r.info.iso fails

r.info.iso dmt
WARNING: date of creation: unknown date format
Traceback (most recent call last):
File “/home/martin/.grass7/addons/scripts/r.info.iso”, line 83, in
main()
File “/home/martin/.grass7/addons/scripts/r.info.iso”, line 75, in main
overwrite=os.getenv(‘GRASS_OVERWRITE’, False))
File “/home/martin/.grass7/addons/etc/mdlib/mdgrass.py”, line 346, in saveXML
template = env.get_template(self.template)
File “/usr/local/lib/python2.7/dist-packages/Jinja2-2.8_devdev_20140629-py2.7.egg/jinja2/environment.py”,
line 801, in get_template
return self._load_template(name, self.make_globals(globals))
File “/usr/local/lib/python2.7/dist-packages/Jinja2-2.8_devdev_20140629-py2.7.egg/jinja2/environment.py”,
line 763, in _load_template
cache_key = self.loader.get_source(self, name)[1]
File “/usr/local/lib/python2.7/dist-packages/Jinja2-2.8_devdev_20140629-py2.7.egg/jinja2/loaders.py”,
line 187, in get_source
raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound:
/home/martin/.grass7/addons/etc/templates/basicTemplate.xml

The mystery is that the file
(/home/martin/.grass7/addons/etc/templates/basicTemplate.xml) exists:

$ file /home/martin/.grass7/addons/etc/templates/basicTemplate.xml
/home/martin/.grass7/addons/etc/templates/basicTemplate.xml: XML document text

I hope that the original author (Matej in cc) will know more. Martin

–
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.eu/mentors/landa

mlennert · January 9, 2015, 10:44am

On 08/01/15 18:44, Martin Landa wrote:

Hi,

2015-01-08 16:37 GMT+01:00 Paulo van Breugel <p.vanbreugel@gmail.com>:

The installation of the metadata addon on GRASS 7 (trunk) fails, with the
message below. Any ideas?

I have fixed installation in r63999-64001.

Is there some documentation about how to write Makefiles for addons that use several .py files ? Similar issues have come up with different addons (cf #2480 and #2534) and it shouldn't be up to you to correct all these issues. Maybe some clear doc could help ?

Moritz

Glynn_Clements1 · January 10, 2015, 9:28pm

Moritz Lennert wrote:

> 2015-01-08 16:37 GMT+01:00 Paulo van Breugel <p.vanbreugel@gmail.com>:
>> The installation of the metadata addon on GRASS 7 (trunk) fails, with the
>> message below. Any ideas?
>
> I have fixed installation in r63999-64001.

Is there some documentation about how to write Makefiles for addons that
use several .py files ? Similar issues have come up with different
addons (cf #2480 and #2534) and it shouldn't be up to you to correct all
these issues. Maybe some clear doc could help ?

The "documentation" for writing Makefiles boils down to: try to find
something similar in the main GRASS tree and use its Makefile as a
reference; if there isn't one, or if that doesn't work, ask on the
developers' list.

Writing documentation for this will result in either wasted effort
(from describing situations which never actually happen) or inadequate
documentation (from failing to describe situations which actually
happen) or (most likely) both.

A couple of points about wx.metadata specifically:

1. Using "parsubdirs" won't work as, because mdlib (presumably) needs
to be built before any of the modules.

2. Files should not be installed using "cp"; use $(INSTALL) for
anything which should be installed with execute permission or
$(INSTALL_DATA) for anything without it.

3. Directories shouldn't be copied with "cp -r". Rather, the
individual destination files should be listed as pre-requisites, so
that "building" the target "builds" the destination files (typically
via pattern rules).

wx.metadata/Makefile should probably include "templates" and "config"
in $(SUBDIRS). These would have their own Makefiles; e.g. for
templates/Makefile, something like:

include $(MODULE_TOPDIR)/include/Make/Other.make

  DSTDIR = $(ETC)/mdlib/templates

  SRCFILES := $(wildcard *.xml)
  DSTFILES := $(patsubst %.xml,$(DSTDIR)/%.xml,$(SRCFILES))

  default: $(DSTFILES)

  $(DSTDIR)/%.xml: %.xml:
    $(INSTALL_DATA) $< $@

This should allow wx.metadata/Makefile to be a simple "directory"
Makefile (i.e. set SUBDIRS, include Dir.make, "default: subdirs").

--
Glynn Clements <glynn@gclements.plus.com>

mlennert · January 12, 2015, 5:18pm

On 10/01/15 22:28, Glynn Clements wrote:

Moritz Lennert wrote:

2015-01-08 16:37 GMT+01:00 Paulo van Breugel <p.vanbreugel@gmail.com>:

The installation of the metadata addon on GRASS 7 (trunk) fails, with the
message below. Any ideas?

I have fixed installation in r63999-64001.

Is there some documentation about how to write Makefiles for addons that
use several .py files ? Similar issues have come up with different
addons (cf #2480 and #2534) and it shouldn't be up to you to correct all
these issues. Maybe some clear doc could help ?

The "documentation" for writing Makefiles boils down to: try to find
something similar in the main GRASS tree and use its Makefile as a
reference; if there isn't one, or if that doesn't work, ask on the
developers' list.

Writing documentation for this will result in either wasted effort
(from describing situations which never actually happen) or inadequate
documentation (from failing to describe situations which actually
happen) or (most likely) both.

A couple of points about wx.metadata specifically:

1. Using "parsubdirs" won't work as, because mdlib (presumably) needs
to be built before any of the modules.

2. Files should not be installed using "cp"; use $(INSTALL) for
anything which should be installed with execute permission or
$(INSTALL_DATA) for anything without it.

3. Directories shouldn't be copied with "cp -r". Rather, the
individual destination files should be listed as pre-requisites, so
that "building" the target "builds" the destination files (typically
via pattern rules).

wx.metadata/Makefile should probably include "templates" and "config"
in $(SUBDIRS). These would have their own Makefiles; e.g. for
templates/Makefile, something like:

  include $(MODULE_TOPDIR)/include/Make/Other.make

  DSTDIR = $(ETC)/mdlib/templates

  SRCFILES := $(wildcard *.xml)
  DSTFILES := $(patsubst %.xml,$(DSTDIR)/%.xml,$(SRCFILES))

  default: $(DSTFILES)

  $(DSTDIR)/%.xml: %.xml:
    $(INSTALL_DATA) $< $@

This should allow wx.metadata/Makefile to be a simple "directory"
Makefile (i.e. set SUBDIRS, include Dir.make, "default: subdirs").

So I guess #2480 and #2534 have to be treated individually on a case-by-case basis ?

Moritz

Glynn_Clements1 · January 13, 2015, 12:20am

Moritz Lennert wrote:

So I guess #2480 and #2534 have to be treated individually on a
case-by-case basis ?

What exactly is the issue there?

If building them manually works, that suggests a problem with
g.extension.

--
Glynn Clements <glynn@gclements.plus.com>

mlennert · January 13, 2015, 9:31am

On 13/01/15 01:20, Glynn Clements wrote:

Moritz Lennert wrote:

So I guess #2480 and #2534 have to be treated individually on a
case-by-case basis ?

What exactly is the issue there?

If building them manually works, that suggests a problem with
g.extension.

In both cases, I can run

make MODULE_TOPDIR=../../../../grass70_release/dist.x86_64-unknown-linux-gnu/

from the respective directories in the grass-addons svn tree without any problems.

AFAIU, #2097 also has similar issues: here g.extension installs everything fine, but then the installed script cannot find the additional (library) files.

So, maybe I should reformulate my original question:

When a python addon uses additional .py files,

1) Do we have general rules as to what should be installed where (r.modis installs additional files in a r.modis directory in the local addons directory : .grass7/addons/r.modis - should addons create such directories there or should they go into .grass7/addons/etc ?

2) How should the Makefile be formulated so that all the additional files get installed by g.extension ? This currently does not seem to work for at least i.segment.hierarchical and v.class.ml.

Moritz

Andy_Wickert · February 4, 2015, 1:02am

Hi all,

I don’t see any replies to this, so I will add one. I have currently run into a similar problem with r.flexure and v.flexure, two new extensions that I added that reference a geophysical computer model I wrote (https://github.com/awickert/gFlex). I would also in the future really like to write GRASS GIS interfaces to a large number of additional glacier/climate/geological models, as the GRASS Python interface makes this really easy, and it is nice to have the data displayed in the same coordinate system and database that holds the field data. So I have a vested interest in finding a general solution!

Are there any ideas on how to do this? What I just did (and mentioned to Vaclav) was this:

try:
import gflex # (my model)
except:
print “”
print “MODULE IMPORT ERROR.”
print “In order to run r.flexure or g.flexure, you must download and install”
print “gFlex. The most recent development version is available from”
print “https://github.com/awickert/gFlex.”
print “Installation instructions are available on the page.”
grass.fatal(“Software dependency must be installed.”)

I also thought of some sort of prompt to the user to download the model, and Vaclav had some ideas as well. He’s going to show me how to arrange my imports so the help files compile properly without gflex available, so my email to the list is instead to discuss a framework for future integration of GRASS GIS with outside (but also open-source) software (and of course solving these immediate issues as well).

Best,

Andy

···

On Tue, Jan 13, 2015 at 10:31 AM, Moritz Lennert <mlennert@club.worldonline.be> wrote:

On 13/01/15 01:20, Glynn Clements wrote:

Moritz Lennert wrote:

So I guess #2480 and #2534 have to be treated individually on a
case-by-case basis ?

What exactly is the issue there?

If building them manually works, that suggests a problem with
g.extension.

In both cases, I can run

make MODULE_TOPDIR=…/…/…/…/grass70_release/dist.x86_64-unknown-linux-gnu/

from the respective directories in the grass-addons svn tree without any problems.

AFAIU, #2097 also has similar issues: here g.extension installs everything fine, but then the installed script cannot find the additional (library) files.

So, maybe I should reformulate my original question:

When a python addon uses additional .py files,

Do we have general rules as to what should be installed where (r.modis installs additional files in a r.modis directory in the local addons directory : .grass7/addons/r.modis - should addons create such directories there or should they go into .grass7/addons/etc ?

How should the Makefile be formulated so that all the additional files get installed by g.extension ? This currently does not seem to work for at least i.segment.hierarchical and v.class.ml.

Moritz

grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Andy_Wickert · February 4, 2015, 11:26am

Or, more succinctly:

If there is a flag to install the core packages (see what Vaclav and I did in lines 296–304 at https://github.com/awickert/gFlex/blob/master/r.flexure/r.flexure.py, for example),

(1) Would you think that this is a good idea?

(2) To what directory should the external programs be downloaded and installed? GRASS? The system?

(3) Oftentimes sudo privileges are needed for the install; should the install go into a non-system-owned directory that is part of the path in GRASS, or be available system-wide?

Thanks,

Andy

···

On Wed, Feb 4, 2015 at 2:02 AM, Andy Wickert <andrewwickert@gmail.com> wrote:

Hi all,

I don’t see any replies to this, so I will add one. I have currently run into a similar problem with r.flexure and v.flexure, two new extensions that I added that reference a geophysical computer model I wrote (https://github.com/awickert/gFlex). I would also in the future really like to write GRASS GIS interfaces to a large number of additional glacier/climate/geological models, as the GRASS Python interface makes this really easy, and it is nice to have the data displayed in the same coordinate system and database that holds the field data. So I have a vested interest in finding a general solution!

Are there any ideas on how to do this? What I just did (and mentioned to Vaclav) was this:

try:
import gflex # (my model)
except:
print “”
print “MODULE IMPORT ERROR.”
print “In order to run r.flexure or g.flexure, you must download and install”
print “gFlex. The most recent development version is available from”
print “https://github.com/awickert/gFlex.”
print “Installation instructions are available on the page.”
grass.fatal(“Software dependency must be installed.”)

I also thought of some sort of prompt to the user to download the model, and Vaclav had some ideas as well. He’s going to show me how to arrange my imports so the help files compile properly without gflex available, so my email to the list is instead to discuss a framework for future integration of GRASS GIS with outside (but also open-source) software (and of course solving these immediate issues as well).

Best,

Andy

On Tue, Jan 13, 2015 at 10:31 AM, Moritz Lennert <mlennert@club.worldonline.be> wrote:

On 13/01/15 01:20, Glynn Clements wrote:

Moritz Lennert wrote:

So I guess #2480 and #2534 have to be treated individually on a
case-by-case basis ?

What exactly is the issue there?

If building them manually works, that suggests a problem with
g.extension.

In both cases, I can run

make MODULE_TOPDIR=…/…/…/…/grass70_release/dist.x86_64-unknown-linux-gnu/

from the respective directories in the grass-addons svn tree without any problems.

AFAIU, #2097 also has similar issues: here g.extension installs everything fine, but then the installed script cannot find the additional (library) files.

So, maybe I should reformulate my original question:

When a python addon uses additional .py files,

Do we have general rules as to what should be installed where (r.modis installs additional files in a r.modis directory in the local addons directory : .grass7/addons/r.modis - should addons create such directories there or should they go into .grass7/addons/etc ?

How should the Makefile be formulated so that all the additional files get installed by g.extension ? This currently does not seem to work for at least i.segment.hierarchical and v.class.ml.

Moritz

grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev