[GRASS-dev] build errors in trunk with make -j 4

Hi,

in trunk I get errors detected building wxpython and v.kridge, but if I
cd into their dirs and re-run make they build ok.

are the pre-deps in their Makefiles healthy?

thanks,
Hamish

Hamish wrote:

in trunk I get errors detected building wxpython and v.kridge, but if I
cd into their dirs and re-run make they build ok.

are the pre-deps in their Makefiles healthy?

v.krige is a known problem, and isn't specific to parallel builds. It
imports modules from wxGUI (even if you just use --help etc), but the
"scripts" directory is built before the "gui" directory, so those
modules aren't present in $PYTHONPATH when v.krige is initially
"built". If you try again when the rest of GRASS has been built, it
will work.

Also, v.krige will use the system Python when it may need to use
$GRASS_PYTHON (as it uses wxPython). It should probably be split into
a front-end script which invokes the "real" script via $GRASS_PYTHON
when (and if) grass.parser() returns; then, --html-description would
work without requiring wxPython or any wxGUI modules.

As for wxGUI:

  default: $(DSTFILES) menustrings.py
    $(MAKE) parsubdirs

  menustrings.py: gui_modules/menudata.py xml/menudata.xml
    GISBASE="$(GISBASE)" \
    $(PYTHON) $< > $@

menudata.py reads $GISBASE/etc/wxpython/xml/menudata.xml, which is
included in $(DSTFILES). But with a parallel build, menudata.xml will
be installed concurrently with building menustrings.py.

The dependency line for menstrings.py should probably be:

  menustrings.py: gui_modules/menudata.py $(ETCDIR)/xml/menudata.xml

This should wait for menudata.xml to be installed before executing the
commands.

--
Glynn Clements <glynn@gclements.plus.com>

On Thu, 2010-01-07 at 00:45 +0000, Glynn Clements wrote:

Hamish wrote:

> in trunk I get errors detected building wxpython and v.kridge, but if I
> cd into their dirs and re-run make they build ok.
>
> are the pre-deps in their Makefiles healthy?

v.krige is a known problem, and isn't specific to parallel builds. It
imports modules from wxGUI (even if you just use --help etc), but the
"scripts" directory is built before the "gui" directory, so those
modules aren't present in $PYTHONPATH when v.krige is initially
"built". If you try again when the rest of GRASS has been built, it
will work.

Also, v.krige will use the system Python when it may need to use
$GRASS_PYTHON (as it uses wxPython). It should probably be split into
a front-end script which invokes the "real" script via $GRASS_PYTHON
when (and if) grass.parser() returns; then, --html-description would
work without requiring wxPython or any wxGUI modules.

Hi Glynn, Hamish, all,

the split is done in r40303, and there are no more errors during make.
It's far than perfect, so testing and comments are welcome :slight_smile:
ATM, the "Plot variogram" in GUI is not functional as it is matter of
global variables. I'm looking at this now.

I don't understand what is the problem with #!/usr/bin/env python. How
can the scripts use GRASS_PYTHON instead? How is the link between this
and import wx?
Thanks in advance for any hint and sorry if I'm missing something
important.

best,
Anne

Anne Ghisla wrote:

> v.krige is a known problem, and isn't specific to parallel builds. It
> imports modules from wxGUI (even if you just use --help etc), but the
> "scripts" directory is built before the "gui" directory, so those
> modules aren't present in $PYTHONPATH when v.krige is initially
> "built". If you try again when the rest of GRASS has been built, it
> will work.
>
> Also, v.krige will use the system Python when it may need to use
> $GRASS_PYTHON (as it uses wxPython). It should probably be split into
> a front-end script which invokes the "real" script via $GRASS_PYTHON
> when (and if) grass.parser() returns; then, --html-description would
> work without requiring wxPython or any wxGUI modules.

Hi Glynn, Hamish, all,

the split is done in r40303, and there are no more errors during make.
It's far than perfect, so testing and comments are welcome :slight_smile:
ATM, the "Plot variogram" in GUI is not functional as it is matter of
global variables. I'm looking at this now.

I don't understand what is the problem with #!/usr/bin/env python.

On Unix (including MacOSX), this will run whichever Python is in the
PATH, which may not work (it may not have wxPython, or it may not have
the correct version of wxPython).

The user can get around that by changing $PATH (although that might
interfere with the use of Python scripts which aren't part of GRASS).

A more significant problem is that the #! line is ignored on Windows,
which uses the .py file association instead, and the user can't easily
change that (the file associations are per-user, not per-process like
environment variables).

Consequently, anything which runs Python scripts which require
wxPython (e.g. g.gui, G_parser()) invokes the script via
$GRASS_PYTHON.

How can the scripts use GRASS_PYTHON instead? How is the link
between this and import wx?

scripts/v.krige/v.krige.py shouldn't use wxPython directly or
indirectly (i.e. it shouldn't import wx or any module which uses wx).
Instead, it should execute the GUI script with e.g.:

  python = os.getenv("GRASS_PYTHON")
  gisbase = os.getenv("GISBASE")
  script = os.path.join(gisbase, "etc", "wxpython", "v_krige_wxGUI.py")
  args = [python, script] + sys.argv[1:]
  os.execvp(python, args)

IOW, Python scripts which may be executed directly by the user from
the command line can only assume the presence of a recent version of
Python (2.4, 2.5 or 2.6) with the standard Python library, plus the
grass.script package. They can't assume that any additional packages
(e.g. wxPython) will be available.

Anything which needs wxPython must be invoked via $GRASS_PYTHON. This
allows the user to specify a suitable version of Python in the event
that the "system" Python isn't suitable.

--
Glynn Clements <glynn@gclements.plus.com>

On Wed, Jan 6, 2010 at 11:30 PM, Hamish <hamish_b@yahoo.com> wrote:

Hi,

in trunk I get errors detected building wxpython and v.kridge, but if I
cd into their dirs and re-run make they build ok.

(not using parallel build)
This problem currently breaks the creation of the GRASS 7 binary snapshot
on the Web server:

...
-------------------------
Started compilation: Wed Jan 13 13:35:27 PST 2010
--
Errors in:
/home/neteler/grass7_svn_head_bin_snapshot/trunk/scripts/v.krige
--
...
--
Finished compilation: Wed Jan 13 13:59:38 PST 2010
make: *** [default] Error 1
ERROR: compilation error.

I hope it can be solved since the binary GRASS 7 isn't packaged
at this point.
Glynn suggested to change the compile order of "gui" and "scripts"
as far as I understand.

Markus

Markus Neteler wrote:

> in trunk I get errors detected building wxpython and v.kridge, but if I
> cd into their dirs and re-run make they build ok.

(not using parallel build)
This problem currently breaks the creation of the GRASS 7 binary snapshot
on the Web server:

...
-------------------------
Started compilation: Wed Jan 13 13:35:27 PST 2010
--
Errors in:
/home/neteler/grass7_svn_head_bin_snapshot/trunk/scripts/v.krige
--
...
--
Finished compilation: Wed Jan 13 13:59:38 PST 2010
make: *** [default] Error 1
ERROR: compilation error.

I hope it can be solved since the binary GRASS 7 isn't packaged
at this point.

This issue should already be resolved by r40303.

The other issue (needing to use $GRASS_PYTHON for actual usage) is
still outstanding.

Glynn suggested to change the compile order of "gui" and "scripts"
as far as I understand.

No; I merely commented that having something in scripts try to use
wxGUI modules would fail due to the order in which they were built.

This should no longer be an issue, and even if it was, changing the
order is the wrong solution. We shouldn't be relying upon the order in
which the subdirectories are built any more than is strictly
necessary.

The only ordering which should be required is:

demolocation + tools + include > lib > [everything else] > locale + man > macosx

The ones at the beginning inherently form the foundation on which the
rest of GRASS is built. The ones at the end are based upon GRASS as a
whole (locale needs to extract translatable strings from C files, some
of which need are generated from non-C files, man builds global and
category indices, macosx creates a GRASS package).

The rest should ideally be able to be built in any order. E.g. I
recently added tools/g.echo[1] to convert MSys pathnames to Windows
pathnames during the build process (previously, g.dirseps was being
used, but that doesn't work when you're creating a manpage for a
subdirectory of lib as g.dirseps hasn't been built yet, and it can't
be built before lib/gis).

[1] g.echo just echoes its argument, but MSys automatically converts
MSys paths to Windows paths when running anything which isn't part of
MSys.

--
Glynn Clements <glynn@gclements.plus.com>

On Thu, Jan 14, 2010 at 1:18 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Neteler wrote:

Hamish:
> in trunk I get errors detected building wxpython and v.kridge, but if I
> cd into their dirs and re-run make they build ok.

...

This issue should already be resolved by r40303.

The other issue (needing to use $GRASS_PYTHON for actual usage) is
still outstanding.

In fact, the problem is even a new one:

[neteler@xblade14 v.krige]$ make
...
LC_ALL=C /home/neteler/grass7_svn_head_bin_snapshot/trunk/dist.i686-pc-linux-gnu/scripts/v.krige
--html-description < /dev/null | grep -v '</body>\|</html>' >
v.krige.tmp.html ; fi
  File "/home/neteler/grass7_svn_head_bin_snapshot/trunk/dist.i686-pc-linux-gnu/scripts/v.krige",
line 150
    class Controller():
                     ^
SyntaxError: invalid syntax
make: *** [v.krige.tmp.html] Error 1
rm v.krige.tmp.html

[neteler@xblade14 v.krige]$ rpm -qf /usr/bin/python
python-2.4.3-8.FC4

Python too old? It will be hard to update on that server.

[skip - thanks for the other detailed explanations]

Markus

Markus Neteler wrote:

In fact, the problem is even a new one:

    class Controller():
                     ^
SyntaxError: invalid syntax

[neteler@xblade14 v.krige]$ rpm -qf /usr/bin/python
python-2.4.3-8.FC4

Python too old? It will be hard to update on that server.

That syntax (new style class with no base classes) was introduced in
Python 2.5. I doubt that it's actually necessary; it could probably be
changed to either an old-style class:

     class Controller:

or a new-style class derived from "object":

     class Controller(object):

I think that we should still try to support 2.4, although the only way
that accidental use of 2.5/2.6 features will be found is through
running the code for real. Using new syntax features will cause
--html-description to fail (as happened here), but using new
functions, modules, classes, methods, etc won't.

--
Glynn Clements <glynn@gclements.plus.com>

On Thu, Jan 14, 2010 at 5:10 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Neteler wrote:

In fact, the problem is even a new one:

class Controller\(\):
                 ^

SyntaxError: invalid syntax

[neteler@xblade14 v.krige]$ rpm -qf /usr/bin/python
python-2.4.3-8.FC4

Python too old? It will be hard to update on that server.

That syntax (new style class with no base classes) was introduced in
Python 2.5. I doubt that it's actually necessary; it could probably be
changed to either an old-style class:

class Controller:

or a new-style class derived from "object":

class Controller\(object\):

I think that we should still try to support 2.4, although the only way
that accidental use of 2.5/2.6 features will be found is through
running the code for real. Using new syntax features will cause
--html-description to fail (as happened here), but using new
functions, modules, classes, methods, etc won't.

Done in r40447. Thanks for pointing that out, hope it is solved now.
class Log in the GUI already used old-style class syntax, so I
chose the same for Controller.

regards,
Anne

--
Glynn Clements <glynn@gclements.plus.com>
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

On Thu, Jan 14, 2010 at 1:18 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

The other issue (needing to use $GRASS_PYTHON for actual usage) is
still outstanding.

Yes, I plan to solve it after the removal of some computation code
left in GUI file and some more refactoring.
Is it ok for you all, or $GRASS_PYTHON issue is more urgent?

regards,
Anne

On Thu, Jan 14, 2010 at 9:35 AM, Anne Ghisla <a.ghisla@gmail.com> wrote:

On Thu, Jan 14, 2010 at 5:10 AM, Glynn Clements wrote:

Markus Neteler wrote:

...

it could probably be
changed to either an old-style class:

class Controller:

or a new-style class derived from "object":

class Controller\(object\):

...

Done in r40447. Thanks for pointing that out, hope it is solved now.
class Log in the GUI already used old-style class syntax, so I
chose the same for Controller.

great, now "compiling" again. Will launch the GRASS 7 snapshot now.

Markus