[GRASS-dev] [GRASS GIS] #1758: Disabling python support does not prevent entry into python directories during build

GDAL · October 6, 2012, 2:59am

#1758: Disabling python support does not prevent entry into python directories
during build
-------------------------+--------------------------------------------------
Reporter: brook | Owner: grass-dev@…
Type: defect | Status: new
Priority: critical | Milestone: 6.4.3
Component: Default | Version: 6.4.2
Keywords: | Platform: All
Cpu: Unspecified |
-------------------------+--------------------------------------------------
The configure option --without-python is presumably supposed to disable
all python support. However, the Makefiles are set to unconditionally
enter the gui/wxpython and lib/python subdirectories. As a result, this
option results in a failed build.

The solution is to conditionally include those directories in the list of
subdirectories for make. Two patches against the 6.4.2 release that fix
this are attached. This has been compile-checked on a NetBSD system both
with and without the --without-python option.

Presumably this should be pulled up to the more recent versions as well.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1758>
GRASS GIS <http://grass.osgeo.org>

GDAL · October 6, 2012, 5:03pm

#1758: Disabling python support does not prevent entry into python directories
during build
-----------------------+----------------------------------------------------
  Reporter: brook | Owner: grass-dev@…
      Type: defect | Status: closed
  Priority: critical | Milestone: 6.4.3
Component: Python | Version: 6.4.2
Resolution: fixed | Keywords:
  Platform: All | Cpu: Unspecified
-----------------------+----------------------------------------------------
Changes (by neteler):

  * status: new => closed
  * resolution: => fixed
  * component: Default => Python

Comment:

Thanks, applied to G6.4 and G6.5 in r53326 and r53327.

For GRASS 7 which highly depends on Python this changes does not make
sense.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1758#comment:1>
GRASS GIS <http://grass.osgeo.org>

GDAL · October 6, 2012, 10:41pm

#1758: Disabling python support does not prevent entry into python directories
during build
-----------------------+----------------------------------------------------
  Reporter: brook | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: critical | Milestone: 6.4.3
Component: Python | Version: 6.4.2
Resolution: | Keywords:
  Platform: All | Cpu: Unspecified
-----------------------+----------------------------------------------------
Changes (by glynn):

* status: closed => reopened
* resolution: fixed =>

Comment:

Replying to [ticket:1758 brook]:
> The configure option --without-python is presumably supposed to disable
all python support.

Incorrect. It's supposed to disable building C/C++ components which link
against the Python library (are there any of these left in 6.x?).

> However, the Makefiles are set to unconditionally enter the gui/wxpython
and lib/python subdirectories.

This is correct.

> As a result, this option results in a failed build.

Error message? Other symptoms?

> The solution is to conditionally include those directories in the list
of subdirectories for make. Two patches against the 6.4.2 release that
fix this are attached. This has been compile-checked on a NetBSD system
both with and without the --without-python option.
>
> Presumably this should be pulled up to the more recent versions as well.

It shouldn't go into any version; r53326 and r53327 are in error. If the
build fails due to a missing Python interpreter, the Makefiles should be
fixed so that such errors are reported and ignored, rather than aborting
the build.

A couple of things which stand out are the menustrings.py rule in
gui/wxpython/Makefile, as well as the rule which runs create__init__.py.
The former is decidedly non-critical (it just means that "make -C locale
pot" won't get the strings from the XML files for the menus). The latter
should ideally be fixed to not require Python; the wxGUI should should
only require Python on the target system, not on the build system.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1758#comment:2>
GRASS GIS <http://grass.osgeo.org>

GDAL · October 7, 2012, 4:04pm

#1758: Disabling python support does not prevent entry into python directories
during build
-----------------------+----------------------------------------------------
  Reporter: brook | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: critical | Milestone: 6.4.3
Component: Python | Version: 6.4.2
Resolution: | Keywords:
  Platform: All | Cpu: Unspecified
-----------------------+----------------------------------------------------

Comment(by brook):

I take it from the above comment that python is a required prerequisite
for grass, even for 6.4.2. If that is the case, it should be listed among
the prerequisites; it is not. If that is not the case, it should be
possible to build grass correctly without python, something that I
expected the --without-python to accomplish. If that is not the correct
way to build grass without python, then perhaps that should instead be
documented.

It is unclear to me what the correct state is:

- Is python a required prerequisite for grass 6.4.2? (I hope it is
documented as such for GRASS 7 as it sounds like it is a requirement.)

- What is the correct way, if any, to build grass 6.4.2 without python?

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1758#comment:3>
GRASS GIS <http://grass.osgeo.org>

GDAL · October 8, 2012, 4:19pm

#1758: Disabling python support does not prevent entry into python directories
during build
-----------------------+----------------------------------------------------
  Reporter: brook | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: critical | Milestone: 6.4.3
Component: Python | Version: 6.4.2
Resolution: | Keywords:
  Platform: All | Cpu: Unspecified
-----------------------+----------------------------------------------------

Comment(by glynn):

Replying to [comment:3 brook]:
> I take it from the above comment that python is a required prerequisite
for grass, even for 6.4.2.
No, it's not required. At least, it shouldn't be.

> If that is not the case, it should be possible to build grass correctly
without python,
Correct.

> something that I expected the --without-python to accomplish.
It should be possible with or without that switch. In fact, the switch
shouldn't even exist; it serves no purpose, as there are no longer any
modules which need to be compiled against the Python library and headers.

> - Is python a required prerequisite for grass 6.4.2?
No.

> (I hope it is documented as such for GRASS 7 as it sounds like it is a
requirement.)
It is a requirement for 7.0, and it's documented.

> - What is the correct way, if any, to build grass 6.4.2 without python?

The presence or absence of Python should not affect the build. If it does,
that's a bug in the build system. Modules which require Python (e.g.
wxGUI) should be installed regardless of the presence or absence of Python
on the build system. Obviously, such modules won't work if you don't have
Python on the target system.

However: there are a couple of inevitable shortcomings if the build system
lacks Python:

* Python scripts (I think this is just v.krige) won't have manual pages.

* The source message catalogue won't contain the strings used in the
wxGUI menus.

* Possibly other minor details. The issue is similar to cross-compiling:
portions of the build system which rely upon the ability to execute the
programs being built won't work if those programs are written in Python
and you don't have Python on the build system. The main thing is that such
issues should not prevent other components from being built.

One issue I have noticed is `gui/wxpython/create__init__.py` which creates
the `__init__.py` files for parts of the wxGUI. In 6.x, this should be a
shell script, as Python isn't a requirement for the build system. Failing
that, it should be included as part of the installation so that the build
process can be completed on the target system.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1758#comment:4>
GRASS GIS <http://grass.osgeo.org>

GDAL · February 18, 2013, 7:52am

#1758: Disabling python support does not prevent entry into python directories
during build
-----------------------+----------------------------------------------------
  Reporter: brook | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: critical | Milestone: 6.4.3
Component: Python | Version: 6.4.2
Resolution: | Keywords:
  Platform: All | Cpu: Unspecified
-----------------------+----------------------------------------------------

Comment(by hamish):

Hi,

fwiw I have an old system which doesn't have a new enough wxWidgets (v2.6)
to run the grass wx gui, so since development on it began I've built there
with `--with-python=/usr/bin/python2.4-config` and `--without-wxwidgets`
without any build errors.

It has been some months since I've rebuilt grass there, but as of about
6.4.2 it still worked fine after using g.gui to set the default to tcl/tk.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1758#comment:5>
GRASS GIS <http://grass.osgeo.org>

GDAL · February 18, 2013, 9:09am

#1758: Disabling python support does not prevent entry into python directories
during build
------------------------+---------------------------------------------------
  Reporter: brook | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: major | Milestone: 6.4.3
Component: Compiling | Version: 6.4.2
Resolution: | Keywords: python, wxgui
  Platform: All | Cpu: Unspecified
------------------------+---------------------------------------------------
Changes (by hamish):

  * keywords: => python, wxgui
  * priority: critical => major
  * component: Python => Compiling

Comment:

[trac is logging me out every 5 minutes.. arrgh]

I just tried rebuilding on the old machine --with-python and --without-
wxwidgets in comment:5, it was ok.

Building with 'make -j 5' I did get an error logged in the gui/wxpython/
dir, but cd'ing to it and running make again was successful, so I suspect
that some Makefile order-of-build dependency issue still exists, but
overall it's happy.

Is the goal to build on a system without any python on it, to build on a
system without wxWidgets on it, or to just not build the wxGUI?

If it's just to avoid building with the wxGUI it's not really a problem,
since most of it is not really built, just copied in place and then left
unused.

> As a result, this option results in a failed build.

what was the error?

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1758#comment:6>
GRASS GIS <http://grass.osgeo.org>

GDAL · February 18, 2013, 9:24am

#1758: Disabling python support does not prevent entry into python directories
during build
------------------------+---------------------------------------------------
  Reporter: brook | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: major | Milestone: 6.4.3
Component: Compiling | Version: 6.4.2
Resolution: | Keywords: python, wxgui
  Platform: All | Cpu: Unspecified
------------------------+---------------------------------------------------

Comment(by hamish):

I just tried 6.4.3svn again --without-python and --without-wxwidgets.

The experience was the same. It still tried to `make` the wxwidgets gui,
and failed with this error:

{{{
...
python -m py_compile /usr/local/src/grass/svn/releasebranch_6_4/dist.i686
-pc-linux-gnu/etc/wxpython/wxplot/profile.py
python -m py_compile /usr/local/src/grass/svn/releasebranch_6_4/dist.i686
-pc-linux-gnu/etc/wxpython/gis_set.py
python -m py_compile /usr/local/src/grass/svn/releasebranch_6_4/dist.i686
-pc-linux-gnu/etc/wxpython/wxgui.py
GISRC=/usr/local/src/grass/svn/releasebranch_6_4/dist.i686-pc-linux-
gnu/demolocation/.grassrc64
GISBASE=/usr/local/src/grass/svn/releasebranch_6_4/dist.i68
6-pc-linux-gnu PATH="/usr/local/src/grass/svn/releasebranch_6_4/dist.i686
-pc-linux-gnu/bin:$PATH"
PYTHONPATH="/usr/local/src/grass/svn/releasebranch_6_4/di
st.i686-pc-linux-gnu/etc/python:$PYTHONPATH"
LD_LIBRARY_PATH="/usr/local/src/grass/svn/releasebranch_6_4/dist.i686-pc-
linux-gnu/bin:/usr/local/src/grass/sv
n/releasebranch_6_4/dist.i686-pc-linux-gnu/lib::" LC_ALL=C python
core/menudata.py > menustrings.py
Traceback (most recent call last):
File "core/menudata.py", line 39, in ?
import wx
ImportError: No module named wx
make[2]: *** [menustrings.py] Error 1
make[2]: Leaving directory
`/usr/local/src/grass/svn/releasebranch_6_4/gui/wxpython'
make[1]: Leaving directory
`/usr/local/src/grass/svn/releasebranch_6_4/gui'
make[1]: Entering directory
`/usr/local/src/grass/svn/releasebranch_6_4/imagery'
...
}}}

but after cd'ing into the dir and trying a second time it built ok.

note python was present on the system (see lack of errors about that in
the above log snippet), I'm not sure what would happen if it wasn't
installed, e.g. with the attempted byte compiling of lib/python/ and
population of dist.`arch`/etc/python/grass/lib/.
Probably you'd get a non-fatal error about it listed in error.log.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1758#comment:7>
GRASS GIS <http://grass.osgeo.org>

GDAL · February 18, 2013, 9:31am

#1758: Disabling python support does not prevent entry into python directories
during build
------------------------+---------------------------------------------------
  Reporter: brook | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: major | Milestone: 6.4.3
Component: Compiling | Version: 6.4.2
Resolution: | Keywords: python, wxgui
  Platform: All | Cpu: Unspecified
------------------------+---------------------------------------------------

Comment(by hamish):

note {{{ ./configure --help | less }}} instructions:

{{{
   --with-python[=path/python-config]
                           enable support for wxGUI Python extensions
                           (python-config with path, e.g.
                           '--with-python=/usr/bin/python2.5-config',
                           default: no)
   --with-wxwidgets[=path/wx-config]
                           enable wxWidgets support (wx-config with path,
                           e.g. '--with-wxwidgets=/usr/local/bin/wx-
config',
                           default: no)
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1758#comment:8>
GRASS GIS <http://grass.osgeo.org>

GDAL · February 19, 2013, 12:25am

#1758: Disabling python support does not prevent entry into python directories
during build
------------------------+---------------------------------------------------
  Reporter: brook | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: major | Milestone: 6.4.3
Component: Compiling | Version: 6.4.2
Resolution: | Keywords: python, wxgui
  Platform: All | Cpu: Unspecified
------------------------+---------------------------------------------------

Comment(by glynn):

Replying to [comment:6 hamish]:

> Is the goal to build on a system without any python on it, to build on a
system without wxWidgets on it, or to just not build the wxGUI?

--with-python and --with-wxwidgets tell the configure script to check for
Python/wxWidgets compilation/linking switches. These are only necessary if
you're building binary modules (programs or shared libraries) which link
against the Python or wx libraries (do any of these still exist? 7.0 has
visualisation/wximgview, but that's largely obsoleted by
scripts/wxpyimgview).

Note that neither switch is enabled by default, so there's no point in
using --without-python or --without-wxwidgets.

If you enable the checks and they fail, configure will fail.

> If it's just to avoid building with the wxGUI it's not really a problem,
since most of it is not really built, just copied in place and then left
unused.

Any Python modules and the wxGUI should be "built" regardless of whether
Python and/or wxWidgets are installed on the build system.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1758#comment:9>
GRASS GIS <http://grass.osgeo.org>

GDAL · March 28, 2014, 8:29pm

#1758: Disabling python support does not prevent entry into python directories
during build
------------------------+---------------------------------------------------
  Reporter: brook | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: minor | Milestone: 7.0.0
Component: Compiling | Version: 6.4.2
Resolution: | Keywords: python, wxgui, g.gui.* modules
  Platform: All | Cpu: Unspecified
------------------------+---------------------------------------------------
Changes (by wenzeslaus):

  * keywords: python, wxgui => python, wxgui, g.gui.* modules
  * priority: major => minor
  * milestone: 6.4.3 => 7.0.0

Comment:

Replying to [comment:9 glynn]:
> Replying to [comment:6 hamish]:
>
> > Is the goal to build on a system without any python on it, to build on
a system without wxWidgets on it, or to just not build the wxGUI?
>
> --with-python and --with-wxwidgets tell the configure script to check
for Python/wxWidgets compilation/linking switches. These are only
necessary if you're building binary modules (programs or shared libraries)
which link against the Python or wx libraries (do any of these still
exist? 7.0 has visualisation/wximgview, but that's largely obsoleted by
scripts/wxpyimgview).
>

So, would it make sense to introduce flags for building without GUI? In
case of GRASS, this seems reasonable. In this case, the question also is
how the documentation should look like if the would be no GUI (e.g.
considering #645 and #1203).

The `--without-python` option always looked like that Python (interpreter)
will not be used. But Python (interpreter) is build requirement, right? It
seems that this is not reflected in source:grass/trunk/REQUIREMENTS.html.

> Note that neither switch is enabled by default, so there's no point in
using --without-python or --without-wxwidgets.
>

I like to do things explicit in cases when it is not clear to me what is
default. Is there any problem with using `--without-...`?

> If you enable the checks and they fail, configure will fail.
>
> > If it's just to avoid building with the wxGUI it's not really a
problem, since most of it is not really built, just copied in place and
then left unused.
>
> Any Python modules and the wxGUI should be "built" regardless of whether
Python and/or wxWidgets are installed on the build system.

This is no longer true because the `g.gui.*` modules needs to generate
their interface and they begin with import of wxPython related things. Is
this a problem? I.e. is building of wxGUI without wxPython present a
requirement?

If yes, we can hopefully rewrite `g.gui.*` modules, `menustrings.py` and
toolboxes generation to be able to generate interface, files for
translations and toolbox files without any importing, but I'm afraid how
the code will look like.

What is the status of this thicket for trunk? E.g., there is no `--with-
python`.

{{{
$ ./configure --help | grep -i python
$ ./configure --help | grep -i wx
   --with-wxwidgets[=path/wx-config]
                           enable wxWidgets support (wx-config with path,
                           e.g. '--with-wxwidgets=/usr/local/bin/wx-
config',
}}}

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1758#comment:10>
GRASS GIS <http://grass.osgeo.org>

GDAL · March 29, 2014, 10:51pm

#1758: Disabling python support does not prevent entry into python directories
during build
------------------------+---------------------------------------------------
  Reporter: brook | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: minor | Milestone: 6.5.0
Component: Compiling | Version: 6.4.2
Resolution: | Keywords: python, wxgui
  Platform: All | Cpu: Unspecified
------------------------+---------------------------------------------------
Changes (by hamish):

* keywords: python, wxgui, g.gui.* modules => python, wxgui
* milestone: 7.0.0 => 6.5.0

Comment:

Replying to [comment:10 wenzeslaus]:
> So, would it make sense to introduce flags for building without GUI?

not really, by "building" wxGUI in GRASS 6 it mostly just means copying
the files over. Splitting the files out into grass-core and grass-gui
packages for e.g. linux distribution happens at the packaging level and is
no burden, and the extra space used by a couple hundred python files in
negligible compared with a couple screenshots in the help pages.

> I like to do things explicit in cases when it is not clear to me what is
default.

fwiw the default action is described in the `./configure --help` usage
text.

> What is the status of this thicket for trunk?

irrelevant, since there has been no request for --without-python in grass
7 AFAIK, so it remains required.

This ticket's concerning GRASS 6, where python is ostensibly not
mandatory, but some confusion is caused by the copying of files anyway,
when what the flag really means is that python will not be used at build-
time.

regards,
Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1758#comment:11>
GRASS GIS <http://grass.osgeo.org>

GDAL · March 31, 2014, 4:31pm

#1758: Disabling python support does not prevent entry into python directories
during build
------------------------+---------------------------------------------------
  Reporter: brook | Owner: grass-dev@…
      Type: defect | Status: reopened
  Priority: minor | Milestone: 6.5.0
Component: Compiling | Version: 6.4.2
Resolution: | Keywords: python, wxgui
  Platform: All | Cpu: Unspecified
------------------------+---------------------------------------------------

Comment(by glynn):

Replying to [comment:10 wenzeslaus]:

> So, would it make sense to introduce flags for building without GUI?

Not really. In general, we try to build everything except where failure
can be predicted based upon configure options.

> The `--without-python` option always looked like that Python
(interpreter) will not be used. But Python (interpreter) is build
requirement, right? It seems that this is not reflected in
source:grass/trunk/REQUIREMENTS.html.

We only have configure checks for things which require them. That
typically means compiled modules which need to know which compiler and/or
linker switches to use.

We used to have --with-python because the GUI included some compiled
modules which linked against the Python library. For that to work, we
needed to know the correct compiler and linker switches for compiling code
which uses the Python library. Now that those modules have been removed,
the checks are no longer necessary.

We still have --with-wxwidgets for the benefit of wximgview, which is a
compiled module which links against the wxWidgets libraries. Although,
this modules may be redundant in light of wxpyimgview.py.

> > Note that neither switch is enabled by default, so there's no point in
using --without-python or --without-wxwidgets.
>
> I like to do things explicit in cases when it is not clear to me what is
default. Is there any problem with using `--without-...`?

No.

> > Any Python modules and the wxGUI should be "built" regardless of
whether Python and/or wxWidgets are installed on the build system.
>
> This is no longer true because the `g.gui.*` modules needs to generate
their interface and they begin with import of wxPython related things. Is
this a problem? I.e. is building of wxGUI without wxPython present a
requirement?

Ideally, we should provide an option to "finalise" the build on the target
system, for the situation where the build system lacks Python (or some
other run-time dependency) but the targets system provides it.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1758#comment:12>
GRASS GIS <http://grass.osgeo.org>