#2150: Cannot call Python scripts from Python on MS Windows
-------------------------------------------+--------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: Python | Version: svn-releasebranch64
Keywords: packaging, MAXREPEAT, scripts | Platform: MSWindows 7
Cpu: Unspecified |
-------------------------------------------+--------------------------------
On MS Windows, GRASS does not ensure that its Python executable is used
when calling Python script from Python script. This is a problem when some
other software installed some other version of Python system-wide.
Currently, GRASS is using Python 2.7.4 and when the 3rd party software
installs Python 2.7.3 this one (EXE and DDL) is used but with 2.7.4
packages which causes the famous `MAXREPEAT` error.
It is not clear who to blame for the issue (GRASS, MS Windows way of
calling programs or 3rd party software installing old Python) but it seems
that it is GRASS who has to fix the problem.
The issue may appear when:
* Python script calls Python script
* wxGUI calls Python scripts (in its own way or using same mechanism as
GRASS Python scripts; example is GUI command console)
* C module calling Python scripts (not included in the ticket title, no
issue reported so far, probably not tested; case of g.gui and parser
starting generated module forms/dialogs)
Provided patches:
* putting DLLs to `bin` not `lib` in r57639 + r57646 and for G6 r57694
(applied and works: GRASS GUI starts but calling Python scripts does not
work)
* ensuring the right Python executable manually in r57910; the non-GUI
part reverted in r57911 as a confusing workaround hiding the problem
It is possible to test using `v.db.univar` module which calls `db.univar`
module (both are Python scripts).
The issue is not so big for G6 but it is very serious for G7 where a lot
of modules are Python scripts. But it is even a bigger problem for addons
and user scripts which are expected to be written mainly in Python using a
lot of existing GRASS modules.
The issue is discussed at many places, let's continue the discussion here
only.
This wouldn't fix the real problem, it would just fix the consequences
(still 2 different Python versions would be used but packages would be
compatible). Sooner or later, we will use this Python version anyway.
#2150: Cannot call Python scripts from Python on MS Windows
-------------------------------------------+--------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: Python | Version: svn-releasebranch64
Keywords: packaging, MAXREPEAT, scripts | Platform: MSWindows 7
Cpu: Unspecified |
-------------------------------------------+--------------------------------
Comment(by glynn):
Replying to [ticket:2150 wenzeslaus]:
> It is not clear who to blame for the issue (GRASS, MS Windows way of
calling programs or 3rd party software installing old Python) but it seems
that it is GRASS who has to fix the problem.
It's GRASS' problem for trying to "bundle" Python.
#2150: Cannot call Python scripts from Python on MS Windows
-------------------------------------------+--------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: Python | Version: svn-releasebranch64
Keywords: packaging, MAXREPEAT, scripts | Platform: MSWindows 7
Cpu: Unspecified |
-------------------------------------------+--------------------------------
Comment(by hellik):
please see also ticket http://trac.osgeo.org/grass/ticket/2138 for some
improvements related to addon python script in wingrass6. Hamis did some
work in wingrass6dev, IMHO we are near a working solution for addon
scripts, some issues are open.
Replying to [ticket:2150 wenzeslaus]:
> On MS Windows, GRASS does not ensure that its Python executable is used
when calling Python script from Python script. This is a problem when some
other software installed some other version of Python system-wide.
>
> Currently, GRASS is using Python 2.7.4 and when the 3rd party software
installs Python 2.7.3 this one (EXE and DDL) is used but with 2.7.4
packages which causes the famous `MAXREPEAT` error.
>
> It is not clear who to blame for the issue (GRASS, MS Windows way of
calling programs or 3rd party software installing old Python) but it seems
that it is GRASS who has to fix the problem.
>
> The issue may appear when:
> * Python script calls Python script
please see ticket mentioned above, Python script calls Python script is
working by Hamish's hack
> * wxGUI calls Python scripts (in its own way or using same mechanism as
GRASS Python scripts; example is GUI command console)
> * C module calling Python scripts (not included in the ticket title, no
issue reported so far, probably not tested; case of g.gui and parser
starting generated module forms/dialogs)
>
> The issue was previously discussed in:
> * [http://lists.osgeo.org/pipermail/grass-dev/2013-April/063285.html
new osgeo4w-python and winGrass643RC3-standalone: error in GUI startup]
([http://osgeo-org.1560.x6.nabble.com/new-osgeo4w-python-and-
winGrass643RC3-standalone-error-in-GUI-startup-td5048496.html nabble])
> * [http://lists.osgeo.org/pipermail/grass-dev/2013-July/065197.html
Python handling in winGRASS7] ([http://osgeo-org.1560.x6.nabble.com/Re-
GRASS-GIS-7-tech-preview-release-preparations-tp5066002p5069419.html
nabble])
> * [http://lists.osgeo.org/pipermail/grass-dev/2013-October/065896.html
Handling of Python scripts on MS Windows] ([http://osgeo-
org.1560.x6.nabble.com/Handling-of-Python-scripts-on-MS-Windows-
td5081335.html nabble])
> * #1941 wxGUI fails with Japanese locale --> mixed Python installs on
Windows
> * #2015 Missing file association dialog pops up all the time
> * #2039 Startup Error
>
> Provided patches:
> * putting DLLs to `bin` not `lib` in r57639 + r57646 and for G6 r57694
(applied and works: GRASS GUI starts but calling Python scripts does not
work)
> * ensuring the right Python executable manually in r57910; the non-GUI
part reverted in r57911 as a confusing workaround hiding the problem
>
> It is possible to test using `v.db.univar` module which calls
`db.univar` module (both are Python scripts).
>
> The issue is not so big for G6
IMHO it's also important for G6 as there are a lot of python addon
scripts.
>but it is very serious for G7 where a lot of modules are Python scripts.
But it is even a bigger problem for addons and user scripts which are
expected to be written mainly in Python using a lot of existing GRASS
modules.
>
> The issue is discussed at many places, let's continue the discussion
here only.
#2150: Cannot call Python scripts from Python on MS Windows
-------------------------------------------+--------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: Python | Version: svn-releasebranch64
Keywords: packaging, MAXREPEAT, scripts | Platform: MSWindows 7
Cpu: Unspecified |
-------------------------------------------+--------------------------------
Comment(by wenzeslaus):
This ticket is possible duplicate of 5-year-old blocker #580 which ends
with suggestions to get rid of GRASS session concept and installing Python
to system.
#2150: Cannot call Python scripts from Python on MS Windows
-------------------------------------------+--------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: Python | Version: svn-releasebranch64
Keywords: packaging, MAXREPEAT, scripts | Platform: MSWindows 7
Cpu: Unspecified |
-------------------------------------------+--------------------------------
Comment(by hellik):
Replying to [comment:4 wenzeslaus]:
> This ticket is possible duplicate of 5-year-old blocker #580 which ends
with suggestions to get rid of GRASS session concept and installing Python
to system.
as mentioned in #2138, IMHO we are near a working solution.
#2150: Cannot call Python scripts from Python on MS Windows
-------------------------------------------+--------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: Python | Version: svn-releasebranch64
Keywords: packaging, MAXREPEAT, scripts | Platform: MSWindows 7
Cpu: Unspecified |
-------------------------------------------+--------------------------------
Comment(by wenzeslaus):
If I understand #2138 correctly, this would mean that we would have to
create a BAT file for every Python script and also user would have to do
the same for his or her own GRASS Python scripts. For sure this is better
than nothing but it seems to me that it resembles r57910 in the way that
it forces the right Python manually.
If r58680 (fix for #2138) and r57910 do the same thing I would say that
r57910 is easier to implement and creates less noise. Anyway, can r58680
be ported to trunk in some way? Or is somebody preparing some other patch
for this issue (#2150)?
By the way, when we can expect Python version 2.7.6 (which will probably
hide this issue) to be packaged in GRASS for MS Windows?
#2150: Cannot call Python scripts from Python on MS Windows
-------------------------------------------+--------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: Python | Version: svn-releasebranch64
Keywords: packaging, MAXREPEAT, scripts | Platform: MSWindows 7
Cpu: Unspecified |
-------------------------------------------+--------------------------------
Comment(by martinl):
Replying to [comment:6 wenzeslaus]:
> If I understand #2138 correctly, this would mean that we would have to
create a BAT file for every Python script and also user would have to do
the same for his or her own GRASS Python scripts. For sure this is better
than nothing but it seems to me that it resembles r57910 in the way that
it forces the right Python manually.
My opinion is that requiring BAT file for Python scripts is much more
worse solution that r57910.
> If r58680 (fix for #2138) and r57910 do the same thing I would say that
r57910 is easier to implement and creates less noise. Anyway, can r58680
be ported to trunk in some way? Or is somebody preparing some other patch
for this issue (#2150)?
I guess nobody is really working on this issue. And if so his work can be
reverted again without providing any better solution;-)
> By the way, when we can expect Python version 2.7.6 (which will probably
hide this issue) to be packaged in GRASS for MS Windows?
#2150: Cannot call Python scripts from Python on MS Windows
-------------------------------------------+--------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: Python | Version: svn-releasebranch64
Keywords: packaging, MAXREPEAT, scripts | Platform: MSWindows 7
Cpu: Unspecified |
-------------------------------------------+--------------------------------
Comment(by glynn):
Replying to [comment:7 martinl]:
> My opinion is that requiring BAT file for Python scripts is much more
worse solution that r57910.
Also, note that there's nothing special about .bat (or .cmd). The
mechanism used for "executing" anything other than binary executables is
the same, whether it's .bat or .py. If .bat works when .py doesn't, that
indicates a fault in the Python installation.
#2150: Cannot call Python scripts from Python on MS Windows
-------------------------------------------+--------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: Python | Version: svn-releasebranch64
Keywords: packaging, MAXREPEAT, scripts | Platform: MSWindows 7
Cpu: Unspecified |
-------------------------------------------+--------------------------------
Comment(by martinl):
Replying to [comment:8 glynn]:
> > My opinion is that requiring BAT file for Python scripts is much more
worse solution that r57910.
>
> Also, note that there's nothing special about .bat (or .cmd). The
mechanism used for "executing" anything other than binary executables is
the same, whether it's .bat or .py. If .bat works when .py doesn't, that
indicates a fault in the Python installation.
Update: as a result of never ending discussion I took liberty to add
generation of bat files in r61136. The missing part is how to solve user
python scripts.
#2150: Cannot call Python scripts from Python on MS Windows
-------------------------------------------+--------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: Python | Version: svn-releasebranch64
Keywords: packaging, MAXREPEAT, scripts | Platform: MSWindows 7
Cpu: Unspecified |
-------------------------------------------+--------------------------------
Comment(by martinl):
Replying to [comment:10 martinl]:
> Update: as a result of never ending discussion I took liberty to add
generation of bat files in r61136. The missing part is how to solve user
python scripts.
Update*: calling Python modules from addons (eg. r.basin) is not currently
working.
#2150: Cannot call Python scripts from Python on MS Windows
-------------------------------------------+--------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: Python | Version: svn-releasebranch64
Keywords: packaging, MAXREPEAT, scripts | Platform: MSWindows 7
Cpu: Unspecified |
-------------------------------------------+--------------------------------
Comment(by martinl):
Replying to [comment:12 glynn]:
> In which case, we may want something like attachment:pybatch.diff
(untested), so that SCRIPT_DIR can be overridden by addons.
That's right, the addon scripts are installed by default to
`$GRASS_ADDON_BASE\scripts`. I took liberty to apply your patch in r61715.
So let's wait for the next build.
Hi Martin, bat files could be handled by g.extension. See also ticket to support compiling C modules on unix. (I cannot search it now.) Python modules on MS Windows are in the same situation as C modules on unix. Vaclav
On Aug 22, 2014 12:01 AM, “GRASS GIS” <trac@osgeo.org> wrote:
#2150: Cannot call Python scripts from Python on MS Windows
-------------------------------------------±-------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: Python | Version: svn-releasebranch64
Keywords: packaging, MAXREPEAT, scripts | Platform: MSWindows 7
Cpu: Unspecified |
-------------------------------------------±-------------------------------
Comment(by martinl):
Replying to [comment:8 glynn]:
My opinion is that requiring BAT file for Python scripts is much more
worse solution that r57910.
Also, note that there’s nothing special about .bat (or .cmd). The
mechanism used for “executing” anything other than binary executables is
the same, whether it’s .bat or .py. If .bat works when .py doesn’t, that
indicates a fault in the Python installation.
Update: as a result of never ending discussion I took liberty to add
generation of bat files in r61136. The missing part is how to solve user
python scripts.
2014-08-22 19:40 GMT+02:00 Vaclav Petras <wenzeslaus@gmail.com>:
That the user would use g.extension to install python script in order to use
it.
ah, ok, you are speaking about _user_ scripts not addons. Right, this
is still not solved as I mentioned in the ticket. Probably it could be
solved on wxGUI level ('Launch script' could create a bat file
automatically). The more advanced users (cmd-like) could create the
bat file manually (not so many on Windows I guess).
#2150: Cannot call Python scripts from Python on MS Windows
-------------------------------------------+--------------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: Python | Version: svn-releasebranch64
Keywords: packaging, MAXREPEAT, scripts | Platform: MSWindows 7
Cpu: Unspecified |
-------------------------------------------+--------------------------------
Comment(by martinl):
Replying to [comment:13 martinl]:
> That's right, the addon scripts are installed by default to
`$GRASS_ADDON_BASE\scripts`. I took liberty to apply your patch in r61715.
So let's wait for the next build.
this issue is solved. Please note that these changes haven't been
backported to `relbr7` yet.