[GRASS-dev] [GRASS GIS] #1941: wxGUI error

GRASS GRASS developer
21

#1941: wxGUI fails with Japanese locale
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by mlennert):

I think that the Python launcher might be the best solution for this. See
the experiments with different combinations of Python versions at [1].

[1] [http://lists.osgeo.org/pipermail/grass-dev/2013-July/065197.html\]

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1941#comment:20&gt;
GRASS GIS <http://grass.osgeo.org>

22

#1941: wxGUI fails with Japanese locale
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by martinl):

Replying to [comment:19 martinl]:
> > [http://msdn.microsoft.com/en-
us/library/windows/desktop/ms682586%28v=vs.85%29.aspx DLL Search Order].
>
> Right, when I launch `python.exe` which comes with standalone installer
it points to `C:Windows\SysWOW64\python27.dll`. When I launch `python.exe`
from `C:\OSGeo4W\bin` it points to the right dll file, ie.
`C:\OSGeo4W\bin\python27.dll`.

Ah, OK, it's probably caused by the fact that the standalone installer
puts `python27.dll` to the different directory (ie. `extralib`) compared
to `python.exe` (`extrabin`). Directory `extralib` is in the path
source:grass/trunk/mswindows/env.bat#L17, but it's probably not enough.
The problem could be solved by moving `python27.dll` to `extrabin`. Then
we could probably move all DLL files to `extrabin` and get rid of this
directory. It's comment practice to have DLL files together with EXE files
in the same directory. I don't know why the standalone installer separates
them to the two different directories, ie. `extrabin` and `extralib`.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1941#comment:21&gt;
GRASS GIS <http://grass.osgeo.org>

23

#1941: wxGUI fails with Japanese locale
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by mlennert):

Replying to [comment:21 martinl]:
> Replying to [comment:19 martinl]:
> > > [http://msdn.microsoft.com/en-
us/library/windows/desktop/ms682586%28v=vs.85%29.aspx DLL Search Order].
> >
> > Right, when I launch `python.exe` which comes with standalone
installer it points to `C:Windows\SysWOW64\python27.dll`. When I launch
`python.exe` from `C:\OSGeo4W\bin` it points to the right dll file, ie.
`C:\OSGeo4W\bin\python27.dll`.
>
> Ah, OK, it's probably caused by the fact that the standalone installer
puts `python27.dll` to the different directory (ie. `extralib`) compared
to `python.exe` (`extrabin`). Directory `extralib` is in the path
source:grass/trunk/mswindows/env.bat#L17, but it's probably not enough.
The problem could be solved by moving `python27.dll` to `extrabin`. Then
we could probably move all DLL files to `extrabin` and get rid of this
directory. It's comment practice to have DLL files together with EXE files
in the same directory. I don't know why the standalone installer separates
them to the two different directories, ie. `extrabin` and `extralib`.

Try it, but ISTR that there is also an issue with the Windows registry,
i.e. that the dlls called depend on registry entries (I think that this
came up as a problem at least in the case of python scripts called from
python scripts).

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1941#comment:22&gt;
GRASS GIS <http://grass.osgeo.org>

24

#1941: wxGUI fails with Japanese locale
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by hellik):

Replying to [comment:21 martinl]:
>
> Ah, OK, it's probably caused by the fact that the standalone installer
puts `python27.dll` to the different directory (ie. `extralib`) compared
to `python.exe` (`extrabin`). Directory `extralib` is in the path
source:grass/trunk/mswindows/env.bat#L17, but it's probably not enough.
The problem could be solved by moving `python27.dll` to `extrabin`. Then
we could probably move all DLL files to `extrabin` and get rid of this
directory. It's comment practice to have DLL files together with EXE files
in the same directory.

that could be the clue. same directory could work.

>I don't know why the standalone installer separates them to the two
different directories, ie. `extrabin` and `extralib`.

also no idea or memory here on my side. since I'm involved in the
winGRASS, the separation in `extrabin` and `extralib` was always there.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1941#comment:23&gt;
GRASS GIS <http://grass.osgeo.org>

25

#1941: wxGUI fails with Japanese locale
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by glynn):

Replying to [comment:19 martinl]:
> > "system" Python is sufficient and not try to use the bundled Python
(i.e. don't set PYTHONHOME, set GRASS_PYTHON to point to the system
version, etc).
>
> But then we will have problem with other python packages (GRASS
dependecies) which are installed through osgeo4w environment as `python-
matplotlib` or `python-numpy`, right? Version of "system" python will be
most probably different from the python version available in osgeo4w
environment.

Add-on packages don't normally care about the minor version, so a package
built for 2.7 should work with 2.7.<anything>.

This doesn't apply to sre_*.py; modules which are part of the base Python
distribution don't expect to be used with anything other than the exact
Python version of which they are part.

> Right, when I launch `python.exe` which comes with standalone installer
it points to `C:Windows\SysWOW64\python27.dll`. When I launch `python.exe`
from `C:\OSGeo4W\bin` it points to the right dll file, ie.
`C:\OSGeo4W\bin\python27.dll`.

That will be because the OSGeo4W version has its python27.dll in the same
directory as python.exe. The directory containing the EXE comes first,
before the system directories; %PATH% comes last.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1941#comment:24&gt;
GRASS GIS <http://grass.osgeo.org>

26

#1941: wxGUI fails with Japanese locale --> mixed Python installs on Windows
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------
Description changed by hamish:

Old description:

I get following error with the wxGUI while running
GRASS7-svn when selecting modules form the menu.
on Windows-7 in Japanese locale.
I guess it is related to utf-8 encoding.

------------------------------------------------
Traceback (most recent call last):
  File "C:\Program Files\GRASS GIS
7.0.svn\etc\gui\wxpython\lmgr\frame.py", line 737, in
OnMenuCmd

cmd = self.GetMenuCmd(event)
  File "C:\Program Files\GRASS GIS
7.0.svn\etc\gui\wxpython\lmgr\frame.py", line 722, in
GetMenuCmd

input = GUI().GetCommandInputMapParamKey(cmdlist[0])
  File "C:\Program Files\GRASS GIS
7.0.svn\etc\gui\wxpython\gui_core\forms.py", line 2301, in
GetCommandInputMapParamKey

gtask.get_interface_description(cmd).decode(enc).encode('utf
-8')))
UnicodeDecodeError
:
'cp932' codec can't decode bytes in position 7074-7075:
illegal multibyte sequence

New description:

I get following error with the wxGUI while running
GRASS7-svn when selecting modules form the menu.
on Windows-7 in Japanese locale.
I guess it is related to utf-8 encoding.

------------------------------------------------
{{{
Traceback (most recent call last):
   File "C:\Program Files\GRASS GIS
7.0.svn\etc\gui\wxpython\lmgr\frame.py", line 737, in
OnMenuCmd

cmd = self.GetMenuCmd(event)
   File "C:\Program Files\GRASS GIS
7.0.svn\etc\gui\wxpython\lmgr\frame.py", line 722, in
GetMenuCmd

input = GUI().GetCommandInputMapParamKey(cmdlist[0])
   File "C:\Program Files\GRASS GIS
7.0.svn\etc\gui\wxpython\gui_core\forms.py", line 2301, in
GetCommandInputMapParamKey

gtask.get_interface_description(cmd).decode(enc).encode('utf
-8')))
UnicodeDecodeError
:
'cp932' codec can't decode bytes in position 7074-7075:
illegal multibyte sequence
}}}

--

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1941#comment:25&gt;
GRASS GIS <http://grass.osgeo.org>

27

#1941: wxGUI fails with Japanese locale --> mixed Python installs on Windows
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by hamish):

Replying to [comment:24 glynn]:
> Add-on packages don't normally care about the minor version, so
> a package built for 2.7 should work with 2.7.<anything>.

As seen in the pdf screenshot, the trouble comes with "import re", which I
think(??) is a built-in.

We recently saw the same trouble here with 'import re' -> no MAXREPEAT and
6.4.3 on a Windows 7 laptop, where there was a slightly older python
version of 2.7(.2?) installed to C:\Python, which I guessed came with a
separate OSGeo4W install(?). For us it was a new laptop not doing anything
else so we could uninstall the non-grass Python from the control panel and
then all was good.

Are we prepending to the PYTHONPATH instead of adding the grass paths to
the end of it? (hopefully the GRASS versions get picked up first then)
AFAIK env.bat & friends in the stand-alone installer now explicitly set
GRASS_PYTHON with with the full path to grass's python.exe.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1941#comment:26&gt;
GRASS GIS <http://grass.osgeo.org>

28

#1941: wxGUI fails with Japanese locale --> mixed Python installs on Windows
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by glynn):

Replying to [comment:26 hamish]:
> As seen in the pdf screenshot, the trouble comes with "import re", which
I think(??) is a built-in.

re.py and sre_*.py are part of the standard library. The error comes
because MAXREPEAT is expected to be in the binary module "_sre", which is
built into python27.dll. This is getting loaded from the Windows/System or
SysWOW64 directory, hence the problem.

> Are we prepending to the PYTHONPATH instead of adding the grass paths to
the end of it? (hopefully the GRASS versions get picked up first then)
AFAIK env.bat & friends in the stand-alone installer now explicitly set
GRASS_PYTHON with with the full path to grass's python.exe.

I don't know whether imports from the standard library use PYTHONPATH or
PYTHONHOME (and neither of those will affect binary modules built into
python27.dll).

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1941#comment:27&gt;
GRASS GIS <http://grass.osgeo.org>

29

#1941: wxGUI fails with Japanese locale --> mixed Python installs on Windows
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by martinl):

Replying to [comment:22 mlennert]:

> > Ah, OK, it's probably caused by the fact that the standalone installer
puts `python27.dll` to the different directory (ie. `extralib`) compared
to `python.exe` (`extrabin`). Directory `extralib` is in the path
source:grass/trunk/mswindows/env.bat#L17, but it's probably not enough.
The problem could be solved by moving `python27.dll` to `extrabin`. Then
we could probably move all DLL files to `extrabin` and get rid of this
directory. It's comment practice to have DLL files together with EXE files
in the same directory. I don't know why the standalone installer separates
them to the two different directories, ie. `extrabin` and `extralib`.
>
> Try it,

it helped, GRASS now starts, python scripts called from command line also
starts. My suggestion is to apply patches r57639 and r57646 also in GRASS
6.x, ie. to remove `extralib` directory from standalone installer. Any
objections?

> but ISTR that there is also an issue with the Windows registry, i.e.
that the dlls called depend on registry entries (I think that this came up
as a problem at least in the case of python scripts called from python
scripts).

Right, it's open issue.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1941#comment:28&gt;
GRASS GIS <http://grass.osgeo.org>

30

#1941: wxGUI fails with Japanese locale --> mixed Python installs on Windows
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by hellik):

Replying to [comment:28 martinl]:
> >
> > Try it,
>
> it helped, GRASS now starts, python scripts called from command line
also starts. My suggestion is to apply patches r57639 and r57646 also in
GRASS 6.x, ie. to remove `extralib` directory from standalone installer.
Any objections?

+1 for backport

>
> > but ISTR that there is also an issue with the Windows registry, i.e.
that the dlls called depend on registry entries (I think that this came up
as a problem at least in the case of python scripts called from python
scripts).
>
> Right, it's open issue.

I can't find any registry entry for calling python dlls here on my
win7-box. isn't it an issue with file extension-association?

Helmut

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1941#comment:29&gt;
GRASS GIS <http://grass.osgeo.org>

31

#1941: wxGUI fails with Japanese locale --> mixed Python installs on Windows
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by wenzeslaus):

Replying to [comment:29 hellik]:
> Replying to [comment:28 martinl]:
> > > but ISTR that there is also an issue with the Windows registry, i.e.
that the dlls called depend on registry entries (I think that this came up
as a problem at least in the case of python scripts called from python
scripts).
> >
> > Right, it's open issue.
>
> I can't find any registry entry for calling python dlls here on my
win7-box. isn't it an issue with file extension-association?
>
For the file association, we have even a separate ticket #2015. But the
comments deal with other problems (actually, I don't know if they are
related or not) and my original problem was never confirmed (although we
had it one more than one computer).

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1941#comment:30&gt;
GRASS GIS <http://grass.osgeo.org>

32

#1941: wxGUI fails with Japanese locale --> mixed Python installs on Windows
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by martinl):

Replying to [comment:29 hellik]:
> > it helped, GRASS now starts, python scripts called from command line
also starts. My suggestion is to apply patches r57639 and r57646 also in
GRASS 6.x, ie. to remove `extralib` directory from standalone installer.
Any objections?
>
> +1 for backport

patch applied in all active branches, now GRASS should start without
'MAX_REPEAT' problem.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1941#comment:31&gt;
GRASS GIS <http://grass.osgeo.org>

33

#1941: wxGUI fails with Japanese locale --> mixed Python installs on Windows
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by annakrat):

Replying to [comment:31 martinl]:
> Replying to [comment:29 hellik]:
> > > it helped, GRASS now starts, python scripts called from command line
also starts. My suggestion is to apply patches r57639 and r57646 also in
GRASS 6.x, ie. to remove `extralib` directory from standalone installer.
Any objections?
> >
> > +1 for backport
>
> patch applied in all active branches, now GRASS should start without
'MAX_REPEAT' problem.

Thanks a lot, there is a ticket already discussing this problem (#2039) so
let's move the discussion there and leave this for Japanese locale.

Venka, can you please test it now again when the unrelated MAXREPEAT
problem should be fixed?

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1941#comment:32&gt;
GRASS GIS <http://grass.osgeo.org>

34

#1941: wxGUI fails with Japanese locale --> mixed Python installs on Windows
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: major | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------
Changes (by Venkat):

  * priority: blocker => major

Comment:

MAXREPEAT issue is fixed and GRASS-7 starts after complaining about
missing HDF5 and MrSid DLL[1]. Could be better if gdal is compiled
without HDF5 and MrSid support.

[1]http://trac.osgeo.org/grass/attachment/ticket/1941/missing_dll.png

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1941#comment:33&gt;
GRASS GIS <http://grass.osgeo.org>

35

#1941: wxGUI fails with Japanese locale --> mixed Python installs on Windows
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: major | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by wenzeslaus):

Replying to [comment:33 Venkat]:
> MAXREPEAT issue is fixed and GRASS-7 starts after complaining about
missing HDF5 and !MrSid DLL. Could be better if gdal is compiled
> without HDF5 and !MrSid support.
>
> attachment:missing_dll.png

Is this still a problem? If yes I suggest you to create a new ticket for
this with the all details and close this ticket where all errors were
solved or are in different tickets.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1941#comment:34&gt;
GRASS GIS <http://grass.osgeo.org>

36

#1941: wxGUI fails with Japanese locale --> mixed Python installs on Windows
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: major | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by Venkat):

Replying to [comment:34 wenzeslaus]:
> Replying to [comment:33 Venkat]:
> > MAXREPEAT issue is fixed and GRASS-7 starts after complaining about
missing HDF5 and !MrSid DLL. Could be better if gdal is compiled
> > without HDF5 and !MrSid support.
> >
> > attachment:missing_dll.png
>
> Is this still a problem? If yes I suggest you to create a new ticket for
this with the all details and close this ticket where all errors were
solved or are in different tickets.

GRASS 7 starts after complaining about not finding HDF5 and MrSID dll's.
The warnings are in the image below that was posted earlier.
https://trac.osgeo.org/grass/attachment/ticket/1941/missing_dll.png

Best

Venka

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1941#comment:35&gt;
GRASS GIS <http://grass.osgeo.org>

37

#1941: wxGUI fails with Japanese locale --> mixed Python installs on Windows
----------------------+-----------------------------------------------------
Reporter: venkat | Owner: grass-dev@…
     Type: defect | Status: new
Priority: major | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Keywords: wingrass | Platform: MSWindows 7
      Cpu: x86-32 |
----------------------+-----------------------------------------------------

Comment(by neteler):

The "Error 127: the specified procedure could not be found"

may have as reason DLL mixup. Please

  * check if you have multiple versions of the same DLL in different
folders,
  * in case see: http://msdn.microsoft.com/en-
us/library/ms682586%28VS.85%29.aspx
  * see for a similar ticket: http://trac.osgeo.org/gdal/ticket/5110

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1941#comment:36&gt;
GRASS GIS <http://grass.osgeo.org>

38

#1941: wxGUI fails with Japanese locale --> mixed Python installs on Windows
--------------------------+-------------------------------------------------
  Reporter: venkat | Owner: grass-dev@…
      Type: defect | Status: closed
  Priority: major | Milestone: 7.0.0
Component: wxGUI | Version: svn-trunk
Resolution: fixed | Keywords: wingrass
  Platform: MSWindows 7 | Cpu: x86-32
--------------------------+-------------------------------------------------
Changes (by annakrat):

  * status: new => closed
  * resolution: => fixed

Comment:

This ticket doesn't make any sense now. Please open a new ticket for the
current problems if necessary. The original problem seems to be fixed.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1941#comment:37&gt;
GRASS GIS <http://grass.osgeo.org>