[GRASS-dev] [GRASS GIS] #3723: winGRASS7.7.svn - python 3 issues

GDAL · January 1, 2019, 9:04pm

#3723: winGRASS7.7.svn - python 3 issues
------------------------------------+-------------------------
Reporter: hellik | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone:
Component: Python | Version: svn-trunk
Keywords: python3, py3, wingrass | CPU: Unspecified
Platform: MSWindows |
------------------------------------+-------------------------
tested with a self compiled winGRASS7.7.svn with python 3

{{{
C:\>g.version -e -g
version=7.7.svn
date=2019
revision=r73896M
build_date=2019-01-01
build_platform=x86_64-w64-mingw32
build_off_t_size=8
proj4=5.2.0
gdal=2.4.0
geos=3.7.0
sqlite=3.26.0
}}}

{{{
C:\>grass77svn --text
Starting GRASS GIS...
WARNING: Concurrent mapset locking is not supported on Windows
Cleaning up temporary files...

           __________ ___ __________ _______________
          / ____/ __ \/ | / ___/ ___/ / ____/ _/ ___/
         / / __/ /_/ / /| | \__ \\_ \ / / __ / / \__ \
        / /_/ / _, _/ ___ |___/ /__/ / / /_/ // / ___/ /
        \____/_/ |_/_/ |_/____/____/ \____/___//____/

Welcome to GRASS GIS 7.7.svn (r73896M)
GRASS GIS homepage: https://grass.osgeo.org
This version running through: Command Prompt
(C:\WINDOWS\system32\cmd.exe)
Help is available with the command: g.manual -i
See the licence terms with: g.version -c
See citation options with: g.version -x
Start the GUI with: g.gui wxpython
When ready to quit enter: exit

C:\>python
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
}}}

then g.gui wxpython errors in

{{{
C:\>g.gui wxpython
Launching <wxpython> GUI in the background, please wait...
Traceback (most recent call last):
File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py",
line 41, in <module>
import __builtin__
ModuleNotFoundError: No module named '__builtin__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
   File "C:\OSGEO4~1\apps\grass\grass-7.7.svn/gui/wxpython/wxgui.py", line
27, in <module>
     from core import globalvar
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\globalvar.py",
line 31, in <module>
     from core.debug import Debug
   File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\debug.py",
line 25, in <module>
     import grass.script as grass
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\__init__.py",
line 5, in <module>
     from .core import *
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py",
line 47, in <module>
     from os import environb as environ
ImportError: cannot import name 'environb' from 'os'
(C:\OSGEO4~1\apps\Python37\lib\os.py)
}}}

it seems that core.py hangs around

{{{
try:
     # python2
     import __builtin__
     from os import environ
     __builtin__.__dict__['_'] =
__builtin__.__dict__['_'].__self__.ugettext
except ImportError:
     # python3
     import builtins as __builtin__
     from os import environb as environ
     unicode = str
     __builtin__.__dict__['_'] = __builtin__.__dict__['_'].__self__.gettext
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3723>
GRASS GIS <https://grass.osgeo.org>

GDAL · January 1, 2019, 9:14pm

#3723: winGRASS7.7.svn - python 3 issues
--------------------------+------------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Python | Version: svn-trunk
Resolution: | Keywords: python3, py3, wingrass
       CPU: Unspecified | Platform: MSWindows
--------------------------+------------------------------------

Comment (by hellik):

Replying to [ticket:3723 hellik]:
> from os import environb as environ
> ImportError: cannot import name 'environb' from 'os'
(C:\OSGEO4~1\apps\Python37\lib\os.py)
> }}}

regarding the environb issue, see

https://docs.python.org/3/library/os.html

{{{
os.environb¶

Bytes version of environ: a mapping object representing the
environment as byte strings. environ and environb are synchronized (modify
environb updates environ, and vice versa).

environb is only available if supports_bytes_environ is True.
}}}

https://docs.python.org/3/library/os.html#os.supports_bytes_environ

{{{
os.supports_bytes_environ

True if the native OS type of the environment is bytes (eg. False on
Windows).
}}}

it seems this isn't available in windows

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3723#comment:1>
GRASS GIS <https://grass.osgeo.org>

GDAL · January 1, 2019, 9:43pm

#3723: winGRASS7.7.svn - python 3 issues
--------------------------+------------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Python | Version: svn-trunk
Resolution: | Keywords: python3, py3, wingrass
       CPU: Unspecified | Platform: MSWindows
--------------------------+------------------------------------

Comment (by hellik):

Replying to [comment:1 hellik]:
> Replying to [ticket:3723 hellik]:
> > from os import environb as environ
> > ImportError: cannot import name 'environb' from 'os'
(C:\OSGEO4~1\apps\Python37\lib\os.py)
> > }}}
>
> regarding the environb issue, see
>
> https://docs.python.org/3/library/os.html
>
> {{{
> os.environb¶
>
> Bytes version of environ: a mapping object representing the
environment as byte strings. environ and environb are synchronized (modify
environb updates environ, and vice versa).
>
> environb is only available if supports_bytes_environ is True.
> }}}
>
> https://docs.python.org/3/library/os.html#os.supports_bytes_environ
>
> {{{
> os.supports_bytes_environ
>
> True if the native OS type of the environment is bytes (eg. False on
Windows).
> }}}
>
> it seems this isn't available in windows
>

see (GRASS related):

https://stackoverflow.com/questions/52269281/fix-import-error-on-using-
environb-in-python#

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3723#comment:2>
GRASS GIS <https://grass.osgeo.org>

GDAL · January 2, 2019, 7:49am

#3723: winGRASS7.7.svn - core.py - python 3 issues
--------------------------+------------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.8.0
Component: Python | Version: svn-trunk
Resolution: | Keywords: python3, py3, wingrass
       CPU: Unspecified | Platform: MSWindows
--------------------------+------------------------------------
Changes (by hellik):

* priority: normal => blocker
* milestone: => 7.8.0

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3723#comment:3>
GRASS GIS <https://grass.osgeo.org>

GDAL · January 3, 2019, 10:43pm

#3723: winGRASS7.7.svn - core.py - python 3 issues
--------------------------+------------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.8.0
Component: Python | Version: svn-trunk
Resolution: | Keywords: python3, py3, wingrass
       CPU: Unspecified | Platform: MSWindows
--------------------------+------------------------------------

Comment (by martinl):

In [changeset:"73906" 73906]:
{{{
#!CommitTicketReference repository="" revision="73906"
avoid usage of os.environb, inspired by
https://stackoverflow.com/questions/52269281/fix-import-error-on-using-
environb-in-python, see #3723
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3723#comment:4>
GRASS GIS <https://grass.osgeo.org>

GDAL · January 3, 2019, 10:45pm

#3723: winGRASS7.7.svn - core.py - python 3 issues
--------------------------+------------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.8.0
Component: Python | Version: svn-trunk
Resolution: | Keywords: python3, py3, wingrass
       CPU: Unspecified | Platform: MSWindows
--------------------------+------------------------------------

Comment (by martinl):

Replying to [comment:2 hellik]:
> https://stackoverflow.com/questions/52269281/fix-import-error-on-using-
environb-in-python#

suggested modification applied in r73906 (for better testing). It will
appear in winGRASS build no. 364+.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3723#comment:5>
GRASS GIS <https://grass.osgeo.org>

GDAL · January 5, 2019, 9:32am

#3723: winGRASS7.7.svn - core.py - python 3 issues
--------------------------+------------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.8.0
Component: Python | Version: svn-trunk
Resolution: | Keywords: python3, py3, wingrass
       CPU: Unspecified | Platform: MSWindows
--------------------------+------------------------------------

Comment (by hellik):

Replying to [comment:5 martinl]:
> Replying to [comment:2 hellik]:
> > https://stackoverflow.com/questions/52269281/fix-import-error-on-
using-environb-in-python#
>
> suggested modification applied in r73906 (for better testing). It will
appear in winGRASS build no. 364+.

tested with

{{{
Welcome to GRASS GIS 7.7.svn (r73906)
}}}

{{{
C:\>g.gui wxpython
Launching <wxpython> GUI in the background, please wait...

C:\>Traceback (most recent call last):
   File "C:\OSGeo4W\apps\grass\grass-7.7.svn/gui/wxpython/wxgui.py", line
27, in
<module>
     from core import globalvar
   File
"C:\OSGeo4W\apps\grass\grass-7.7.svn\gui\wxpython\core\globalvar.py", lin
e 31, in <module>
     from core.debug import Debug
   File "C:\OSGeo4W\apps\grass\grass-7.7.svn\gui\wxpython\core\debug.py",
line 77
, in <module>
     Debug = DebugMsg()
   File "C:\OSGeo4W\apps\grass\grass-7.7.svn\gui\wxpython\core\debug.py",
line 39
, in __init__
     self.SetLevel()
   File "C:\OSGeo4W\apps\grass\grass-7.7.svn\gui\wxpython\core\debug.py",
line 45
, in SetLevel
     self.debuglevel = int(grass.gisenv().get('WX_DEBUG', 0))
   File
"C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 1082, in gisenv
     s = read_command("g.gisenv", flags='n', env=env)
   File
"C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 502, in read_command
     process = pipe_command(*args, **kwargs)
   File
"C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 471, in pipe_command
     return start_command(*args, **kwargs)
   File
"C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 396, in start_command
     if debug_level() > 0:
   File
"C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 1670, in debug_level
     _debug_level = int(gisenv().get('DEBUG', 0))
   File
"C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 1082, in gisenv
     s = read_command("g.gisenv", flags='n', env=env)
   File
"C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 502, in read_command
     process = pipe_command(*args, **kwargs)
   File
"C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 471, in pipe_command
     return start_command(*args, **kwargs)
   File
"C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 401, in start_command
     return Popen(args, **popts)
   File
"C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 66, in __init__
     cmd = shutil_which(args[0])
   File
"C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 220, in shutil_which
     if any(cmd.lower().endswith(ext.lower()) for ext in pathext):
   File
"C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
ne 220, in <genexpr>
     if any(cmd.lower().endswith(ext.lower()) for ext in pathext):
TypeError: endswith first arg must be bytes or a tuple of bytes, not str
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3723#comment:6>
GRASS GIS <https://grass.osgeo.org>

GDAL · January 5, 2019, 8:12pm

#3723: winGRASS7.7.svn - core.py - python 3 issues
--------------------------+------------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.8.0
Component: Python | Version: svn-trunk
Resolution: | Keywords: python3, py3, wingrass
       CPU: Unspecified | Platform: MSWindows
--------------------------+------------------------------------

Comment (by hellik):

Replying to [comment:6 hellik]:
> [...]
> if any(cmd.lower().endswith(ext.lower()) for ext in pathext):
> File
"C:\OSGeo4W\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py", li
> ne 220, in <genexpr>
> if any(cmd.lower().endswith(ext.lower()) for ext in pathext):
> TypeError: endswith first arg must be bytes or a tuple of bytes, not str
> }}}
>
>

now also tested with the OSGeo4W 64bit version.

{{{
C:\>g.version -g
version=7.7.svn
date=2019
revision=r73906
build_date=2019-01-04
build_platform=x86_64-w64-mingw32
build_off_t_size=8
}}}

same error

{{{
C:\>g.gui wxpython
Launching <wxpython> GUI in the background, please wait...
Traceback (most recent call last):
   File "C:\OSGEO4~1\apps\grass\grass-7.7.svn/gui/wxpython/wxgui.py", line
27, in <module>
     from core import globalvar
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\globalvar.py",
line 31, in <module>
     from core.debug import Debug
   File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\debug.py",
line 77, in <module>
     Debug = DebugMsg()
   File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\debug.py",
line 39, in __init__
     self.SetLevel()
   File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\debug.py",
line 45, in SetLevel
     self.debuglevel = int(grass.gisenv().get('WX_DEBUG', 0))
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py",
line 1082, in gisenv
     s = read_command("g.gisenv", flags='n', env=env)
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py",
line 502, in read_command
     process = pipe_command(*args, **kwargs)
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py",
line 471, in pipe_command
     return start_command(*args, **kwargs)
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py",
line 396, in start_command
     if debug_level() > 0:
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py",
line 1670, in debug_level
     _debug_level = int(gisenv().get('DEBUG', 0))
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py",
line 1082, in gisenv
     s = read_command("g.gisenv", flags='n', env=env)
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py",
line 502, in read_command
     process = pipe_command(*args, **kwargs)
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py",
line 471, in pipe_command
     return start_command(*args, **kwargs)
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py",
line 401, in start_command
     return Popen(args, **popts)
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py",
line 66, in __init__
     cmd = shutil_which(args[0])
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py",
line 220, in shutil_which
     if any(cmd.lower().endswith(ext.lower()) for ext in pathext):
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\etc\python\grass\script\core.py",
line 220, in <genexpr>
     if any(cmd.lower().endswith(ext.lower()) for ext in pathext):
TypeError: endswith first arg must be bytes or a tuple of bytes, not str
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3723#comment:7>
GRASS GIS <https://grass.osgeo.org>

GDAL · January 12, 2019, 10:06am

#3723: winGRASS7.7.svn - core.py - python 3 issues
--------------------------+------------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.8.0
Component: Python | Version: svn-trunk
Resolution: | Keywords: python3, py3, wingrass
       CPU: Unspecified | Platform: MSWindows
--------------------------+------------------------------------

Comment (by martinl):

In [changeset:"73930" 73930]:
{{{
#!CommitTicketReference repository="" revision="73930"
wingrass77 core python3 issues, see #3723
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3723#comment:8>
GRASS GIS <https://grass.osgeo.org>

GDAL · January 12, 2019, 10:25am

#3723: winGRASS7.7.svn - core.py - python 3 issues
--------------------------+------------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.8.0
Component: Python | Version: svn-trunk
Resolution: | Keywords: python3, py3, wingrass
       CPU: Unspecified | Platform: MSWindows
--------------------------+------------------------------------

Comment (by martinl):

Reported issue should be fixed in r73930. You can test it with WinGRASS
daily build no. 372+. See related #3731

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3723#comment:9>
GRASS GIS <https://grass.osgeo.org>

GDAL · January 13, 2019, 12:43pm

#3723: winGRASS7.7.svn - core.py - python 3 issues
--------------------------+------------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: closed
  Priority: blocker | Milestone: 7.8.0
Component: Python | Version: svn-trunk
Resolution: fixed | Keywords: python3, py3, wingrass
       CPU: Unspecified | Platform: MSWindows
--------------------------+------------------------------------
Changes (by martinl):

* status: new => closed
* resolution: => fixed

Comment:

Build no.372 tested. Original issue seems to be fixed. Feel free to reopen
if needed.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3723#comment:10>
GRASS GIS <https://grass.osgeo.org>

GDAL · January 16, 2019, 7:03am

#3723: winGRASS7.7.svn - core.py - python 3 issues
--------------------------+------------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: closed
  Priority: blocker | Milestone: 7.8.0
Component: Python | Version: svn-trunk
Resolution: fixed | Keywords: python3, py3, wingrass
       CPU: Unspecified | Platform: MSWindows
--------------------------+------------------------------------

Comment (by martinl):

Build no. 375 fails with new issues

{{{
Traceback (most recent call last):
   File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\wxgui.py", line
103, in OnInit
     workspace=self.workspaceFile)
   File "C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\lmgr\frame.py",
line 131, in __init__
     self._moduleTreeBuilder =
LayerManagerModuleTree(message_handler=add_menu_error)
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\lmgr\menudata.py", line
69, in __init__
     message_handler=message_handler)
   File
"C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\core\menutree.py", line
72, in __init__
     xmlTree = etree.parse(filename)
   File "C:\OSGEO4~1\apps\Python37\lib\xml\etree\ElementTree.py", line
1197, in parse
     tree.parse(source, parser)
   File "C:\OSGEO4~1\apps\Python37\lib\xml\etree\ElementTree.py", line 598,
in parse
     self._root = parser._parse_whole(source)
xml.etree.ElementTree.ParseError: no element found: line 1, column 0
}}}

Reason:
`C:\OSGEO4~1\apps\grass\grass-7.7.svn\gui\wxpython\xml\module_tree_menudata.xml`
is empty.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3723#comment:11>
GRASS GIS <https://grass.osgeo.org>