[GRASS-user] Trying to use GRASS with Jupyter note books in Windows

I’m trying to follow this setup:
https://github.com/ncsu-geoforall-lab/GIS714-assignments/blob/main/GRASS_GIS_Foundations/JupyterOnWindows_Tutorial.ipynb

I have stand-alone GRASS GIS 8.3 installed in Windows in C:\GRASS GIS 8.3

I followed the 7 setup steps (changing set path to reflect that its installed in C:\GRASS GIS 8.3
Then when i run this first part:

import os
import subprocess
import sys

Ask GRASS GIS where its Python packages are.

sys.path.append(
subprocess.check_output([“grass”, “–config”, “python_path”], text=True).strip()
)

Import GRASS packages

import grass.script as gs
import grass.jupyter as gj


I get the following error:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Cell In[1], line 7
      3 import sys
      5 # Ask GRASS GIS where its Python packages are.
      6 sys.path.append(
----> 7     subprocess.check_output(["grass", "--config", "python_path"], text=True).strip()
      8 )
     10 # Import GRASS packages
     11 import grass.script as gs

File C:\GRASS GIS 8.3\Python39\lib\subprocess.py:424, in check_output(timeout, *popenargs, **kwargs)
    421         empty = b''
    422     kwargs['input'] = empty
--> 424 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
    425            **kwargs).stdout

File C:\GRASS GIS 8.3\Python39\lib\subprocess.py:505, in run(input, capture_output, timeout, check, *popenargs, **kwargs)
    502     kwargs['stdout'] = PIPE
    503     kwargs['stderr'] = PIPE
--> 505 with Popen(*popenargs, **kwargs) as process:
    506     try:
    507         stdout, stderr = process.communicate(input, timeout=timeout)

File C:\GRASS GIS 8.3\Python39\lib\subprocess.py:951, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask)
    947         if self.text_mode:
    948             self.stderr = io.TextIOWrapper(self.stderr,
    949                     encoding=encoding, errors=errors)
--> 951     self._execute_child(args, executable, preexec_fn, close_fds,
    952                         pass_fds, cwd, env,
    953                         startupinfo, creationflags, shell,
    954                         p2cread, p2cwrite,
    955                         c2pread, c2pwrite,
    956                         errread, errwrite,
    957                         restore_signals,
    958                         gid, gids, uid, umask,
    959                         start_new_session)
    960 except:
    961     # Cleanup if the child failed starting.
    962     for f in filter(None, (self.stdin, self.stdout, self.stderr)):

File C:\GRASS GIS 8.3\Python39\lib\subprocess.py:1420, in Popen._execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_gid, unused_gids, unused_uid, unused_umask, unused_start_new_session)
   1418 # Start the process
   1419 try:
-> 1420     hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
   1421                              # no special security
   1422                              None, None,
   1423                              int(not close_fds),
   1424                              creationflags,
   1425                              env,
   1426                              cwd,
   1427                              startupinfo)
   1428 finally:
   1429     # Child is launched. Close the parent's copy of those pipe
   1430     # handles that only the child should have open.  You need
   (...)
   1433     # pipe will not close when the child process exits and the
   1434     # ReadFile will hang.
   1435     self._close_pipe_fds(p2cread, p2cwrite,
   1436                          c2pread, c2pwrite,
   1437                          errread, errwrite)

FileNotFoundError: [WinError 2] The system cannot find the file specified

Can someone help figure out what the issue is?
Thanks,
Vishal

Hi Vishal,

The executable on Windows is not called “grass” like on Linux but “grass8” or “grass83” (to avoid a strange Python import error). That should do the trick. If not, try simply providing a full path yourself including the extension (C:.….…bat).

Best,
Vaclav

On Wed, 27 Sept 2023 at 21:20, Vishal Mehta via grass-user <grass-user@lists.osgeo.org> wrote:

I’m trying to follow this setup:
https://github.com/ncsu-geoforall-lab/GIS714-assignments/blob/main/GRASS_GIS_Foundations/JupyterOnWindows_Tutorial.ipynb

I have stand-alone GRASS GIS 8.3 installed in Windows in C:\GRASS GIS 8.3

I followed the 7 setup steps (changing set path to reflect that its installed in C:\GRASS GIS 8.3
Then when i run this first part:

import os
import subprocess
import sys

Ask GRASS GIS where its Python packages are.

sys.path.append(
subprocess.check_output([“grass”, “–config”, “python_path”], text=True).strip()
)

Import GRASS packages

import grass.script as gs
import grass.jupyter as gj


I get the following error:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Cell In[1], line 7
      3 import sys
      5 # Ask GRASS GIS where its Python packages are.
      6 sys.path.append(
----> 7     subprocess.check_output(["grass", "--config", "python_path"], text=True).strip()
      8 )
     10 # Import GRASS packages
     11 import grass.script as gs

File C:\GRASS GIS 8.3\Python39\lib\subprocess.py:424, in check_output(timeout, *popenargs, **kwargs)
    421         empty = b''
    422     kwargs['input'] = empty
--> 424 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
    425            **kwargs).stdout

File C:\GRASS GIS 8.3\Python39\lib\subprocess.py:505, in run(input, capture_output, timeout, check, *popenargs, **kwargs)
    502     kwargs['stdout'] = PIPE
    503     kwargs['stderr'] = PIPE
--> 505 with Popen(*popenargs, **kwargs) as process:
    506     try:
    507         stdout, stderr = process.communicate(input, timeout=timeout)

File C:\GRASS GIS 8.3\Python39\lib\subprocess.py:951, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask)
    947         if self.text_mode:
    948             self.stderr = io.TextIOWrapper(self.stderr,
    949                     encoding=encoding, errors=errors)
--> 951     self._execute_child(args, executable, preexec_fn, close_fds,
    952                         pass_fds, cwd, env,
    953                         startupinfo, creationflags, shell,
    954                         p2cread, p2cwrite,
    955                         c2pread, c2pwrite,
    956                         errread, errwrite,
    957                         restore_signals,
    958                         gid, gids, uid, umask,
    959                         start_new_session)
    960 except:
    961     # Cleanup if the child failed starting.
    962     for f in filter(None, (self.stdin, self.stdout, self.stderr)):

File C:\GRASS GIS 8.3\Python39\lib\subprocess.py:1420, in Popen._execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_gid, unused_gids, unused_uid, unused_umask, unused_start_new_session)
   1418 # Start the process
   1419 try:
-> 1420     hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
   1421                              # no special security
   1422                              None, None,
   1423                              int(not close_fds),
   1424                              creationflags,
   1425                              env,
   1426                              cwd,
   1427                              startupinfo)
   1428 finally:
   1429     # Child is launched. Close the parent's copy of those pipe
   1430     # handles that only the child should have open.  You need
   (...)
   1433     # pipe will not close when the child process exits and the
   1434     # ReadFile will hang.
   1435     self._close_pipe_fds(p2cread, p2cwrite,
   1436                          c2pread, c2pwrite,
   1437                          errread, errwrite)

FileNotFoundError: [WinError 2] The system cannot find the file specified

Can someone help figure out what the issue is?
Thanks,
Vishal


grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

On Thu, Sep 28, 2023 at 5:22 AM Vaclav Petras via grass-user
<grass-user@lists.osgeo.org> wrote:

Hi Vishal,

The executable on Windows is not called "grass" like on Linux but "grass8" or "grass83" (to avoid a strange Python
import error). That should do the trick. If not, try simply providing a full path yourself including the extension (C:\...\...bat).

I have taken liberty to cite this note here:

https://grasswiki.osgeo.org/wiki/GRASS_GIS_Jupyter_notebooks#Windows_tricks

Best,
Markus

Thanks Markus and Vaclav,

This is what worked for me on Windows:

Ask GRASS GIS where its Python packages are.

sys.path.append(
subprocess.check_output([“C:\GRASS GIS 8.3/grass83.bat”, “–config”, “python_path”], text=True).strip()
)

Also, can I use bash within the jupyter notebook? I’m not familiar with the gs.script syntax; I’m very comfortable with the old bash ways…
so any day I would rather
glist rast than
gs.list_grouped(type=[‘raster’])

What i’m hoping for is a way to mix and match within a single workflow, taking advantage of things like the interactive map via python what-have-you’s, but sticking to the bash commands for pretty much everything else.

I saw some notebooks with something like
%%bash

but that didn’t work for me…

Thanks,
Vishal

On Thu, Sep 28, 2023 at 12:46 AM Markus Neteler <neteler@osgeo.org> wrote:

On Thu, Sep 28, 2023 at 5:22 AM Vaclav Petras via grass-user
<grass-user@lists.osgeo.org> wrote:

Hi Vishal,

The executable on Windows is not called “grass” like on Linux but “grass8” or “grass83” (to avoid a strange Python
import error). That should do the trick. If not, try simply providing a full path yourself including the extension (C:.….…bat).

I have taken liberty to cite this note here:

https://grasswiki.osgeo.org/wiki/GRASS_GIS_Jupyter_notebooks#Windows_tricks

Best,
Markus

On Thu, Sep 28, 2023 at 10:09 PM Vishal Mehta <vishalm1975@gmail.com> wrote:

Thanks Markus and Vaclav,

This is what worked for me on Windows:
# Ask GRASS GIS where its Python packages are.
sys.path.append(
    subprocess.check_output(["C:\GRASS GIS 8.3/grass83.bat", "--config", "python_path"], text=True).strip()
)

Perfect, added to the Wiki page, too.

Also, can I use bash within the jupyter notebook?

Seems yes, after:

pip install bash_kernel

Markus

Thanks Markus,
I installed bash kernel; and it seems that it is there (#lsmagic lists bash as one of the cell magics)

but I get the error below:

##bash
g.list type=rast

---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
Cell In[20], line 1
----> 1 get_ipython().run_cell_magic('bash', '', 'g.list rast\n')

File C:\GRASS GIS 8.3\Python39\lib\site-packages\IPython\core\interactiveshell.py:2493, in InteractiveShell.run_cell_magic(self, magic_name, line, cell)
   2491 with self.builtin_trap:
   2492     args = (magic_arg_s, cell)
-> 2493     result = fn(*args, **kwargs)
   2495 # The code below prevents the output from being displayed
   2496 # when using magics with decorator @output_can_be_silenced
   2497 # when the last Python token in the expression is a ';'.
   2498 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False):

File C:\GRASS GIS 8.3\Python39\lib\site-packages\IPython\core\magics\script.py:154, in ScriptMagics._make_script_magic.<locals>.named_script_magic(line, cell)
    152 else:
    153     line = script
--> 154 return self.shebang(line, cell)

File C:\GRASS GIS 8.3\Python39\lib\site-packages\IPython\core\magics\script.py:314, in ScriptMagics.shebang(self, line, cell)
    309 if args.raise_error and p.returncode != 0:
    310     # If we get here and p.returncode is still None, we must have
    311     # killed it but not yet seen its return code. We don't wait for it,
    312     # in case it's stuck in uninterruptible sleep. -9 = SIGKILL
    313     rc = p.returncode or -9
--> 314     raise CalledProcessError(rc, cell)

CalledProcessError: Command 'b'g.list rast\n'' returned non-zero exit status 127.
--

Also I am unable to install extensions r.stream.distance:
!g.extension r.stream.distance
as well as through the GUI

both give me a "cannot find executable git" error

Best,
Vishal

···

Vishal K. Mehta, PhD
Sr. Scientist
Stockholm Environment Institute - US
501 D St
Davis CA 95616
https://www.sei.org/people/vishal-mehta/

On Sat, Sep 30, 2023 at 12:36 AM Vishal Mehta <vishalm1975@gmail.com> wrote:

Thanks Markus,
I installed bash kernel; and it seems that it is there (#lsmagic lists bash as one of the cell magics)

but I get the error below:

##bash
g.list type=rast

[...]

Probably worth a bug report in GitHub.

Also I am unable to install extensions r.stream.distance:

!g.extension r.stream.distance

as well as through the GUI

both give me a "cannot find executable git" error

This has been this bug:
https://github.com/OSGeo/grass/issues/3077

which was fixed last week:
https://github.com/OSGeo/grass/pull/3166

It will be fixed in GRASS GIS 8.3.1 - release in preparation.

Best,
Markus

On Thu, 28 Sept 2023 at 16:09, Vishal Mehta <vishalm1975@gmail.com> wrote:

Also, can I use bash within the jupyter notebook? I’m not familiar with the gs.script syntax; I’m very comfortable with the old bash ways…
so any day I would rather
glist rast than
gs.list_grouped(type=[‘raster’])

That’s really your choice as a user of Jupyter, GRASS GIS just provides the Python API in addition to the command line interface. The command line interface is there and you can use it from Bash or whatever else.

What i’m hoping for is a way to mix and match within a single workflow, taking advantage of things like the interactive map via python what-have-you’s, but sticking to the bash commands for pretty much everything else.

Use the exclamation mark lines in Jupyter (with IPython kernel). That passes the line to your operating system. If you simply want general command line syntax, that’s the way to go on any platform. With that, you can mix and match easily and that’s relatively common thing to do. I’m doing it a lot when experimenting, but for sharing I usually go with one syntax which then needs to be Python.

I saw some notebooks with something like
%%bash

but that didn’t work for me…

I would have to check, but the I would guess that your Windows does not have Bash on path? This %%bash creates quite nice experience on Linux (locally, cloud, etc.), but I don’t know if it is the right choice on Windows, but that’s up to you to decide.