[GRASS-user] exwcuting grass python script from php web page

GRASS GRASS user
1

Hi all!
I wrote a python script with grass to be run from a php web page.
The python script works perfectly if I run it from the terminal but when I try to run it from the php web page, the grass related part seems to be ignored and the grass session is not launched. I set the grass gis environment following this link [0].
From the php side, I try to launch the python script using both exec(), system() and also a bash script to run the python script but I always got the same error.
Attached you can find the python script.

Does anybody have any ideas on how to fix it?

Thanks in advance for the help!
Roberta

[0] https://grasswiki.osgeo.org/wiki/Working_with_GRASS_without_starting_it_explicitly#Python:_GRASS_GIS_7_with_existing_location


Working with GRASS without starting it explicitly - GRASS-Wiki - GRASS-Wiki - GRASS-Wiki
GRASS sessions. It is possible to access GRASS modules without explicitly starting a “GRASS session”. GRASS libraries require certain environment variables to be set. In fact a “GRASS session” is just a set of processes (e.g. a shell and/or GUI) which have the necessary environment settings, specifically:
grasswiki.osgeo.org
(attachments)

importgrass.py (2.04 KB)

2

On 28/09/20 12:10, roberta fagandini wrote:

Hi all!
I wrote a python script with grass to be run from a php web page.
The python script works perfectly if I run it from the terminal but when I try to run it from the php web page, the grass related part seems to be ignored and the grass session is not launched. I set the grass gis environment following this link [0].
From the php side, I try to launch the python script using both exec(), system() and also a bash script to run the python script but I always got the same error.

Maybe you could tell us what error you get ?

Moritz

3

Sorry, maybe I didn’t explain myself well because actually, I don’t get any error from the php web page, simply the code related to grass is not executed.For instance gsetup.init(gisbase, gisdb, location, mapset) does not create any file in my /tmp/ folder and the print(gscript.gisenv()) is not printed.
I checked the import of the grass libraries (print(sys.modules)) and they seem to be correctly imported.

···

Da: Moritz Lennert mlennert@club.worldonline.be
Inviato: lunedì 28 settembre 2020 17:29
A: roberta fagandini robifagandini@hotmail.it; grass-user@lists.osgeo.org grass-user@lists.osgeo.org
Oggetto: Re: [GRASS-user] exwcuting grass python script from php web page

On 28/09/20 12:10, roberta fagandini wrote:

Hi all!
I wrote a python script with grass to be run from a php web page.
The python script works perfectly if I run it from the terminal but when
I try to run it from the php web page, the grass related part seems to
be ignored and the grass session is not launched. I set the grass gis
environment following this link [0].
From the php side, I try to launch the python script using both exec(),
system() and also a bash script to run the python script but I always
got the same error.

Maybe you could tell us what error you get ?

Moritz

4

On 28/09/20 17:51, roberta fagandini wrote:

Sorry, maybe I didn't explain myself well because actually, I don't get any error from the php web page, simply the code related to grass is not executed.For instance gsetup.init(gisbase, gisdb, location, mapset) does not create any file in my /tmp/ folder and the print(gscript.gisenv()) is not printed.
I checked the import of the grass libraries (print(sys.modules)) and they seem to be correctly imported.

Have you checked your webservers error logs ?

How do you call the python script from PhP ?

Are you sure that your web server has sufficient permissions to access the relevant directories ?

Moritz

------------------------------------------------------------------------
*Da:* Moritz Lennert <mlennert@club.worldonline.be>
*Inviato:* lunedì 28 settembre 2020 17:29
*A:* roberta fagandini <robifagandini@hotmail.it>; grass-user@lists.osgeo.org <grass-user@lists.osgeo.org>
*Oggetto:* Re: [GRASS-user] exwcuting grass python script from php web page
On 28/09/20 12:10, roberta fagandini wrote:

Hi all!
I wrote a python script with grass to be run from a php web page.
The python script works perfectly if I run it from the terminal but when I try to run it from the php web page, the grass related part seems to be ignored and the grass session is not launched. I set the grass gis environment following this link [0].
From the php side, I try to launch the python script using both exec(), system() and also a bash script to run the python script but I always got the same error.

Maybe you could tell us what error you get ?

Moritz

5

On 28/09/20 17:51, roberta fagandini wrote:

Sorry, maybe I didn't explain myself well because actually, I don't get
any error from the php web page, simply the code related to grass is not
executed.For instance gsetup.init(gisbase, gisdb, location, mapset) does
not create any file in my /tmp/ folder and the print(gscript.gisenv())
is not printed.
I checked the import of the grass libraries (print(sys.modules)) and
they seem to be correctly imported.

Have you checked your webservers error logs ?

I have already checked and this is the error in the webserver log file

File "importgrass.py", line 82, in <module>
    main()
  File "importgrass.py", line 71, in main
    rcfile = gsetup.init(gisbase, gisdb, location, mapset)
  File "/usr/lib/grass74/etc/python/grass/script/setup.py", line 170, in init
    config_dir = os.path.join(os.getenv('HOME'), config_dirname)
  File "/usr/lib/python3.6/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

How do you call the python script from PhP?

I tried both with:
<?php
       $command = escapeshellcmd('/usr/bin/python3 importgrass.py ');
        echo exec($command, $output, $return);
?>

and

<?php
       $command = escapeshellcmd('/usr/bin/python3 importgrass.py ');
        echo system($command, $return);
?>

and

<?php
       $intestazione='#!/bin/bash'.PHP_EOL;
       $command = escapeshellcmd('/usr/bin/python3 importgrass.py ');
       $myfile = fopen("./tmp/script.sh", "w") or die("Unable to open file!");
       fwrite($myfile, $intestazione);
       fwrite($myfile, $command);
       fclose($myfile);
       echo exec("/bin/sh ./tmp/script.sh", $output, $return);
       print_r($output);
?>

Are you sure that your web server has sufficient permissions to access
the relevant directories ?

Yes, I'm pretty sure! If I run the python script from terminal as www-data it works perfectly.

Moritz

Thanks!
Roberta

------------------------------------------------------------------------
*Da:* Moritz Lennert <mlennert@club.worldonline.be>
*Inviato:* lunedì 28 settembre 2020 17:29
*A:* roberta fagandini <robifagandini@hotmail.it>;
grass-user@lists.osgeo.org <grass-user@lists.osgeo.org>
*Oggetto:* Re: [GRASS-user] exwcuting grass python script from php web page
On 28/09/20 12:10, roberta fagandini wrote:

Hi all!
I wrote a python script with grass to be run from a php web page.
The python script works perfectly if I run it from the terminal but when
I try to run it from the php web page, the grass related part seems to
be ignored and the grass session is not launched. I set the grass gis
environment following this link [0].
From the php side, I try to launch the python script using both exec(),
system() and also a bash script to run the python script but I always
got the same error.

Maybe you could tell us what error you get ?

Moritz

6

Hi Roberta,

I'm really no expert in this domain, so just doing some educated guessing.

On 28/09/20 18:37, roberta fagandini wrote:

On 28/09/20 17:51, roberta fagandini wrote:

Sorry, maybe I didn't explain myself well because actually, I don't get
any error from the php web page, simply the code related to grass is not executed.For instance gsetup.init(gisbase, gisdb, location, mapset) does not create any file in my /tmp/ folder and the print(gscript.gisenv()) is not printed.
I checked the import of the grass libraries (print(sys.modules)) and they seem to be correctly imported.

Have you checked your webservers error logs ?

I have already checked and this is the error in the webserver log file

File "importgrass.py", line 82, in <module>
main()
File "importgrass.py", line 71, in main
rcfile = gsetup.init(gisbase, gisdb, location, mapset)
File "/usr/lib/grass74/etc/python/grass/script/setup.py", line 170, in init
config_dir = os.path.join(os.getenv('HOME'), config_dirname)
File "/usr/lib/python3.6/posixpath.py", line 80, in join
a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

First of all, the importgrass.py you sent only has 72 lines, so I don't know what the line 82 reference in your above error points to.

But most importantly, I would really recommend that you upgrade to a newer version of GRASS GIS. Current stable is 7.8. There's been a lot of improvement in terms of python3 compatibility and maybe this is an issue that has already been resolved.

It sounds like an issue with a variable containing a path. Apparently it is of type NoneType...

How do you call the python script from PhP?

I tried both with:
<?php
$command = escapeshellcmd('/usr/bin/python3 importgrass.py ');
echo exec($command, $output, $return);
?>

and

<?php
$command = escapeshellcmd('/usr/bin/python3 importgrass.py ');
echo system($command, $return);
?>

and

<?php
$intestazione='#!/bin/bash'.PHP_EOL;
$command = escapeshellcmd('/usr/bin/python3 importgrass.py ');
$myfile = fopen("./tmp/script.sh", "w") or die("Unable to open file!");
fwrite($myfile, $intestazione);
fwrite($myfile, $command);
fclose($myfile);
echo exec("/bin/sh ./tmp/script.sh", $output, $return);
print_r($output);
?>

What happens when you replace '/usr/bin/python3 importgrass.py ' by, for example '/usr/bin/grass --version', just to check if other executables work ?

Moritz

7

Hi Moritz,

Hi Roberta,

I’m really no expert in this domain, so just doing some educated guessing.

On 28/09/20 18:37, roberta fagandini wrote:

On 28/09/20 17:51, roberta fagandini wrote:

Sorry, maybe I didn’t explain myself well because actually, I don’t get
any error from the php web page, simply the code related to grass is not
executed.For instance gsetup.init(gisbase, gisdb, location, mapset) does
not create any file in my /tmp/ folder and the print(gscript.gisenv())
is not printed.
I checked the import of the grass libraries (print(sys.modules)) and
they seem to be correctly imported.

Have you checked your webservers error logs ?

I have already checked and this is the error in the webserver log file

File “importgrass.py”, line 82, in
main()
File “importgrass.py”, line 71, in main
rcfile = gsetup.init(gisbase, gisdb, location, mapset)
File “/usr/lib/grass74/etc/python/grass/script/setup.py”, line 170,
in init
config_dir = os.path.join(os.getenv(‘HOME’), config_dirname)
File “/usr/lib/python3.6/posixpath.py”, line 80, in join
a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

First of all, the importgrass.py you sent only has 72 lines, so I don’t
know what the line 82 reference in your above error points to.

This is simply due to some print that I added to the code before checking the error log.

But most importantly, I would really recommend that you upgrade to a
newer version of GRASS GIS. Current stable is 7.8. There’s been a lot of
improvement in terms of python3 compatibility and maybe this is an issue
that has already been resolved.

It sounds like an issue with a variable containing a path. Apparently it
is of type NoneType…

I tried to upgrade GRASS using the repository ubuntugis unstable but it caused some problems with the installation of postgis. Do you have another solution?

How do you call the python script from PhP?

I tried both with:

<?php $command = escapeshellcmd('/usr/bin/python3 importgrass.py '); echo exec($command, $output, $return); ?>

and

<?php $command = escapeshellcmd('/usr/bin/python3 importgrass.py '); echo system($command, $return); ?>

and

<?php $intestazione='#!/bin/bash'.PHP_EOL; $command = escapeshellcmd('/usr/bin/python3 importgrass.py '); $myfile = fopen("./tmp/script.sh", "w") or die("Unable to open file!"); fwrite($myfile, $intestazione); fwrite($myfile, $command); fclose($myfile); echo exec("/bin/sh ./tmp/script.sh", $output, $return); print_r($output); ?>

What happens when you replace '/usr/bin/python3 importgrass.py ’ by, for
example ‘/usr/bin/grass --version’, just to check if other executables
work ?

I tried to run ‘/usr/bin/grass --version’ but I got a similar error:

File “/usr/bin/grass”, line 2005, in
main()
File “/usr/bin/grass”, line 1804, in main
grass_config_dir = get_grass_config_dir()
File “/usr/bin/grass”, line 398, in get_grass_config_dir
directory = os.path.join(os.getenv(‘HOME’), grass_config_dirname)
File “/usr/lib/python2.7/posixpath.py”, line 70, in join
elif path == ‘’ or path.endswith(‘/’):
AttributeError: ‘NoneType’ object has no attribute ‘endswith’

Moritz

Thanks again!
Roberta

8

Hi Roberta,

On Wed, Sep 30, 2020 at 5:12 PM roberta fagandini
<robifagandini@hotmail.it> wrote:

On 28/09/20 18:37, roberta fagandini wrote:
> On 28/09/20 17:51, roberta fagandini wrote:
>> Sorry, maybe I didn't explain myself well because actually, I don't get
>> any error from the php web page, simply the code related to grass is not
>> executed.For instance gsetup.init(gisbase, gisdb, location, mapset) does
>> not create any file in my /tmp/ folder and the print(gscript.gisenv())
>> is not printed.
>> I checked the import of the grass libraries (print(sys.modules)) and
>> they seem to be correctly imported.
>
> Have you checked your webservers error logs ?
>
> I have already checked and this is the error in the webserver log file

I observe that:

> File "importgrass.py", line 82, in <module>
> main()
> File "importgrass.py", line 71, in main
> rcfile = gsetup.init(gisbase, gisdb, location, mapset)
> File "/usr/lib/grass74/etc/python/grass/script/setup.py", line 170,

... you use GRASS GIS 7.4? Pretty old and outdated.

> in init
> config_dir = os.path.join(os.getenv('HOME'), config_dirname)
> File "/usr/lib/python3.6/posixpath.py", line 80, in join

... you try to use GRASS GIS 7.4 with Python 3. For this you need
GRASS GIS 7.8 as already suggested.

> a = os.fspath(a)
> TypeError: expected str, bytes or os.PathLike object, not NoneType

...

I tried to upgrade GRASS using the repository ubuntugis unstable but it caused some problems with
the installation of postgis.

Which problems exactly?

> $command = escapeshellcmd('/usr/bin/python3 importgrass.py ');

... Python 3, fine. But here:

I tried to run '/usr/bin/grass --version' but I got a similar error:

File "/usr/bin/grass", line 2005, in <module>
    main()
  File "/usr/bin/grass", line 1804, in main
    grass_config_dir = get_grass_config_dir()
  File "/usr/bin/grass", line 398, in get_grass_config_dir
    directory = os.path.join(os.getenv('HOME'), grass_config_dirname)
  File "/usr/lib/python2.7/posixpath.py", line 70, in join

... again Python 2.

This mixture will not work. Do you have by chance multiple GRASS GIS
versions installed?

I'd suggest removing all installed GRASS packages and reinstalling 7.8
cleanly again.

best
Markus

9

Hi Markus!

We compiled GRASS 7.8 and this solved the problem of python3/python2 but we got the same error:

File “importgrass78.py”, line 79, in
main()
File “importgrass78.py”, line 68, in main
rcfile = gsetup.init(gisbase, gisdb, location, mapset)
File “/usr/local/grass78/etc/python/grass/script/setup.py”, line 175, in init
config_dir = os.path.join(os.getenv(‘HOME’), config_dirname)
File “/usr/lib/python3.6/posixpath.py”, line 80, in join
a = os.fspath(a)

This issue is due to running the GRASS script from php since the server is not able to retrieve the HOME path of the user www-data. Therefore, we modified the setup.py script specifying the home path in a new variable:

add addons to the PATH

copied and simplified from lib/init/grass.py

if mswin:
config_dirname = “GRASS7”
config_dir = os.path.join(os.getenv(‘APPDATA’), config_dirname)
else:
user_home = ‘/home/ubuntu/’
config_dirname = “.grass7”
config_dir = os.path.join(user_home, config_dirname)

After these changes, we were able to run GRASS from a php web page but we got a new error using certain modules like v.in.ogr:

Check if OGR layer <sql_statement> contains polygons…
0%^H^H^H^H^H 33%^H^H^H^H^H 66%^H^H^H^H^H 100%^H^H^H^H^H
WARNING: Vector map already exists and will be overwritten
Creating attribute table for layer <sql_statement>…
ERROR: Unable to determine user’s home directory
Traceback (most recent call last):
File “path3d.py”, line 297, in
main()
File “path3d.py”, line 267, in main
gscript.run_command(‘v.in.ogr’, input=‘{}/tmp/output.gpkg’.format(outdir0), layer='sql_statement$
File “/usr/local/grass78/etc/python/grass/script/core.py”, line 441, in run_command
return handle_errors(returncode, returncode, args, kwargs)
File “/usr/local/grass78/etc/python/grass/script/core.py”, line 343, in handle_errors
returncode=returncode)

The error is always the same, the php web page is not able to retrieve the home path. We solved this error setting the server home path as HOME environment variable. Adding the following lines to our GRASS python script, everything works fine and we are now able to run the script from our php web page!

home = ‘/home/ubuntu/’
os.environ[‘HOME’] = home

Hope this can help in case anyone has the same problem!

Regarding the problem installing GRASS GIS from ubuntugis unstable, I don’t know exactly what te problem is, but using ubuntugis unstable caused some conflicts with Postgis and QGIS server packages respectively installed using https://wiki.postgresql.org/wiki/Apt and https://qgis.org/ubuntu-ltr

Thanks again for yout help and your suggestions!

Roberta

···

Da: Markus Neteler neteler@osgeo.org
Inviato: domenica 4 ottobre 2020 11:04
A: roberta fagandini robifagandini@hotmail.it
Cc: Moritz Lennert mlennert@club.worldonline.be; grass-user@lists.osgeo.org grass-user@lists.osgeo.org
Oggetto: Re: [GRASS-user] R: exwcuting grass python script from php web page

Hi Roberta,

On Wed, Sep 30, 2020 at 5:12 PM roberta fagandini
robifagandini@hotmail.it wrote:

On 28/09/20 18:37, roberta fagandini wrote:

On 28/09/20 17:51, roberta fagandini wrote:

Sorry, maybe I didn’t explain myself well because actually, I don’t get
any error from the php web page, simply the code related to grass is not
executed.For instance gsetup.init(gisbase, gisdb, location, mapset) does
not create any file in my /tmp/ folder and the print(gscript.gisenv())
is not printed.
I checked the import of the grass libraries (print(sys.modules)) and
they seem to be correctly imported.

Have you checked your webservers error logs ?

I have already checked and this is the error in the webserver log file

I observe that:

File “importgrass.py”, line 82, in
main()
File “importgrass.py”, line 71, in main
rcfile = gsetup.init(gisbase, gisdb, location, mapset)
File “/usr/lib/grass74/etc/python/grass/script/setup.py”, line 170,

… you use GRASS GIS 7.4? Pretty old and outdated.

in init
config_dir = os.path.join(os.getenv(‘HOME’), config_dirname)
File “/usr/lib/python3.6/posixpath.py”, line 80, in join

… you try to use GRASS GIS 7.4 with Python 3. For this you need
GRASS GIS 7.8 as already suggested.

a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType
…
I tried to upgrade GRASS using the repository ubuntugis unstable but it caused some problems with
the installation of postgis.

Which problems exactly?

$command = escapeshellcmd('/usr/bin/python3 importgrass.py ');

… Python 3, fine. But here:

I tried to run ‘/usr/bin/grass --version’ but I got a similar error:

File “/usr/bin/grass”, line 2005, in
main()
File “/usr/bin/grass”, line 1804, in main
grass_config_dir = get_grass_config_dir()
File “/usr/bin/grass”, line 398, in get_grass_config_dir
directory = os.path.join(os.getenv(‘HOME’), grass_config_dirname)
File “/usr/lib/python2.7/posixpath.py”, line 70, in join

… again Python 2.

This mixture will not work. Do you have by chance multiple GRASS GIS
versions installed?

I’d suggest removing all installed GRASS packages and reinstalling 7.8
cleanly again.

best
Markus

10

Hi Roberta,
Glad you got it working!

Would you mind adding your findings here?
https://grasswiki.osgeo.org/wiki/GRASS_and_PHP

Thanks,
Markus