Now that 7.8.7 has been released, I started looking into updating the Debian package to 8.0.x which requires rebuilding libgdal-grass and qgis.
Initial tests with libgdal-grass are not promising. Both gdalinfo and ogrinfo show errors because not all libraries are linked. We saw this too when 7.2 was released: https://trac.osgeo.org/gdal/ticket/6785
The attached patch helps to some extent, but does not resolve all issues. ogrinfo still shows errors:
Warning 1: GRASS warning: GISBASE environment variable was not set, using:
/usr/lib/grass80
/usr/lib/grass80/driver/db/sqlite: error while loading shared libraries: libgrass_dbmidriver.8.0.so: cannot open shared object file: No such file or directory
dbmi: Protocol error
Warning 1: GRASS warning: Unable to start driver <sqlite>
ERROR 1: Cannot open database sqlite by driver /tmp/spearfish60_grass7/PERMANENT/sqlite/sqlite.db, check if GISBASE environment variable is set, the driver is available and the database is accessible.
INFO: Open of `/tmp/spearfish60_grass7/PERMANENT/vector/roads/head'
using driver `OGR_GRASS' successful.
1: roads (Line String)
The grass drivers lacks the RPATH that was set for ogr_GRASS:
On Fri, Feb 25, 2022 at 7:24 AM Sebastiaan Couwenberg
<sebastic@xs4all.nl> wrote:
On 2/24/22 15:23, Vaclav Petras wrote:
> On Thu, 24 Feb 2022 at 03:33, Sebastiaan Couwenberg wrote:
>> Should we perhaps take this opportunity to move the grass libraries to
>> default library search paths as raised on the debian-gis list?
>>
>> https://lists.debian.org/debian-gis/2021/12/msg00023.html
>
> I vote yes. I think the reasons to do this are no longer valid. Doing it
> the standard way seems to me to be the best way forward. Any
> suggestions/PRs?
Not using --prefix=/usr/lib will violate FHS, so not an option for the
Debian package.
Are users of libgdal-grass expected to set LD_LIBRARY_PATH like the
grass executable does to find the grass libraries?
the creation of a LD_LIBRARY_PATH file is suggested:
/etc/ld.so.conf.d/grass.conf
That violates the Debian policy, and triggers a lintian error:
"
It is not okay to install libraries in a different directory and then
modify the run-time link path. Shared libraries should go into
/usr/lib. Alternatively, they can require binaries to set the RPATH to
find the library.
"
I vote yes. I think the reasons to do this are no longer valid. Doing it
the standard way seems to me to be the best way forward. Any
suggestions/PRs?
Not using --prefix=/usr/lib will violate FHS, so not an option for the
Debian package.
Using --prefix now results in every file from GRASS being installed into --prefix, so I assume the current --prefix behavior is wrong from Debian perspective, yes? Any suggestions on correct behavior?
I vote yes. I think the reasons to do this are no longer valid. Doing it
the standard way seems to me to be the best way forward. Any
suggestions/PRs?
Not using --prefix=/usr/lib will violate FHS, so not an option for the
Debian package.
Using --prefix now results in every file from GRASS being installed into
--prefix, so I assume the current --prefix behavior is wrong from Debian
perspective, yes? Any suggestions on correct behavior?
The 8.x prefix behavior is no different from 7.x.
If we want to stop using --prefix=/usr/lib and have FHS complianance while also having the shared libraries in the default library search path several changes in GRASS will be required.
The resulting structure should result in something like:
This assumes that the grass shared libraries should not be considered private which does seem to be the case with their use by libgdal-grass being the exception.
Alternatively the GRASS executables need to have RPATH set, e.g. with:
-Wl,-rpath,/usr/lib/grass80/lib
To remove the need for changing the library search path. This is the road of least resistance.
On Fri, 25 Feb 2022 at 09:07, Sebastiaan Couwenberg <sebastic@xs4all.nl> wrote:
If we want to stop using --prefix=/usr/lib and have FHS complianance
while also having the shared libraries in the default library search
path several changes in GRASS will be required.
The resulting structure should result in something like:
Makes sense. Any good examples of how to do this with Autotools, esp. given that macOS and Windows still need a single directory?
This assumes that the grass shared libraries should not be considered
private which does seem to be the case with their use by libgdal-grass
being the exception.
We keep a stable API anyway for custom user executables such as addons, so not private in that sense. Libraries can also be accessed from Python through an API which wraps ctypes calls.
I’m not sure what is the current situation in QGIS, but at least historically, that would be another exception, so perhaps not an exception at all, but a rule.
The executables like g.region are somewhat internal, so that’s perhaps the /usr/libexec/grass group.
Alternatively the GRASS executables need to have RPATH set, e.g. with:
-Wl,-rpath,/usr/lib/grass80/lib
I’m not sure what are the risks involved here. I have seen -rpath-link used in the code, so that’s not good enough.
To remove the need for changing the library search path. This is the
road of least resistance.
# ogrinfo -ro -so /tmp/spearfish60_grass7/PERMANENT/vector/roads/head
Warning 1: GRASS warning: GISBASE environment variable was not set, using:
/usr/lib/grass80
/usr/lib/grass80/driver/db/sqlite: error while loading shared libraries: libgrass_dbmidriver.8.0.so: cannot open shared object file: No such file or directory
dbmi: Protocol error
Warning 1: GRASS warning: Unable to start driver <sqlite>
ERROR 1: Cannot open database sqlite by driver /tmp/spearfish60_grass7/PERMANENT/sqlite/sqlite.db, check if GISBASE environment variable is set, the driver is available and the database is accessible.
INFO: Open of `/tmp/spearfish60_grass7/PERMANENT/vector/roads/head'
using driver `OGR_GRASS' successful.
1: roads (Line String)
FWIW grass (7.8.7-1) in unstable, grass (7.8.6-1) in testing, and grass (7.6.0-1) in buster have the same issue.
On 25 Feb 2022, at 15:43, Vaclav Petras <wenzeslaus@gmail.com> wrote:
On Fri, 25 Feb 2022 at 09:07, Sebastiaan Couwenberg <sebastic@xs4all.nl> wrote:
If we want to stop using --prefix=/usr/lib and have FHS complianance
while also having the shared libraries in the default library search
path several changes in GRASS will be required.
The resulting structure should result in something like:
Makes sense. Any good examples of how to do this with Autotools, esp. given that macOS and Windows still need a single directory?
I’d warmly support this new suggested structure. And the sooner the better! It would in general work fine on macOS too, only prefix need to be "/usr/local” (no need to put everything in on directory), and the Python package need an alternative path.
On Thu, Feb 24, 2022 at 9:33 AM Sebastiaan Couwenberg
<sebastic@xs4all.nl> wrote:
Now that 7.8.7 has been released, I started looking into updating the
Debian package to 8.0.x which requires rebuilding libgdal-grass and qgis.
Initial tests with libgdal-grass are not promising. Both gdalinfo and
ogrinfo show errors because not all libraries are linked. We saw this
too when 7.2 was released: https://trac.osgeo.org/gdal/ticket/6785
The attached patch helps to some extent, but does not resolve all
issues.
Warning 1: GRASS warning: GISBASE environment variable was not set, using:
/usr/lib/grass80
/usr/lib/grass80/driver/db/sqlite: error while loading shared libraries:
libgrass_dbmidriver.8.0.so: cannot open shared object file: No such file
or directory
dbmi: Protocol error
Warning 1: GRASS warning: Unable to start driver <sqlite>
ERROR 1: Cannot open database sqlite by driver
/tmp/spearfish60_grass7/PERMANENT/sqlite/sqlite.db, check if GISBASE
environment variable is set, the driver is available and the database
is accessible.
INFO: Open of `/tmp/spearfish60_grass7/PERMANENT/vector/roads/head'
using driver `OGR_GRASS' successful.
1: roads (Line String)
The grass drivers lacks the RPATH that was set for ogr_GRASS:
On Thu, Feb 24, 2022 at 9:33 AM Sebastiaan Couwenberg
<sebastic@xs4all.nl> wrote:
Now that 7.8.7 has been released, I started looking into updating the
Debian package to 8.0.x which requires rebuilding libgdal-grass and qgis.
Initial tests with libgdal-grass are not promising. Both gdalinfo and
ogrinfo show errors because not all libraries are linked. We saw this
too when 7.2 was released: https://trac.osgeo.org/gdal/ticket/6785
The attached patch helps to some extent, but does not resolve all
issues.
Did you propose the path to GDAL in GitHub yet?
No, there was no need after all.
I was led astray by the error from the grass sqlite driver which doesn't need changes in gdal-grass.
libgdal-grass and qgis built with grass 8 have only just been available in unstable.
The transition to proj 9 is currently ongoing so and qgis hasn't been rebuilt yet, but gdal has, so qgis isn't very usualy right now with dependencies on different versions of proj.
Should we perhaps take this opportunity to move the grass libraries to
default library search paths as raised on the debian-gis list?