[GRASS-user] compiling grass-7.8.5 on conda environment

GRASS GRASS user
, ,
1

Dear all,

I am trying to compile grass 7.8.5 in a conda environment (on centos7, managed HPC) adapting this instructions here.
conda create -y -n $conda_env python=3.8.5
conda activate $conda_env
conda install -c conda-forge geos gdal==3.3.1 -y
conda install -c conda-forge pdal fftw -y
conda install -c biobuilds libxml2
conda install -c conda-forge libiconv

compile works well. make gives a lot of errors. When I run make again in one of the folder, I get this errors:
/home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/libgrass_gis.7.8.so: undefined reference to libiconv' /home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/[libgrass_gis.7.8.so](http://libgrass_gis.7.8.so): undefined reference to libiconv_open’
/home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/libgrass_gis.7.8.so: undefined reference to `libiconv_close’

I found this answer, but I am not sure how to continue from there.
Thanks a lot
Laura

2

Hi Laura,

See whether the following is helpful to you. It uses everything from conda and has some local fixes for iconv. I didn’t test 7.8.5 specifically, only the 7.8 release branch.

GRASS GIS on HPC Henry2
https://github.com/ncsu-geoforall-lab/grass-gis-on-hpc-henry2/

Some more comments:

On Tue, Aug 3, 2021 at 9:12 AM Laura Poggio <laura.poggio@gmail.com> wrote:

I am trying to compile grass 7.8.5 in a conda environment (on centos7, managed HPC) adapting this instructions here.

Nobody touched that repo for a while, but development happened elsewhere. We have a CentOS 7 build partially using conda in the main repo’s CI. However, my experience was that the CentOS 7 Docker container in CI was very different from the CentOS 7 environment on HPC which has many customizations.

https://github.com/OSGeo/grass/blob/master/.github/workflows/centos.yml

There is also a conda-based build for macOS, but that would need to be adapted

conda create -y -n $conda_env python=3.8.5
conda activate $conda_env
conda install -c conda-forge geos gdal==3.3.1 -y
conda install -c conda-forge pdal fftw -y
conda install -c biobuilds libxml2
conda install -c conda-forge libiconv

The GRASS GIS on Henry 2 repo has an environment file you can use.

compile works well. make gives a lot of errors. When I run make again in one of the folder, I get this errors:
/home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/libgrass_gis.7.8.so: undefined reference to libiconv' /home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/[libgrass_gis.7.8.so](http://libgrass_gis.7.8.so): undefined reference to libiconv_open’
/home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/libgrass_gis.7.8.so: undefined reference to `libiconv_close’

I was not able to create a proper fix for GRASS GIS configuration, but the repo has a somewhat hacky patch applied locally which injects libiconv into more places.

Let me know how this goes. With the scripts in GRASS GIS on Henry 2, I can install new versions easily, but I would like to see it more streamlined with less local customizations.

Best,
Vaclav

I found this answer, but I am not sure how to continue from there.
Thanks a lot
Laura

grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

3

Hi Vaclav,
thanks a lot!
with the patch for the makefile in the repository I managed to solve the previous error. Now I am getting a different error apparently linked to some library conflict (same combination of libraries works in a different environment):

/home/user/conda3/envs/sg_py3_geo//lib//libspatialite.so.7: undefined reference to GEOSFrechetDistance' /home/user/conda3/envs/sg_py3_geo//lib//libspatialite.so.7: undefined reference to GEOSFrechetDistance_r’
/home/user/conda3/envs/sg_py3_geo//lib//libspatialite.so.7: undefined reference to GEOSFrechetDistanceDensify' /home/user/conda3/envs/sg_py3_geo//lib//libgdal.so.27: undefined reference to GEOSMakeValid_r’
/home/user/conda3/envs/sg_py3_geo//lib//libspatialite.so.7: undefined reference to `GEOSFrechetDistanceDensify_r’

Tomorrow I will try again with a clean conda environment implementing all of your scripts.
Thanks a lot!

Laura

On Tue, 3 Aug 2021 at 15:32, Vaclav Petras <wenzeslaus@gmail.com> wrote:

Hi Laura,

See whether the following is helpful to you. It uses everything from conda and has some local fixes for iconv. I didn’t test 7.8.5 specifically, only the 7.8 release branch.

GRASS GIS on HPC Henry2
https://github.com/ncsu-geoforall-lab/grass-gis-on-hpc-henry2/

Some more comments:

On Tue, Aug 3, 2021 at 9:12 AM Laura Poggio <laura.poggio@gmail.com> wrote:

I am trying to compile grass 7.8.5 in a conda environment (on centos7, managed HPC) adapting this instructions here.

Nobody touched that repo for a while, but development happened elsewhere. We have a CentOS 7 build partially using conda in the main repo’s CI. However, my experience was that the CentOS 7 Docker container in CI was very different from the CentOS 7 environment on HPC which has many customizations.

https://github.com/OSGeo/grass/blob/master/.github/workflows/centos.yml

There is also a conda-based build for macOS, but that would need to be adapted

conda create -y -n $conda_env python=3.8.5
conda activate $conda_env
conda install -c conda-forge geos gdal==3.3.1 -y
conda install -c conda-forge pdal fftw -y
conda install -c biobuilds libxml2
conda install -c conda-forge libiconv

The GRASS GIS on Henry 2 repo has an environment file you can use.

compile works well. make gives a lot of errors. When I run make again in one of the folder, I get this errors:
/home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/libgrass_gis.7.8.so: undefined reference to libiconv' /home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/[libgrass_gis.7.8.so](http://libgrass_gis.7.8.so): undefined reference to libiconv_open’
/home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/libgrass_gis.7.8.so: undefined reference to `libiconv_close’

I was not able to create a proper fix for GRASS GIS configuration, but the repo has a somewhat hacky patch applied locally which injects libiconv into more places.

Let me know how this goes. With the scripts in GRASS GIS on Henry 2, I can install new versions easily, but I would like to see it more streamlined with less local customizations.

Best,
Vaclav

I found this answer, but I am not sure how to continue from there.
Thanks a lot
Laura

grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

4

Hi Vaclav,
thanks a lot. Your patch worked perfectly. The remaining problems were solved by loading the conda environment path into LD_LIBRARY.
I will look into your repository if there is a better solution. But at least the environment works and grass can be updated to 7.8 and python3.

Thanks a lot again
Laura

On Tue, 3 Aug 2021 at 17:21, Laura Poggio <laura.poggio@gmail.com> wrote:

Hi Vaclav,
thanks a lot!
with the patch for the makefile in the repository I managed to solve the previous error. Now I am getting a different error apparently linked to some library conflict (same combination of libraries works in a different environment):

/home/user/conda3/envs/sg_py3_geo//lib//libspatialite.so.7: undefined reference to GEOSFrechetDistance' /home/user/conda3/envs/sg_py3_geo//lib//libspatialite.so.7: undefined reference to GEOSFrechetDistance_r’
/home/user/conda3/envs/sg_py3_geo//lib//libspatialite.so.7: undefined reference to GEOSFrechetDistanceDensify' /home/user/conda3/envs/sg_py3_geo//lib//libgdal.so.27: undefined reference to GEOSMakeValid_r’
/home/user/conda3/envs/sg_py3_geo//lib//libspatialite.so.7: undefined reference to `GEOSFrechetDistanceDensify_r’

Tomorrow I will try again with a clean conda environment implementing all of your scripts.
Thanks a lot!

Laura

On Tue, 3 Aug 2021 at 15:32, Vaclav Petras <wenzeslaus@gmail.com> wrote:

Hi Laura,

See whether the following is helpful to you. It uses everything from conda and has some local fixes for iconv. I didn’t test 7.8.5 specifically, only the 7.8 release branch.

GRASS GIS on HPC Henry2
https://github.com/ncsu-geoforall-lab/grass-gis-on-hpc-henry2/

Some more comments:

On Tue, Aug 3, 2021 at 9:12 AM Laura Poggio <laura.poggio@gmail.com> wrote:

I am trying to compile grass 7.8.5 in a conda environment (on centos7, managed HPC) adapting this instructions here.

Nobody touched that repo for a while, but development happened elsewhere. We have a CentOS 7 build partially using conda in the main repo’s CI. However, my experience was that the CentOS 7 Docker container in CI was very different from the CentOS 7 environment on HPC which has many customizations.

https://github.com/OSGeo/grass/blob/master/.github/workflows/centos.yml

There is also a conda-based build for macOS, but that would need to be adapted

conda create -y -n $conda_env python=3.8.5
conda activate $conda_env
conda install -c conda-forge geos gdal==3.3.1 -y
conda install -c conda-forge pdal fftw -y
conda install -c biobuilds libxml2
conda install -c conda-forge libiconv

The GRASS GIS on Henry 2 repo has an environment file you can use.

compile works well. make gives a lot of errors. When I run make again in one of the folder, I get this errors:
/home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/libgrass_gis.7.8.so: undefined reference to libiconv' /home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/[libgrass_gis.7.8.so](http://libgrass_gis.7.8.so): undefined reference to libiconv_open’
/home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/libgrass_gis.7.8.so: undefined reference to `libiconv_close’

I was not able to create a proper fix for GRASS GIS configuration, but the repo has a somewhat hacky patch applied locally which injects libiconv into more places.

Let me know how this goes. With the scripts in GRASS GIS on Henry 2, I can install new versions easily, but I would like to see it more streamlined with less local customizations.

Best,
Vaclav

I found this answer, but I am not sure how to continue from there.
Thanks a lot
Laura

grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

5

On Fri, Aug 6, 2021 at 1:33 AM Laura Poggio <laura.poggio@gmail.com> wrote:

Hi Vaclav,
thanks a lot. Your patch worked perfectly. The remaining problems were solved by loading the conda environment path into LD_LIBRARY.

That’s what I do there.

I will look into your repository if there is a better solution.

Not a one I know of, but I wouldn’t mind using something else. I think the reason for this being needed during compilation is that the things in GRASS GIS are actually running during compilation (modules help to generate parts of documentation).

However, feel free to check my scripts there. It is a fully reproducible build workflow on our HPC, but maybe just a little adaptation is needed.

But at least the environment works and grass can be updated to 7.8 and python3.

The scripts can install multiple versions which is good just for updating to latest release as need while keeping the old versions around for reproducibility. We are using it to be able to use also the development versions and update them often without disturbing people who are using them.

Let me know if anything there needs more explanation.

Vaclav

Thanks a lot again
Laura

On Tue, 3 Aug 2021 at 17:21, Laura Poggio <laura.poggio@gmail.com> wrote:

Hi Vaclav,
thanks a lot!
with the patch for the makefile in the repository I managed to solve the previous error. Now I am getting a different error apparently linked to some library conflict (same combination of libraries works in a different environment):

/home/user/conda3/envs/sg_py3_geo//lib//libspatialite.so.7: undefined reference to GEOSFrechetDistance' /home/user/conda3/envs/sg_py3_geo//lib//libspatialite.so.7: undefined reference to GEOSFrechetDistance_r’
/home/user/conda3/envs/sg_py3_geo//lib//libspatialite.so.7: undefined reference to GEOSFrechetDistanceDensify' /home/user/conda3/envs/sg_py3_geo//lib//libgdal.so.27: undefined reference to GEOSMakeValid_r’
/home/user/conda3/envs/sg_py3_geo//lib//libspatialite.so.7: undefined reference to `GEOSFrechetDistanceDensify_r’

Tomorrow I will try again with a clean conda environment implementing all of your scripts.
Thanks a lot!

Laura

On Tue, 3 Aug 2021 at 15:32, Vaclav Petras <wenzeslaus@gmail.com> wrote:

Hi Laura,

See whether the following is helpful to you. It uses everything from conda and has some local fixes for iconv. I didn’t test 7.8.5 specifically, only the 7.8 release branch.

GRASS GIS on HPC Henry2
https://github.com/ncsu-geoforall-lab/grass-gis-on-hpc-henry2/

Some more comments:

On Tue, Aug 3, 2021 at 9:12 AM Laura Poggio <laura.poggio@gmail.com> wrote:

I am trying to compile grass 7.8.5 in a conda environment (on centos7, managed HPC) adapting this instructions here.

Nobody touched that repo for a while, but development happened elsewhere. We have a CentOS 7 build partially using conda in the main repo’s CI. However, my experience was that the CentOS 7 Docker container in CI was very different from the CentOS 7 environment on HPC which has many customizations.

https://github.com/OSGeo/grass/blob/master/.github/workflows/centos.yml

There is also a conda-based build for macOS, but that would need to be adapted

conda create -y -n $conda_env python=3.8.5
conda activate $conda_env
conda install -c conda-forge geos gdal==3.3.1 -y
conda install -c conda-forge pdal fftw -y
conda install -c biobuilds libxml2
conda install -c conda-forge libiconv

The GRASS GIS on Henry 2 repo has an environment file you can use.

compile works well. make gives a lot of errors. When I run make again in one of the folder, I get this errors:
/home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/libgrass_gis.7.8.so: undefined reference to libiconv' /home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/[libgrass_gis.7.8.so](http://libgrass_gis.7.8.so): undefined reference to libiconv_open’
/home/user/grasspy3/grass-7.8.5/dist.x86_64-pc-linux-gnu/lib/libgrass_gis.7.8.so: undefined reference to `libiconv_close’

I was not able to create a proper fix for GRASS GIS configuration, but the repo has a somewhat hacky patch applied locally which injects libiconv into more places.

Let me know how this goes. With the scripts in GRASS GIS on Henry 2, I can install new versions easily, but I would like to see it more streamlined with less local customizations.

Best,
Vaclav

I found this answer, but I am not sure how to continue from there.
Thanks a lot
Laura

grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user