[GRASS-dev] Compilation options

GRASS GRASS developer
1

Dear all,

Does someone of you know which modules benefit from --with-openmp and --with-opencl compilation options?

I found this explanation of compilation options: https://grass.osgeo.org/grass72/source/snapshot/REQUIREMENTS.html

But it does not cover OpenCL or OpenMP (as well as some few more other option).

If I wanted to find out myself, could I just search the code for modules that import the relevant headers of the respective libraries?

Or is that a bit more complex?

Another question is if it would be recommended to use the different options, as I understood e.g. using --with-pthread is not necessarily a good choice…

Thanks in advance.

Cheers

Stefan

2

Hi again,

Seek and ye shall find…

So, there is:

https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7 and

https://grasswiki.osgeo.org/wiki/GPU

Which helped me a bit further.

However, my impression is that not all info there (esp. on OpenMP) is up to date (e.g. latest development like here: https://lists.osgeo.org/pipermail/grass-dev/2017-August/085711.html). Neither did it became very clear to me which modules already have robust and working support and which are just candidates or where initial implementations are available (sometimes outside the code base)…

Cheers

Stefan

···

From: grass-dev [mailto:grass-dev-bounces@lists.osgeo.org] On Behalf Of Stefan Blumentrath
Sent: mandag 23. oktober 2017 09.24
To: GRASS developers list (grass-dev@lists.osgeo.org) grass-dev@lists.osgeo.org
Subject: [GRASS-dev] Compilation options

Dear all,

Does someone of you know which modules benefit from --with-openmp and --with-opencl compilation options?

I found this explanation of compilation options: https://grass.osgeo.org/grass72/source/snapshot/REQUIREMENTS.html

But it does not cover OpenCL or OpenMP (as well as some few more other option).

If I wanted to find out myself, could I just search the code for modules that import the relevant headers of the respective libraries?

Or is that a bit more complex?

Another question is if it would be recommended to use the different options, as I understood e.g. using --with-pthread is not necessarily a good choice…

Thanks in advance.

Cheers

Stefan

3

On Tue, Oct 24, 2017 at 10:27 AM, Stefan Blumentrath
<Stefan.Blumentrath@nina.no> wrote:

Hi again,

Seek and ye shall find…

So, there is:

https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7 and

https://grasswiki.osgeo.org/wiki/GPU

Which helped me a bit further.

However, my impression is that not all info there (esp. on OpenMP) is up to
date (e.g. latest development like here:
https://lists.osgeo.org/pipermail/grass-dev/2017-August/085711.html).
Neither did it became very clear to me which modules already have robust and
working support and which are just candidates or where initial
implementations are available (sometimes outside the code base)…

I updated it slightly to reflect the new modules, as always, more work
is needed on that page.
https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7

Anna

Cheers

Stefan

From: grass-dev [mailto:grass-dev-bounces@lists.osgeo.org] On Behalf Of
Stefan Blumentrath
Sent: mandag 23. oktober 2017 09.24
To: GRASS developers list (grass-dev@lists.osgeo.org)
<grass-dev@lists.osgeo.org>
Subject: [GRASS-dev] Compilation options

Dear all,

Does someone of you know which modules benefit from --with-openmp and
--with-opencl compilation options?

I found this explanation of compilation options:
https://grass.osgeo.org/grass72/source/snapshot/REQUIREMENTS.html

But it does not cover OpenCL or OpenMP (as well as some few more other
option).

If I wanted to find out myself, could I just search the code for modules
that import the relevant headers of the respective libraries?

Or is that a bit more complex?

Another question is if it would be recommended to use the different options,
as I understood e.g. using --with-pthread is not necessarily a good choice…

Thanks in advance.

Cheers

Stefan

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

4

Hei Anna,

Thanks for looking into this.

I did some tests with GRASS 7.2.1svn (r70188) and GRASS 7.3.svn (r70829) both compiled with --with-openmp

It seems that r.proj does not use multiple threads in GRASS 7.2 (not tested in 7.3. yet). v.surf.rst uses more cores in GRASS 7.2 with OpenMP but that does not bring down execution time at all. OpenMP seems to be disabled for v.surf.rst in GRASS 7.3. v.surf.bspline however, is significantly faster in GRASS 7.3 with OpenMP...

I will do some more, systematic tests and update the wiki entry as far as I can, if not someone with knowledge gets there first...

Cheers
Stefan

P.S.: Some preliminary test results:
# OpenMP support test
$ OMP_NUM_THREADS=10
$ export OMP_NUM_THREADS
$ time v.surf.rst input=points elevation=test --o --qq

real 51m42.224s
user 398m2.766s
sys 0m18.874s

$ OMP_NUM_THREADS=1
$ export OMP_NUM_THREADS
$ time v.surf.rst input=points elevation=test --o --qq

real 53m22.508s
user 53m10.607s
sys 0m4.793s

$ OMP_NUM_THREADS=10
$ export OMP_NUM_THREADS
$ time v.surf.bspline elevrand column=value output=elev_full --o
real 0m20.509s
user 3m11.412s
sys 0m1.963s

$ OMP_NUM_THREADS=1
$ export OMP_NUM_THREADS
$ time v.surf.bspline elevrand column=value output=elev_full --o
real 1m15.434s
user 1m14.738s
sys 0m0.447s

-----Original Message-----
From: Anna Petrášová [mailto:kratochanna@gmail.com]
Sent: fredag 27. oktober 2017 17.00
To: Stefan Blumentrath <Stefan.Blumentrath@nina.no>
Cc: GRASS developers list (grass-dev@lists.osgeo.org) <grass-dev@lists.osgeo.org>
Subject: Re: [GRASS-dev] Compilation options

On Tue, Oct 24, 2017 at 10:27 AM, Stefan Blumentrath <Stefan.Blumentrath@nina.no> wrote:

Hi again,

Seek and ye shall find…

So, there is:

https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7 and

https://grasswiki.osgeo.org/wiki/GPU

Which helped me a bit further.

However, my impression is that not all info there (esp. on OpenMP) is
up to date (e.g. latest development like here:
https://lists.osgeo.org/pipermail/grass-dev/2017-August/085711.html).
Neither did it became very clear to me which modules already have
robust and working support and which are just candidates or where
initial implementations are available (sometimes outside the code
base)…

I updated it slightly to reflect the new modules, as always, more work is needed on that page.
https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7

Anna

Cheers

Stefan

From: grass-dev [mailto:grass-dev-bounces@lists.osgeo.org] On Behalf
Of Stefan Blumentrath
Sent: mandag 23. oktober 2017 09.24
To: GRASS developers list (grass-dev@lists.osgeo.org)
<grass-dev@lists.osgeo.org>
Subject: [GRASS-dev] Compilation options

Dear all,

Does someone of you know which modules benefit from --with-openmp and
--with-opencl compilation options?

I found this explanation of compilation options:
https://grass.osgeo.org/grass72/source/snapshot/REQUIREMENTS.html

But it does not cover OpenCL or OpenMP (as well as some few more other
option).

If I wanted to find out myself, could I just search the code for
modules that import the relevant headers of the respective libraries?

Or is that a bit more complex?

Another question is if it would be recommended to use the different
options, as I understood e.g. using --with-pthread is not necessarily
a good choice…

Thanks in advance.

Cheers

Stefan

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

5

v.surf.rst in trunk has nproc parameter, have you tried to use it?

Anna

···

On Oct 29, 2017 6:51 AM, “Stefan Blumentrath” <Stefan.Blumentrath@nina.no> wrote:

Hei Anna,

Thanks for looking into this.

I did some tests with GRASS 7.2.1svn (r70188) and GRASS 7.3.svn (r70829) both compiled with --with-openmp

It seems that r.proj does not use multiple threads in GRASS 7.2 (not tested in 7.3. yet). v.surf.rst uses more cores in GRASS 7.2 with OpenMP but that does not bring down execution time at all. OpenMP seems to be disabled for v.surf.rst in GRASS 7.3. v.surf.bspline however, is significantly faster in GRASS 7.3 with OpenMP…

I will do some more, systematic tests and update the wiki entry as far as I can, if not someone with knowledge gets there first…

Cheers
Stefan

P.S.: Some preliminary test results:

OpenMP support test

$ OMP_NUM_THREADS=10
$ export OMP_NUM_THREADS
$ time v.surf.rst input=points elevation=test --o --qq

real 51m42.224s
user 398m2.766s
sys 0m18.874s

$ OMP_NUM_THREADS=1
$ export OMP_NUM_THREADS
$ time v.surf.rst input=points elevation=test --o --qq

real 53m22.508s
user 53m10.607s
sys 0m4.793s

$ OMP_NUM_THREADS=10
$ export OMP_NUM_THREADS
$ time v.surf.bspline elevrand column=value output=elev_full --o
real 0m20.509s
user 3m11.412s
sys 0m1.963s

$ OMP_NUM_THREADS=1
$ export OMP_NUM_THREADS
$ time v.surf.bspline elevrand column=value output=elev_full --o
real 1m15.434s
user 1m14.738s
sys 0m0.447s

-----Original Message-----
From: Anna Petrášová [mailto:kratochanna@gmail.com]
Sent: fredag 27. oktober 2017 17.00
To: Stefan Blumentrath <Stefan.Blumentrath@nina.no>
Cc: GRASS developers list (grass-dev@lists.osgeo.org) <grass-dev@lists.osgeo.org>
Subject: Re: [GRASS-dev] Compilation options

On Tue, Oct 24, 2017 at 10:27 AM, Stefan Blumentrath <Stefan.Blumentrath@nina.no> wrote:

Hi again,

Seek and ye shall find…

So, there is:

https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7 and

https://grasswiki.osgeo.org/wiki/GPU

Which helped me a bit further.

However, my impression is that not all info there (esp. on OpenMP) is
up to date (e.g. latest development like here:
https://lists.osgeo.org/pipermail/grass-dev/2017-August/085711.html).
Neither did it became very clear to me which modules already have
robust and working support and which are just candidates or where
initial implementations are available (sometimes outside the code
base)…

I updated it slightly to reflect the new modules, as always, more work is needed on that page.
https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7

Anna

Cheers

Stefan

From: grass-dev [mailto:grass-dev-bounces@lists.osgeo.org] On Behalf
Of Stefan Blumentrath
Sent: mandag 23. oktober 2017 09.24
To: GRASS developers list (grass-dev@lists.osgeo.org)
<grass-dev@lists.osgeo.org>
Subject: [GRASS-dev] Compilation options

Dear all,

Does someone of you know which modules benefit from --with-openmp and
–with-opencl compilation options?

I found this explanation of compilation options:
https://grass.osgeo.org/grass72/source/snapshot/REQUIREMENTS.html

But it does not cover OpenCL or OpenMP (as well as some few more other
option).

If I wanted to find out myself, could I just search the code for
modules that import the relevant headers of the respective libraries?

Or is that a bit more complex?

Another question is if it would be recommended to use the different
options, as I understood e.g. using --with-pthread is not necessarily
a good choice…

Thanks in advance.

Cheers

Stefan

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

6

Thanks for the hint.

I just tried:

time v.surf.rst elevrand zcol=value elevation=elev_full --o nprocs=10

Reading features from vector map …

Processing segments in parallel…

Starting parallel work

v.surf.rst complete.

real 0m33.814s

user 0m35.786s

sys 0m0.126s

Seems, that also with nprocs=10 and OMP_NUM_THREADS=10 v.surf.rst only uses one CPU in GRASS 7.3.svn (despite the message about parallel work)…

Cheers

Stefan

···

v.surf.rst in trunk has nproc parameter, have you tried to use it?

Anna

On Oct 29, 2017 6:51 AM, “Stefan Blumentrath” <Stefan.Blumentrath@nina.no> wrote:

Hei Anna,

Thanks for looking into this.

I did some tests with GRASS 7.2.1svn (r70188) and GRASS 7.3.svn (r70829) both compiled with --with-openmp

It seems that r.proj does not use multiple threads in GRASS 7.2 (not tested in 7.3. yet). v.surf.rst uses more cores in GRASS 7.2 with OpenMP but that does not bring down execution time at all. OpenMP seems to be disabled for v.surf.rst in GRASS 7.3. v.surf.bspline however, is significantly faster in GRASS 7.3 with OpenMP…

I will do some more, systematic tests and update the wiki entry as far as I can, if not someone with knowledge gets there first…

Cheers
Stefan

P.S.: Some preliminary test results:

OpenMP support test

$ OMP_NUM_THREADS=10
$ export OMP_NUM_THREADS
$ time v.surf.rst input=points elevation=test --o --qq

real 51m42.224s
user 398m2.766s
sys 0m18.874s

$ OMP_NUM_THREADS=1
$ export OMP_NUM_THREADS
$ time v.surf.rst input=points elevation=test --o --qq

real 53m22.508s
user 53m10.607s
sys 0m4.793s

$ OMP_NUM_THREADS=10
$ export OMP_NUM_THREADS
$ time v.surf.bspline elevrand column=value output=elev_full --o
real 0m20.509s
user 3m11.412s
sys 0m1.963s

$ OMP_NUM_THREADS=1
$ export OMP_NUM_THREADS
$ time v.surf.bspline elevrand column=value output=elev_full --o
real 1m15.434s
user 1m14.738s
sys 0m0.447s

-----Original Message-----
From: Anna Petrášová [mailto:kratochanna@gmail.com]
Sent: fredag 27. oktober 2017 17.00
To: Stefan Blumentrath <Stefan.Blumentrath@nina.no>
Cc: GRASS developers list (grass-dev@lists.osgeo.org) <grass-dev@lists.osgeo.org>
Subject: Re: [GRASS-dev] Compilation options

On Tue, Oct 24, 2017 at 10:27 AM, Stefan Blumentrath <Stefan.Blumentrath@nina.no> wrote:

Hi again,

Seek and ye shall find…

So, there is:

https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7 and

https://grasswiki.osgeo.org/wiki/GPU

Which helped me a bit further.

However, my impression is that not all info there (esp. on OpenMP) is
up to date (e.g. latest development like here:
https://lists.osgeo.org/pipermail/grass-dev/2017-August/085711.html).
Neither did it became very clear to me which modules already have
robust and working support and which are just candidates or where
initial implementations are available (sometimes outside the code
base)…

I updated it slightly to reflect the new modules, as always, more work is needed on that page.
https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7

Anna

Cheers

Stefan

From: grass-dev [mailto:grass-dev-bounces@lists.osgeo.org] On Behalf
Of Stefan Blumentrath
Sent: mandag 23. oktober 2017 09.24
To: GRASS developers list (grass-dev@lists.osgeo.org)
<grass-dev@lists.osgeo.org>
Subject: [GRASS-dev] Compilation options

Dear all,

Does someone of you know which modules benefit from --with-openmp and
–with-opencl compilation options?

I found this explanation of compilation options:
https://grass.osgeo.org/grass72/source/snapshot/REQUIREMENTS.html

But it does not cover OpenCL or OpenMP (as well as some few more other
option).

If I wanted to find out myself, could I just search the code for
modules that import the relevant headers of the respective libraries?

Or is that a bit more complex?

Another question is if it would be recommended to use the different
options, as I understood e.g. using --with-pthread is not necessarily
a good choice…

Thanks in advance.

Cheers

Stefan

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

7

BTW: Would be nice to unify that in the long run, as a nprocs option would make parallelisation of a module also more evident and more convenient to control, compared to environment variables. Though it might be misleading if GRASS is not complied with OpenMP support (don`t know if that is the case in standard packages (OSGeo4W, UbuntuGIS…)).

There is no standard parser option for the number of parallel processes. Temporal modules use nprocs option as well, so maybe that can be considered a de facto standard (and possibly added to the parser?).

r.sun.mp calls it “threads” and other addons might again use yet other terms…

Just a thought, I can open a ticket if you think it is worth it…

Cheers

Stefan

···

v.surf.rst in trunk has nproc parameter, have you tried to use it?

Anna

On Oct 29, 2017 6:51 AM, “Stefan Blumentrath” <Stefan.Blumentrath@nina.no> wrote:

Hei Anna,

Thanks for looking into this.

I did some tests with GRASS 7.2.1svn (r70188) and GRASS 7.3.svn (r70829) both compiled with --with-openmp

It seems that r.proj does not use multiple threads in GRASS 7.2 (not tested in 7.3. yet). v.surf.rst uses more cores in GRASS 7.2 with OpenMP but that does not bring down execution time at all. OpenMP seems to be disabled for v.surf.rst in GRASS 7.3. v.surf.bspline however, is significantly faster in GRASS 7.3 with OpenMP…

I will do some more, systematic tests and update the wiki entry as far as I can, if not someone with knowledge gets there first…

Cheers
Stefan

P.S.: Some preliminary test results:

OpenMP support test

$ OMP_NUM_THREADS=10
$ export OMP_NUM_THREADS
$ time v.surf.rst input=points elevation=test --o --qq

real 51m42.224s
user 398m2.766s
sys 0m18.874s

$ OMP_NUM_THREADS=1
$ export OMP_NUM_THREADS
$ time v.surf.rst input=points elevation=test --o --qq

real 53m22.508s
user 53m10.607s
sys 0m4.793s

$ OMP_NUM_THREADS=10
$ export OMP_NUM_THREADS
$ time v.surf.bspline elevrand column=value output=elev_full --o
real 0m20.509s
user 3m11.412s
sys 0m1.963s

$ OMP_NUM_THREADS=1
$ export OMP_NUM_THREADS
$ time v.surf.bspline elevrand column=value output=elev_full --o
real 1m15.434s
user 1m14.738s
sys 0m0.447s

-----Original Message-----
From: Anna Petrášová [mailto:kratochanna@gmail.com]
Sent: fredag 27. oktober 2017 17.00
To: Stefan Blumentrath <Stefan.Blumentrath@nina.no>
Cc: GRASS developers list (grass-dev@lists.osgeo.org) <grass-dev@lists.osgeo.org>
Subject: Re: [GRASS-dev] Compilation options

On Tue, Oct 24, 2017 at 10:27 AM, Stefan Blumentrath <Stefan.Blumentrath@nina.no> wrote:

Hi again,

Seek and ye shall find…

So, there is:

https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7 and

https://grasswiki.osgeo.org/wiki/GPU

Which helped me a bit further.

However, my impression is that not all info there (esp. on OpenMP) is
up to date (e.g. latest development like here:
https://lists.osgeo.org/pipermail/grass-dev/2017-August/085711.html).
Neither did it became very clear to me which modules already have
robust and working support and which are just candidates or where
initial implementations are available (sometimes outside the code
base)…

I updated it slightly to reflect the new modules, as always, more work is needed on that page.
https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7

Anna

Cheers

Stefan

From: grass-dev [mailto:grass-dev-bounces@lists.osgeo.org] On Behalf
Of Stefan Blumentrath
Sent: mandag 23. oktober 2017 09.24
To: GRASS developers list (grass-dev@lists.osgeo.org)
<grass-dev@lists.osgeo.org>
Subject: [GRASS-dev] Compilation options

Dear all,

Does someone of you know which modules benefit from --with-openmp and
–with-opencl compilation options?

I found this explanation of compilation options:
https://grass.osgeo.org/grass72/source/snapshot/REQUIREMENTS.html

But it does not cover OpenCL or OpenMP (as well as some few more other
option).

If I wanted to find out myself, could I just search the code for
modules that import the relevant headers of the respective libraries?

Or is that a bit more complex?

Another question is if it would be recommended to use the different
options, as I understood e.g. using --with-pthread is not necessarily
a good choice…

Thanks in advance.

Cheers

Stefan

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

8

On Sun, Oct 29, 2017 at 4:30 PM, Stefan Blumentrath
<Stefan.Blumentrath@nina.no> wrote:

Thanks for the hint.

I just tried:

time v.surf.rst elevrand zcol=value elevation=elev_full --o nprocs=10

Reading features from vector map ...

Processing segments in parallel...

Starting parallel work

v.surf.rst complete.

real 0m33.814s

user 0m35.786s

sys 0m0.126s

Seems, that also with nprocs=10 and OMP_NUM_THREADS=10 v.surf.rst only uses
one CPU in GRASS 7.3.svn (despite the message about parallel work)…

For me it works, try different dataset, you might see more difference
for larger datasets maybe:

GRASS 7.3.svn (nc_spm_08_coastal):~/dev/grass/trunk2 > time v.surf.rst
--overwrite input=elev_lidrural_mrpts@PERMANENT elevation=test
nprocs=4
Reading features from vector map ...
WARNING: Some points outside of region (ignored)
WARNING: There are points outside specified 2D/3D region - 9663 points
         ignored
Ignoring 511 points (too dense)
Processing segments in parallel...
Starting parallel work
WARNING: Overshoot - increase in tension suggested. Overshoot occurs at
         (662,749) cell. Z-value 98.513438, zmin 103.513005, zmax
         153.065828.
v.surf.rst complete.

real 0m58.009s
user 3m41.476s
sys 0m0.304s

Cheers

Stefan

From: Anna Petrášová [mailto:kratochanna@gmail.com]
Sent: søndag 29. oktober 2017 12.31
To: Stefan Blumentrath <Stefan.Blumentrath@nina.no>
Cc: GRASS-dev <grass-dev@lists.osgeo.org>
Subject: RE: [GRASS-dev] Compilation options

v.surf.rst in trunk has nproc parameter, have you tried to use it?

Anna

On Oct 29, 2017 6:51 AM, "Stefan Blumentrath" <Stefan.Blumentrath@nina.no>
wrote:

Hei Anna,

Thanks for looking into this.

I did some tests with GRASS 7.2.1svn (r70188) and GRASS 7.3.svn (r70829)
both compiled with --with-openmp

It seems that r.proj does not use multiple threads in GRASS 7.2 (not tested
in 7.3. yet). v.surf.rst uses more cores in GRASS 7.2 with OpenMP but that
does not bring down execution time at all. OpenMP seems to be disabled for
v.surf.rst in GRASS 7.3. v.surf.bspline however, is significantly faster in
GRASS 7.3 with OpenMP...

I will do some more, systematic tests and update the wiki entry as far as I
can, if not someone with knowledge gets there first...

Cheers
Stefan

P.S.: Some preliminary test results:
# OpenMP support test
$ OMP_NUM_THREADS=10
$ export OMP_NUM_THREADS
$ time v.surf.rst input=points elevation=test --o --qq

real 51m42.224s
user 398m2.766s
sys 0m18.874s

$ OMP_NUM_THREADS=1
$ export OMP_NUM_THREADS
$ time v.surf.rst input=points elevation=test --o --qq

real 53m22.508s
user 53m10.607s
sys 0m4.793s

$ OMP_NUM_THREADS=10
$ export OMP_NUM_THREADS
$ time v.surf.bspline elevrand column=value output=elev_full --o
real 0m20.509s
user 3m11.412s
sys 0m1.963s

$ OMP_NUM_THREADS=1
$ export OMP_NUM_THREADS
$ time v.surf.bspline elevrand column=value output=elev_full --o
real 1m15.434s
user 1m14.738s
sys 0m0.447s

-----Original Message-----
From: Anna Petrášová [mailto:kratochanna@gmail.com]
Sent: fredag 27. oktober 2017 17.00
To: Stefan Blumentrath <Stefan.Blumentrath@nina.no>
Cc: GRASS developers list (grass-dev@lists.osgeo.org)
<grass-dev@lists.osgeo.org>
Subject: Re: [GRASS-dev] Compilation options

On Tue, Oct 24, 2017 at 10:27 AM, Stefan Blumentrath
<Stefan.Blumentrath@nina.no> wrote:

Hi again,

Seek and ye shall find…

So, there is:

https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7 and

https://grasswiki.osgeo.org/wiki/GPU

Which helped me a bit further.

However, my impression is that not all info there (esp. on OpenMP) is
up to date (e.g. latest development like here:
https://lists.osgeo.org/pipermail/grass-dev/2017-August/085711.html).
Neither did it became very clear to me which modules already have
robust and working support and which are just candidates or where
initial implementations are available (sometimes outside the code
base)…

I updated it slightly to reflect the new modules, as always, more work is
needed on that page.
https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7

Anna

Cheers

Stefan

From: grass-dev [mailto:grass-dev-bounces@lists.osgeo.org] On Behalf
Of Stefan Blumentrath
Sent: mandag 23. oktober 2017 09.24
To: GRASS developers list (grass-dev@lists.osgeo.org)
<grass-dev@lists.osgeo.org>
Subject: [GRASS-dev] Compilation options

Dear all,

Does someone of you know which modules benefit from --with-openmp and
--with-opencl compilation options?

I found this explanation of compilation options:
https://grass.osgeo.org/grass72/source/snapshot/REQUIREMENTS.html

But it does not cover OpenCL or OpenMP (as well as some few more other
option).

If I wanted to find out myself, could I just search the code for
modules that import the relevant headers of the respective libraries?

Or is that a bit more complex?

Another question is if it would be recommended to use the different
options, as I understood e.g. using --with-pthread is not necessarily
a good choice…

Thanks in advance.

Cheers

Stefan

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

9

Thanks Anna,

Yes, now I can confirm that it works. The problem was that I did not adjust the region to where the points are which influenced computation so OpenMP did not kick-in.

So now:
g.region -p vector=elev_lidrural_mrpts res=1 -a
time v.surf.rst --overwrite input=elev_lidrural_mrpts@PERMANENT elevation=test nprocs=10
Reading features from vector map ...
WARNING: Strip exists with insufficient data
Ignoring 701 points (too dense)
Processing segments in parallel...
Starting parallel work
WARNING: Overshoot - increase in tension suggested. Overshoot occurs at
         (85,878) cell. Z-value 158.741920, zmin 102.227686, zmax
         153.065828.
100%
v.surf.rst complete.

real 0m19.670s
user 2m19.833s
sys 0m0.446s

Compared to:
time v.surf.rst --overwrite input=elev_lidrural_mrpts@PERMANENT elevation=test nprocs=1
(...)
real 2m0.889s
user 2m0.332s
sys 0m0.251s

Cool!

Cheers
Stefan

P.S.: using /usr/bin/time -v it seems that RAM usage increased by a factor of ~2.25 between 1 and 10 cores...

-----Original Message-----
From: Anna Petrášová [mailto:kratochanna@gmail.com]
Sent: søndag 29. oktober 2017 23.00
To: Stefan Blumentrath <Stefan.Blumentrath@nina.no>
Cc: GRASS-dev <grass-dev@lists.osgeo.org>
Subject: Re: [GRASS-dev] Compilation options

On Sun, Oct 29, 2017 at 4:30 PM, Stefan Blumentrath <Stefan.Blumentrath@nina.no> wrote:

Thanks for the hint.

I just tried:

time v.surf.rst elevrand zcol=value elevation=elev_full --o nprocs=10

Reading features from vector map ...

Processing segments in parallel...

Starting parallel work

v.surf.rst complete.

real 0m33.814s

user 0m35.786s

sys 0m0.126s

Seems, that also with nprocs=10 and OMP_NUM_THREADS=10 v.surf.rst only
uses one CPU in GRASS 7.3.svn (despite the message about parallel
work)…

For me it works, try different dataset, you might see more difference for larger datasets maybe:

GRASS 7.3.svn (nc_spm_08_coastal):~/dev/grass/trunk2 > time v.surf.rst --overwrite input=elev_lidrural_mrpts@PERMANENT elevation=test
nprocs=4
Reading features from vector map ...
WARNING: Some points outside of region (ignored)
WARNING: There are points outside specified 2D/3D region - 9663 points
         ignored
Ignoring 511 points (too dense)
Processing segments in parallel...
Starting parallel work
WARNING: Overshoot - increase in tension suggested. Overshoot occurs at
         (662,749) cell. Z-value 98.513438, zmin 103.513005, zmax
         153.065828.
v.surf.rst complete.

real 0m58.009s
user 3m41.476s
sys 0m0.304s

Cheers

Stefan

From: Anna Petrášová [mailto:kratochanna@gmail.com]
Sent: søndag 29. oktober 2017 12.31
To: Stefan Blumentrath <Stefan.Blumentrath@nina.no>
Cc: GRASS-dev <grass-dev@lists.osgeo.org>
Subject: RE: [GRASS-dev] Compilation options

v.surf.rst in trunk has nproc parameter, have you tried to use it?

Anna

On Oct 29, 2017 6:51 AM, "Stefan Blumentrath"
<Stefan.Blumentrath@nina.no>
wrote:

Hei Anna,

Thanks for looking into this.

I did some tests with GRASS 7.2.1svn (r70188) and GRASS 7.3.svn
(r70829) both compiled with --with-openmp

It seems that r.proj does not use multiple threads in GRASS 7.2 (not
tested in 7.3. yet). v.surf.rst uses more cores in GRASS 7.2 with
OpenMP but that does not bring down execution time at all. OpenMP
seems to be disabled for v.surf.rst in GRASS 7.3. v.surf.bspline
however, is significantly faster in GRASS 7.3 with OpenMP...

I will do some more, systematic tests and update the wiki entry as far
as I can, if not someone with knowledge gets there first...

Cheers
Stefan

P.S.: Some preliminary test results:
# OpenMP support test
$ OMP_NUM_THREADS=10
$ export OMP_NUM_THREADS
$ time v.surf.rst input=points elevation=test --o --qq

real 51m42.224s
user 398m2.766s
sys 0m18.874s

$ OMP_NUM_THREADS=1
$ export OMP_NUM_THREADS
$ time v.surf.rst input=points elevation=test --o --qq

real 53m22.508s
user 53m10.607s
sys 0m4.793s

$ OMP_NUM_THREADS=10
$ export OMP_NUM_THREADS
$ time v.surf.bspline elevrand column=value output=elev_full --o
real 0m20.509s
user 3m11.412s
sys 0m1.963s

$ OMP_NUM_THREADS=1
$ export OMP_NUM_THREADS
$ time v.surf.bspline elevrand column=value output=elev_full --o
real 1m15.434s
user 1m14.738s
sys 0m0.447s

-----Original Message-----
From: Anna Petrášová [mailto:kratochanna@gmail.com]
Sent: fredag 27. oktober 2017 17.00
To: Stefan Blumentrath <Stefan.Blumentrath@nina.no>
Cc: GRASS developers list (grass-dev@lists.osgeo.org)
<grass-dev@lists.osgeo.org>
Subject: Re: [GRASS-dev] Compilation options

On Tue, Oct 24, 2017 at 10:27 AM, Stefan Blumentrath
<Stefan.Blumentrath@nina.no> wrote:

Hi again,

Seek and ye shall find…

So, there is:

https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7 and

https://grasswiki.osgeo.org/wiki/GPU

Which helped me a bit further.

However, my impression is that not all info there (esp. on OpenMP) is
up to date (e.g. latest development like here:
https://lists.osgeo.org/pipermail/grass-dev/2017-August/085711.html).
Neither did it became very clear to me which modules already have
robust and working support and which are just candidates or where
initial implementations are available (sometimes outside the code
base)…

I updated it slightly to reflect the new modules, as always, more work
is needed on that page.
https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7

Anna

Cheers

Stefan

From: grass-dev [mailto:grass-dev-bounces@lists.osgeo.org] On Behalf
Of Stefan Blumentrath
Sent: mandag 23. oktober 2017 09.24
To: GRASS developers list (grass-dev@lists.osgeo.org)
<grass-dev@lists.osgeo.org>
Subject: [GRASS-dev] Compilation options

Dear all,

Does someone of you know which modules benefit from --with-openmp and
--with-opencl compilation options?

I found this explanation of compilation options:
https://grass.osgeo.org/grass72/source/snapshot/REQUIREMENTS.html

But it does not cover OpenCL or OpenMP (as well as some few more
other option).

If I wanted to find out myself, could I just search the code for
modules that import the relevant headers of the respective libraries?

Or is that a bit more complex?

Another question is if it would be recommended to use the different
options, as I understood e.g. using --with-pthread is not necessarily
a good choice…

Thanks in advance.

Cheers

Stefan

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

10

I just ported r.sun.mp from addons to r.sun, so you are welcome to test that as well.

Anna

···

On Oct 30, 2017 5:28 AM, “Stefan Blumentrath” <Stefan.Blumentrath@nina.no> wrote:

Thanks Anna,

Yes, now I can confirm that it works. The problem was that I did not adjust the region to where the points are which influenced computation so OpenMP did not kick-in.

So now:
g.region -p vector=elev_lidrural_mrpts res=1 -a
time v.surf.rst --overwrite input=elev_lidrural_mrpts@PERMANENT elevation=test nprocs=10

Reading features from vector map …

WARNING: Strip exists with insufficient data
Ignoring 701 points (too dense)

Processing segments in parallel…
Starting parallel work
WARNING: Overshoot - increase in tension suggested. Overshoot occurs at

(85,878) cell. Z-value 158.741920, zmin 102.227686, zmax
153.065828.
100%
v.surf.rst complete.

real 0m19.670s
user 2m19.833s
sys 0m0.446s

Compared to:
time v.surf.rst --overwrite input=elev_lidrural_mrpts@PERMANENT elevation=test nprocs=1
(…)
real 2m0.889s
user 2m0.332s
sys 0m0.251s

Cool!

Cheers
Stefan

P.S.: using /usr/bin/time -v it seems that RAM usage increased by a factor of ~2.25 between 1 and 10 cores…

-----Original Message-----
From: Anna Petrášová [mailto:kratochanna@gmail.com]

Sent: søndag 29. oktober 2017 23.00
To: Stefan Blumentrath <Stefan.Blumentrath@nina.no>
Cc: GRASS-dev <grass-dev@lists.osgeo.org>

Subject: Re: [GRASS-dev] Compilation options

On Sun, Oct 29, 2017 at 4:30 PM, Stefan Blumentrath <Stefan.Blumentrath@nina.no> wrote:

Thanks for the hint.

I just tried:

time v.surf.rst elevrand zcol=value elevation=elev_full --o nprocs=10

Reading features from vector map …

Processing segments in parallel…

Starting parallel work

v.surf.rst complete.

real 0m33.814s

user 0m35.786s

sys 0m0.126s

Seems, that also with nprocs=10 and OMP_NUM_THREADS=10 v.surf.rst only
uses one CPU in GRASS 7.3.svn (despite the message about parallel
work)…

For me it works, try different dataset, you might see more difference for larger datasets maybe:

GRASS 7.3.svn (nc_spm_08_coastal):~/dev/grass/trunk2 > time v.surf.rst --overwrite input=elev_lidrural_mrpts@PERMANENT elevation=test
nprocs=4
Reading features from vector map …
WARNING: Some points outside of region (ignored)
WARNING: There are points outside specified 2D/3D region - 9663 points
ignored
Ignoring 511 points (too dense)
Processing segments in parallel…
Starting parallel work
WARNING: Overshoot - increase in tension suggested. Overshoot occurs at
(662,749) cell. Z-value 98.513438, zmin 103.513005, zmax
153.065828.
v.surf.rst complete.

real 0m58.009s
user 3m41.476s
sys 0m0.304s

Cheers

Stefan

From: Anna Petrášová [mailto:kratochanna@gmail.com]
Sent: søndag 29. oktober 2017 12.31
To: Stefan Blumentrath <Stefan.Blumentrath@nina.no>
Cc: GRASS-dev <grass-dev@lists.osgeo.org>
Subject: RE: [GRASS-dev] Compilation options

v.surf.rst in trunk has nproc parameter, have you tried to use it?

Anna

On Oct 29, 2017 6:51 AM, “Stefan Blumentrath”
<Stefan.Blumentrath@nina.no>
wrote:

Hei Anna,

Thanks for looking into this.

I did some tests with GRASS 7.2.1svn (r70188) and GRASS 7.3.svn
(r70829) both compiled with --with-openmp

It seems that r.proj does not use multiple threads in GRASS 7.2 (not
tested in 7.3. yet). v.surf.rst uses more cores in GRASS 7.2 with
OpenMP but that does not bring down execution time at all. OpenMP
seems to be disabled for v.surf.rst in GRASS 7.3. v.surf.bspline
however, is significantly faster in GRASS 7.3 with OpenMP…

I will do some more, systematic tests and update the wiki entry as far
as I can, if not someone with knowledge gets there first…

Cheers
Stefan

P.S.: Some preliminary test results:

OpenMP support test

$ OMP_NUM_THREADS=10
$ export OMP_NUM_THREADS
$ time v.surf.rst input=points elevation=test --o --qq

real 51m42.224s
user 398m2.766s
sys 0m18.874s

$ OMP_NUM_THREADS=1
$ export OMP_NUM_THREADS
$ time v.surf.rst input=points elevation=test --o --qq

real 53m22.508s
user 53m10.607s
sys 0m4.793s

$ OMP_NUM_THREADS=10
$ export OMP_NUM_THREADS
$ time v.surf.bspline elevrand column=value output=elev_full --o
real 0m20.509s
user 3m11.412s
sys 0m1.963s

$ OMP_NUM_THREADS=1
$ export OMP_NUM_THREADS
$ time v.surf.bspline elevrand column=value output=elev_full --o
real 1m15.434s
user 1m14.738s
sys 0m0.447s

-----Original Message-----
From: Anna Petrášová [mailto:kratochanna@gmail.com]
Sent: fredag 27. oktober 2017 17.00
To: Stefan Blumentrath <Stefan.Blumentrath@nina.no>
Cc: GRASS developers list (grass-dev@lists.osgeo.org)
<grass-dev@lists.osgeo.org>
Subject: Re: [GRASS-dev] Compilation options

On Tue, Oct 24, 2017 at 10:27 AM, Stefan Blumentrath
<Stefan.Blumentrath@nina.no> wrote:

Hi again,

Seek and ye shall find…

So, there is:

https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7 and

https://grasswiki.osgeo.org/wiki/GPU

Which helped me a bit further.

However, my impression is that not all info there (esp. on OpenMP) is
up to date (e.g. latest development like here:
https://lists.osgeo.org/pipermail/grass-dev/2017-August/085711.html).
Neither did it became very clear to me which modules already have
robust and working support and which are just candidates or where
initial implementations are available (sometimes outside the code
base)…

I updated it slightly to reflect the new modules, as always, more work
is needed on that page.
https://grasswiki.osgeo.org/wiki/OpenMP#OpenMP_support_in_GRASS_7

Anna

Cheers

Stefan

From: grass-dev [mailto:grass-dev-bounces@lists.osgeo.org] On Behalf
Of Stefan Blumentrath
Sent: mandag 23. oktober 2017 09.24
To: GRASS developers list (grass-dev@lists.osgeo.org)
<grass-dev@lists.osgeo.org>
Subject: [GRASS-dev] Compilation options

Dear all,

Does someone of you know which modules benefit from --with-openmp and
–with-opencl compilation options?

I found this explanation of compilation options:
https://grass.osgeo.org/grass72/source/snapshot/REQUIREMENTS.html

But it does not cover OpenCL or OpenMP (as well as some few more
other option).

If I wanted to find out myself, could I just search the code for
modules that import the relevant headers of the respective libraries?

Or is that a bit more complex?

Another question is if it would be recommended to use the different
options, as I understood e.g. using --with-pthread is not necessarily
a good choice…

Thanks in advance.

Cheers

Stefan

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev