Hi,
just saw that this article got published (congrats, Jaro and team):
Hofierka, J., Lacko, M., Zubal, S., 2017. Parallelization of
interpolation, solar radiation and water flow simulation modules in
GRASS GIS using OpenMP. Computers & Geosciences.
doi:10.1016/j.cageo.2017.07.007
Questions: since the code is already in addons, how about merging the
changes into trunk?
Markus
On 04/08/17 12:35, Markus Neteler wrote:
Hi,
just saw that this article got published (congrats, Jaro and team):
Hofierka, J., Lacko, M., Zubal, S., 2017. Parallelization of
interpolation, solar radiation and water flow simulation modules in
GRASS GIS using OpenMP. Computers & Geosciences.
doi:10.1016/j.cageo.2017.07.007Questions: since the code is already in addons, how about merging the
changes into trunk?
I'm no expert on OpenMP. I imagine it is integrated at compile time ? does this mean that openmp will become a dependency of GRASS GIS ?
Moritz
Thanks Maris,
[BTW: your mail was sent only to me, so I'm forwarding to the list.]
On 04/08/17 16:04, Maris Nartiss wrote:
1. It is not a dependency, as OpenMP code should run just fine also
when compiled without OpenMP. Those are just comments in code thus
backwards safe.
Right. I now see that all code is conditionalized on _OPENMP being set and I suppose that happens when you use --with-openmp as a compiler switch ?
2. It is not as easy as it seems, as every place, where
parallelisation is introduced, needs to be examined carefully to not
introduce hard to track issues when common variables are changed and
to not make even worse performance by making CPU cache dirty when it
isn't. Of course these problems affect only binaries compiled with
OpenMP. Thus OpenMP support in GRASS GIS could be marked as "use at
own risk".
This thus means that merging the OpenMP version into the trunk version will not have any consequences for the "normal" user. However, does it mean that maintenance of the modules will become more complex ?
I see that in v.surf.rst the OpenMP stuff has already been integrated. It's only been 7 months, but up to now noone has complained, so I think we could integrate openmp into the two other modules as well (r.sun + r.sim.water).
Moritz
Hi Markus,
This is just a small contribution and I would like acknowledge help of reviewers to improve the manuscript. I believe more such grass-based papers in scientific journals will further promote GRASS within the wider scientific community.
Of course, we would welcome if this could go to the trunk version. Vaclav helped us is v.surf.rst and it is already there and I hope he could also help with r.sim.water and r.sun. He can contact me or my colleagues and we will provide any assistance.
Best wishes,
Jaro
Markus Neteler napísal(a):
Hi,
just saw that this article got published (congrats, Jaro and team):
Hofierka, J., Lacko, M., Zubal, S., 2017. Parallelization of
interpolation, solar radiation and water flow simulation modules in
GRASS GIS using OpenMP. Computers & Geosciences.
doi:10.1016/j.cageo.2017.07.007Questions: since the code is already in addons, how about merging the
changes into trunk?Markus
Dear Jaro and all,
During recent code sprint in Boston I ported r.sim.water.mp to GRASS
r.sim.water and r.sim.sediment. I tested it with r.sim.sediment and it
seems to work, so let me know if there are any reasons parallelization
shouldn't be used with r.sim.sediment. I removed all the commented
parts which you probably put there during experimenting. I didn't
remove the version in addons, because you have it linked in your
paper, but maybe we should put a note in the r.sim.water.mp manual,
that the latest version is in the core? As for r.sun, when I have
time, I will look at it.
Anna
On Sat, Aug 5, 2017 at 4:24 AM, Jaro Hofierka <jhofierka@gmail.com> wrote:
Hi Markus,
This is just a small contribution and I would like acknowledge help of
reviewers to improve the manuscript. I believe more such grass-based papers
in scientific journals will further promote GRASS within the wider
scientific community.
Of course, we would welcome if this could go to the trunk version. Vaclav
helped us is v.surf.rst and it is already there and I hope he could also
help with r.sim.water and r.sun. He can contact me or my colleagues and we
will provide any assistance.Best wishes,
Jaro
Markus Neteler napísal(a):
Hi,
just saw that this article got published (congrats, Jaro and team):
Hofierka, J., Lacko, M., Zubal, S., 2017. Parallelization of
interpolation, solar radiation and water flow simulation modules in
GRASS GIS using OpenMP. Computers & Geosciences.
doi:10.1016/j.cageo.2017.07.007Questions: since the code is already in addons, how about merging the
changes into trunk?Markus
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev
Hi Anna,
many thanks! Please keep it in addons for a while, as it is referenced in the paper. Yes it is possible to use it in r.sim.sediment as well because actually we have parallelized the simulation library (simlib) used by r.sim.sediment as well. But it needs some modification of Makefile and main.c to include openMP. Otherwise it runs without OpenMP.
Also please note that I have changed the maximum number of walkers in waterglobs.h to 20 million from 7 million. This might be needed to compute large grids, but the simulation algorithm should be checked because sometimes it reports error (nwalk > maxw) when assigns walkers (this is not related to openMP).
Best wishes,
Jaro
Anna Petrášová napísal(a):
Dear Jaro and all,
During recent code sprint in Boston I ported r.sim.water.mp to GRASS
r.sim.water and r.sim.sediment. I tested it with r.sim.sediment and it
seems to work, so let me know if there are any reasons parallelization
shouldn't be used with r.sim.sediment. I removed all the commented
parts which you probably put there during experimenting. I didn't
remove the version in addons, because you have it linked in your
paper, but maybe we should put a note in the r.sim.water.mp manual,
that the latest version is in the core? As for r.sun, when I have
time, I will look at it.Anna
On Sat, Aug 5, 2017 at 4:24 AM, Jaro Hofierka <jhofierka@gmail.com> wrote:
Hi Markus,
This is just a small contribution and I would like acknowledge help of
reviewers to improve the manuscript. I believe more such grass-based papers
in scientific journals will further promote GRASS within the wider
scientific community.
Of course, we would welcome if this could go to the trunk version. Vaclav
helped us is v.surf.rst and it is already there and I hope he could also
help with r.sim.water and r.sun. He can contact me or my colleagues and we
will provide any assistance.Best wishes,
Jaro
Markus Neteler napísal(a):
Hi,
just saw that this article got published (congrats, Jaro and team):
Hofierka, J., Lacko, M., Zubal, S., 2017. Parallelization of
interpolation, solar radiation and water flow simulation modules in
GRASS GIS using OpenMP. Computers & Geosciences.
doi:10.1016/j.cageo.2017.07.007Questions: since the code is already in addons, how about merging the
changes into trunk?Markus
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev
Hi,
yes I already changed the r.sim.sediment, so it is running there, I
just wanted to check there is no theoretical issue since you haven't
mentioned it in the paper. I kept the max number of walkers lower as
it was, the problem if I remember correctly is that is uses static
allocations, so if we raise it, it will take a lot of memory. So that
would need to be rewritten. Perhaps you should create a ticket about
that error, to keep track of it. Thank you
On Thu, Aug 24, 2017 at 3:38 PM, Jaro Hofierka <jhofierka@gmail.com> wrote:
Hi Anna,
many thanks! Please keep it in addons for a while, as it is referenced in
the paper. Yes it is possible to use it in r.sim.sediment as well because
actually we have parallelized the simulation library (simlib) used by
r.sim.sediment as well. But it needs some modification of Makefile and
main.c to include openMP. Otherwise it runs without OpenMP.
Also please note that I have changed the maximum number of walkers in
waterglobs.h to 20 million from 7 million. This might be needed to compute
large grids, but the simulation algorithm should be checked because
sometimes it reports error (nwalk > maxw) when assigns walkers (this is not
related to openMP).Best wishes,
Jaro
Anna Petrášová napísal(a):
Dear Jaro and all,
During recent code sprint in Boston I ported r.sim.water.mp to GRASS
r.sim.water and r.sim.sediment. I tested it with r.sim.sediment and it
seems to work, so let me know if there are any reasons parallelization
shouldn't be used with r.sim.sediment. I removed all the commented
parts which you probably put there during experimenting. I didn't
remove the version in addons, because you have it linked in your
paper, but maybe we should put a note in the r.sim.water.mp manual,
that the latest version is in the core? As for r.sun, when I have
time, I will look at it.Anna
On Sat, Aug 5, 2017 at 4:24 AM, Jaro Hofierka <jhofierka@gmail.com> wrote:
Hi Markus,
This is just a small contribution and I would like acknowledge help of
reviewers to improve the manuscript. I believe more such grass-based
papers
in scientific journals will further promote GRASS within the wider
scientific community.
Of course, we would welcome if this could go to the trunk version. Vaclav
helped us is v.surf.rst and it is already there and I hope he could also
help with r.sim.water and r.sun. He can contact me or my colleagues and
we
will provide any assistance.Best wishes,
Jaro
Markus Neteler napísal(a):
Hi,
just saw that this article got published (congrats, Jaro and team):
Hofierka, J., Lacko, M., Zubal, S., 2017. Parallelization of
interpolation, solar radiation and water flow simulation modules in
GRASS GIS using OpenMP. Computers & Geosciences.
doi:10.1016/j.cageo.2017.07.007Questions: since the code is already in addons, how about merging the
changes into trunk?Markus
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev