[GRASS-dev] Re: v.krige.py Documentation

Il giorno mer, 02/09/2009 alle 07.30 -0400,
grass-dev-request@lists.osgeo.org ha scritto:

Greetings,

I am attempting to use the v.krige.py extension with RC 5 by testing
it with 20,000 points that it is trying to allocate 3 gigabytes of
memory for and failing (which is odd because I have 8 GB on this
machine and top reports that I have 4.3 GB free). I was able to
successfully krig 5,000 points.

Hello Jeshua,

first of all thanks for testing v.krige. Your case study is very useful
for v.krige improvement.

Ultimately I need to be able to handle around a million points - not
looking like it will be possible based on my test.

Can anyone tell me what the required memory is based on? Is it based
on the region, the points, the block size, etc?

On the number of points and on the region. They both contribute to
create large intermediate and output objects.

R memory usage use deserves some consideration: you can find information
here [0,1]. I know it is a limitation when handling large datasets: a
good improvement for the code will be for example to store objects on
the hard drive instead of RAM. Alternatively, v.krige could work on a
slice of data at once, but I don't know if it is possible - worth to ask
to Roger Bivand and Edzer Pebesma.

Also, I can find very little documentation for it. For instance I am
not sure what the block size is used for, how is it different than the
cell size set with g.region?

Block size is a parameter used by block kriging, different from region
resolution - see krige() documentation and examples [2].

It looks very interesting if I were able to actually use it.

The only documentation I can find for it is the man page and the wiki
page neither of which really address any of these questions.

That's right - as v.krige is just a wrapper, its documentation should
contain all useful pointers to the code it includes. Will improve this
asap.

Note that I can only run it from the command line, when I try to start
the wxpython interface it displays a box for an instant and then quits
without any error messages.

I suppose you run Windows: that's what happen when you use head revision
of v.krige and an outdated revision of GRASS. The module is presently
developed with develbranch_6, so needs recent updates (r38834 or
higher). I plan to improve backward compatibility in the near future.

Hope this helps!

best regards,
Anne Ghisla

Thanks,

Jeshua Lacock, Owner
<http://OpenOSX.com>
phone: 208.462.4171

[0]
http://stat.ethz.ch/R-manual/R-patched/library/base/html/Memory-limits.html
[1] http://www.matthewckeller.com/html/memory.html
[2] http://sekhon.berkeley.edu/library/gstat/html/krige.html

2009/9/2 Anne Ghisla <a.ghisla@gmail.com>:

Il giorno mer, 02/09/2009 alle 07.30 -0400,
grass-dev-request@lists.osgeo.org ha scritto:

Greetings,

I am attempting to use the v.krige.py extension with RC 5 by testing
it with 20,000 points that it is trying to allocate 3 gigabytes of
memory for and failing (which is odd because I have 8 GB on this
machine and top reports that I have 4.3 GB free). I was able to
successfully krig 5,000 points.

Hello Jeshua,

first of all thanks for testing v.krige. Your case study is very useful
for v.krige improvement.

Ultimately I need to be able to handle around a million points - not
looking like it will be possible based on my test.

Hi,

Note that with large point counts, and possibly high point density,
local smoothing approaches are probably more efficient. Check out RST
interpolation in GRASS for cases of >1 million points. I am pretty
sure that it scales efficiently to >50 million points.

Cheers,
Dylan

Can anyone tell me what the required memory is based on? Is it based
on the region, the points, the block size, etc?

On the number of points and on the region. They both contribute to
create large intermediate and output objects.

R memory usage use deserves some consideration: you can find information
here [0,1]. I know it is a limitation when handling large datasets: a
good improvement for the code will be for example to store objects on
the hard drive instead of RAM. Alternatively, v.krige could work on a
slice of data at once, but I don't know if it is possible - worth to ask
to Roger Bivand and Edzer Pebesma.

Also, I can find very little documentation for it. For instance I am
not sure what the block size is used for, how is it different than the
cell size set with g.region?

Block size is a parameter used by block kriging, different from region
resolution - see krige() documentation and examples [2].

It looks very interesting if I were able to actually use it.

The only documentation I can find for it is the man page and the wiki
page neither of which really address any of these questions.

That's right - as v.krige is just a wrapper, its documentation should
contain all useful pointers to the code it includes. Will improve this
asap.

Note that I can only run it from the command line, when I try to start
the wxpython interface it displays a box for an instant and then quits
without any error messages.

I suppose you run Windows: that's what happen when you use head revision
of v.krige and an outdated revision of GRASS. The module is presently
developed with develbranch_6, so needs recent updates (r38834 or
higher). I plan to improve backward compatibility in the near future.

Hope this helps!

best regards,
Anne Ghisla

Thanks,

Jeshua Lacock, Owner
<http://OpenOSX.com>
phone: 208.462.4171

[0]
http://stat.ethz.ch/R-manual/R-patched/library/base/html/Memory-limits.html
[1] http://www.matthewckeller.com/html/memory.html
[2] http://sekhon.berkeley.edu/library/gstat/html/krige.html

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

On Sep 2, 2009, at 6:05 AM, Anne Ghisla wrote:

first of all thanks for testing v.krige. Your case study is very useful
for v.krige improvement.

Thank you Anne for your efforts and reply.

Ultimately I need to be able to handle around a million points - not
looking like it will be possible based on my test.

Can anyone tell me what the required memory is based on? Is it based
on the region, the points, the block size, etc?

On the number of points and on the region. They both contribute to
create large intermediate and output objects.

R memory usage use deserves some consideration: you can find information
here [0,1]. I know it is a limitation when handling large datasets: a
good improvement for the code will be for example to store objects on
the hard drive instead of RAM. Alternatively, v.krige could work on a
slice of data at once, but I don't know if it is possible - worth to ask
to Roger Bivand and Edzer Pebesma.

I am not very experienced with the Krig algorithm, perhaps someone could offer some advice, however slicing it into tiles seems problematic to me because I am under the impression that I need the neighboring data for optimal interpolation.

Also, I can find very little documentation for it. For instance I am
not sure what the block size is used for, how is it different than the
cell size set with g.region?

Block size is a parameter used by block kriging, different from region
resolution - see krige() documentation and examples [2].

Thanks.

It looks very interesting if I were able to actually use it.

The only documentation I can find for it is the man page and the wiki
page neither of which really address any of these questions.

That's right - as v.krige is just a wrapper, its documentation should
contain all useful pointers to the code it includes. Will improve this
asap.

Understood, thanks again.

Note that I can only run it from the command line, when I try to start
the wxpython interface it displays a box for an instant and then quits
without any error messages.

I suppose you run Windows: that's what happen when you use head revision
of v.krige and an outdated revision of GRASS. The module is presently
developed with develbranch_6, so needs recent updates (r38834 or
higher). I plan to improve backward compatibility in the near future.

Actually I am on Mac OS X using 6.4 RC 5.

I checked out v.krig yesterday. Are you suggesting that it requires GRASS 6.4 from SVN (I am not sure what is meant by r38834)?

Best,

Jeshua Lacock, Owner
<http://OpenOSX.com>
phone: 208.462.4171

Il giorno mer, 02/09/2009 alle 16.23 -0600, Jeshua Lacock ha scritto:

On Sep 2, 2009, at 6:05 AM, Anne Ghisla wrote:

[..]

> I suppose you run Windows: that's what happen when you use head
> revision
> of v.krige and an outdated revision of GRASS. The module is presently
> developed with develbranch_6, so needs recent updates (r38834 or
> higher). I plan to improve backward compatibility in the near future.

Actually I am on Mac OS X using 6.4 RC 5.
I checked out v.krige yesterday. Are you suggesting that it requires
GRASS 6.4 from SVN (I am not sure what is meant by r38834)?

Right, it requires to compile GRASS from SVN.
You can obtain suitable source with a command like:

svn co https://svn.osgeo.org/grass/grass/branches/develbranch_6
grass6_devel

See these pages [0,1] for further information.
If you feel unconfortable with compiling, you can first follow Dylan's
advice and use v.surf.rst [2] with your current version of GRASS.

regards,
Anne

[0] https://trac.osgeo.org/grass/wiki/DownloadSource
[1] http://grass.osgeo.org/wiki/Compiling_on_MacOSX
[2] http://grass.osgeo.org/grass64/manuals/html64_user/v.surf.rst.html

Hi,

2009/9/3 Jeshua Lacock <jeshua@openosx.com>:

Actually I am on Mac OS X using 6.4 RC 5.

I checked out v.krig yesterday. Are you suggesting that it requires GRASS
6.4 from SVN (I am not sure what is meant by r38834)?

just v.krige's GUI requires wxGUI from GRASS 6.5. Some important
improvements hasn't been backported (because of freeze stage) to GRASS
6.4 (releasebranch_6_4).

v.krige's CLI should work also with GRASS 6.4.

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa

On Thu, Sep 3, 2009 at 12:21 PM, Martin Landa<landa.martin@gmail.com> wrote:

Hi,

2009/9/3 Jeshua Lacock <jeshua@openosx.com>:

Actually I am on Mac OS X using 6.4 RC 5.

I checked out v.krig yesterday. Are you suggesting that it requires GRASS
6.4 from SVN (I am not sure what is meant by r38834)?

just v.krige's GUI requires wxGUI from GRASS 6.5. Some important
improvements hasn't been backported (because of freeze stage) to GRASS
6.4 (releasebranch_6_4).

Please mark them for 6.4.1.

Markus

v.krige's CLI should work also with GRASS 6.4.

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev