[GRASS-dev] [GRASS GIS] #567: v.kernel: suspected i/o bottleneck

#567: v.kernel: suspected i/o bottleneck
-------------------------+--------------------------------------------------
Reporter: dylan | Owner: grass-dev@lists.osgeo.org
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.5.0
Component: default | Version: svn-develbranch6
Keywords: v.kernel | Platform: Linux
      Cpu: x86-32 |
-------------------------+--------------------------------------------------
v.kernel seems to have a disk i/o bottleneck, that starts to cause
problems with moderately large input vector files. I use this command with
<3000 point features, with a small region setting: approx 500x500 cells,
regularly without issue. Trying out a larger dataset, 22k point features
-- same region geometry, results in a lot of disk activity and very slow
run-times. I took a peak at the source code, and it looks like there are
several calls to a spatial index. There was some chatter on the mailing
list a while back regarding improvement in the way that the spatial index
is created. I wonder if this would improve v.kernel when working with a
large number of features.

The reason that I suspect some kind of I/O issue -- I hear a lot of disk
trashing while this module is running. Also note that it is using very
little memory... perhaps some caching would speed things up.

Some details on the environment:
{{{
# region info:
rows: 632
cols: 364
cells: 230048

# command and arguments
# file 'soilweb_queries' has approx 22k records
v.kernel --o in=soilweb_queries out=swq std=50000
}}}

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/567&gt;
GRASS GIS <http://grass.osgeo.org>

#567: v.kernel: suspected i/o bottleneck
--------------------------+-------------------------------------------------
  Reporter: dylan | Owner: grass-dev@lists.osgeo.org
      Type: enhancement | Status: new
  Priority: normal | Milestone: 6.5.0
Component: default | Version: svn-develbranch6
Resolution: | Keywords: v.kernel
  Platform: Linux | Cpu: x86-32
--------------------------+-------------------------------------------------
Comment (by hamish):

can you run it with "time"? ie:

{{{
time v.kernel --o in=soilweb_queries out=swq std=50000
}}}

that will say how much time is spent doing the work, and how much time is
taken up by the kernel dealing i/o stuff, etc.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/567#comment:1&gt;
GRASS GIS <http://grass.osgeo.org>

#567: v.kernel: suspected i/o bottleneck
--------------------------+-------------------------------------------------
  Reporter: dylan | Owner: grass-dev@lists.osgeo.org
      Type: enhancement | Status: new
  Priority: normal | Milestone: 6.5.0
Component: default | Version: svn-develbranch6
Resolution: | Keywords: v.kernel
  Platform: Linux | Cpu: x86-32
--------------------------+-------------------------------------------------
Comment (by dylan):

Some data:
{{{
std,real,user,sys
100,1.621,1.516,0.100
500,1.854,1.612,0.244
1000,2.680,2.032,0.632
5000,25.090,12.993,12.085
8000,61.72,31.078,30.610
10000,97.877,48.631,49.163
50000,2640,
}}}

Hmm... the sys times do not seem to dominate, rather they approach that of
the usr time.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/567#comment:2&gt;
GRASS GIS <http://grass.osgeo.org>

#567: v.kernel: suspected i/o bottleneck
--------------------------+-------------------------------------------------
  Reporter: dylan | Owner: grass-dev@…
      Type: enhancement | Status: closed
  Priority: normal | Milestone: 6.5.0
Component: Default | Version: svn-develbranch6
Resolution: fixed | Keywords: v.kernel
  Platform: Linux | Cpu: x86-32
--------------------------+-------------------------------------------------
Changes (by neteler):

  * status: new => closed
  * resolution: => fixed

Comment:

v.kernel has been made faster in r54108, r53983, r53982, r53956, r53896.
See also trac #1800. Closing this report as fixed.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/567#comment:3&gt;
GRASS GIS <http://grass.osgeo.org>