[GRASS-dev] [GRASS GIS] #96: v.surf.bspline intermittent failure - maybe on large datasets

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: unspecified
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by neteler):

Replying to [comment:14 cmbarton]:
> I just tried out the updates in GRASS 7 trunk. The column option still
doesn't work right.

To my knowledge a "make distclean" is needed here.

Markus

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/96#comment:15&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: unspecified
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by cmbarton):

I always do a make distclean. So this is not the problem.

Michael

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/96#comment:16&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: unspecified
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by cmbarton):

Following Helena's suggestion, I did a make distclean BEFORE running svn
up. Still same result. Cannot use 3D points and 2D points with attribute
don't work.

So v.surf.bspline has gone for me from working under constrained
conditions to not working at all.

BTW, using the layer=-1 for 3D points is poor interface design. There
should be a flag for this. It should not depend on using an illegal value
for layers.

A couple new results that I'll report elsewhere. The GUI no longer
launches automatically and the r.bilinear is greyed out.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/96#comment:17&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: unspecified
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by hamish):

Replying to [comment:17 cmbarton]:
> BTW, using the layer=-1 for 3D points is poor interface design.
> There should be a flag for this. It should not depend on using
> an illegal value for layers.

well, if it helps, think of it as "layer=none" in a system constrained to
integers. maybe not ideal, but not totally evil either. A programmer's
trick which has snuck into the UI..

Hamish

ps- something weird with server reboots, ldap, or something, I'm having to
re-logon into trac every hour or so the last couple of days.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/96#comment:18&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 6.4.0
Component: Raster | Version: unspecified
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by cmbarton):

Replying to [comment:18 hamish]:
> Replying to [comment:17 cmbarton]:
> > BTW, using the layer=-1 for 3D points is poor interface design.
> > There should be a flag for this. It should not depend on using
> > an illegal value for layers.
>
> well, if it helps, think of it as "layer=none" in a system constrained
to integers. maybe not ideal, but not totally evil either. A programmer's
trick which has snuck into the UI..

Yeah. I know. But it wreaks havoc with trying to make an autogenerated
GUI. separate functions need to have separate arguments/flags.

Michael

>
>
> Hamish
>
> ps- something weird with server reboots, ldap, or something, I'm having
to re-logon into trac every hour or so the last couple of days.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/96#comment:19&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Changes (by mmetz):

  * version: unspecified => svn-releasebranch64
  * milestone: 6.4.0 => 7.0.0

Comment:

Replying to [comment:17 cmbarton]:
> Cannot use 3D points and 2D points with attribute don't work.
That's a bug in the wxGUI, works fine from the command line.
>
> BTW, using the layer=-1 for 3D points is poor interface design. There
should be a flag for this. It should not depend on using an illegal value
for layers.

I don't think it is up to the GUI to define what is an illegal value. If
at all, it should use the module's definition of legal and illegal values
and not come up with its own definitions. Many modules accept -1 as a
legal value for the layer option, meaning all layers. Another option would
be to leave the layer option empty (don't specify it) which requires that
it must be possible to clear the layer option in the wxGUI.
>
> The GUI no longer launches automatically and the r.bilinear is greyed
out.
The wxGUI needs to be updated and the r.bilinear entry removed for grass7.
This module has been merged into r.resamp.interp (see r.bilinear manual in
grass64).

worksforme,

Markus M

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/96#comment:20&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by cmbarton):

The GUI is not defining what is an illegal value. -1 is not a valid value
for a layer. There is no layer -1. Using 3D points instead of a column in
an attribute table has nothing to do with picking layers. It is a choice
of using a value from the attribute table OR using the z value of 3D
points.

It is easy for the GUI to create a spin box for layer selection that runs
from -1 to a max. But that would be the case for all interfaces that use
layers. What happens with d.vect layer=-1?

The problem is using the layer selection to make the module do something
that has nothing to do with layer selection. For ease of parsing in
scripts and for consistent creation of a GUI interface, each distinct
function should have it's own argument (option or flag). Trying to combine
two different functions into one argument leads to problems in using the
module syntax for anything beyond human command-line use.

I guess I don't see what the problem is in having a z flag.

BTW, I see that Martin has already fixed the r.bilinear entry a day or two
ago.

Michael

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/96#comment:21&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by mmetz):

Replying to [comment:17 cmbarton]:
> So v.surf.bspline has gone for me from working under constrained
conditions to not working at all.

Please post the exact commands you used. BTW, when using 3D vector points,
just don't specify the layer option (it's optional anyway).

There are two vectors in the nc_spm_08 dataset that are suitable for
testing the column and 3D points mode:

precip_30ynormals@PERMANENT

and

precip_30ynormals_3d@PERMANENT

A good spline step value for these vectors is 50000 (the default is meant
for LiDAR points with 1m spacing).

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/96#comment:22&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by mmetz):

Replying to [comment:21 cmbarton]:
>
> It is easy for the GUI to create a spin box for layer selection that
runs from -1 to a max. But that would be the case for all interfaces that
use layers. What happens with d.vect layer=-1?
>
In grass7, nearly all modules have a layer option because of direct OGR
read access added by Martin. Layer=-1 means, use all layers, as for
d.vect. For most modules, the layer option is optional, i.e. it must be
possible in the GUI to either leave it empty or specify a layer if
necessary.

> The problem is using the layer selection to make the module do something
that has nothing to do with layer selection. For ease of parsing in
scripts and for consistent creation of a GUI interface, each distinct
function should have it's own argument (option or flag). Trying to combine
two different functions into one argument leads to problems in using the
module syntax for anything beyond human command-line use.

I agree, there should be a possibility to choose where the z values are
coming from.
>
> I guess I don't see what the problem is in having a z flag.
>
Added in trunk r41098, please test. Works for me in nc_spm_08 with both
precip_30ynormals@PERMANENT and precip_30ynormals_3d@PERMANENT. When using
z coordinates of precip_30ynormals_3D@PERMANENT, be aware that these are
elevation, not precipitation.

If it passes your testing, backport to all grass6 branches?

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/96#comment:23&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by hamish):

> backport to all grass6 branches?

my vote is no, at least not to 6.4 .. critical bugfixes only at this point
please!

  * if one of the guis is broken, fix the gui, don't rework the modules to
work around deficiencies in something else.
  * in grass7 bring v.surf.bspline and v.surf.rst's usage into sync.
  * this module existed in 6.2 so layer=-1 must remain at the module level
for backwards compatibility in future 6.x, and can be explained as a
tooltip.

I wonder how we could allow string input there for layer="all" and
layer="none" without ruining the built-in integer checks?

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/96#comment:24&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by hamish):

(this module being designed for massive LIDAR point datasets, where there
is no DB layer & attribute table, and probably no topology built either
FWTW. that's not to say the module shouldn't be useful for other tasks,
but that's what the module designers were thinking about)

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/96#comment:25&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by mmetz):

Replying to [comment:25 hamish]:
> (this module being designed for massive LIDAR point datasets, where
there is no DB layer & attribute table, and probably no topology built
either FWTW. that's not to say the module shouldn't be useful for other
tasks, but that's what the module designers were thinking about)

Yes, most probably the module was designed to interpolate a DTM that is
the result of the whole LiDAR processing procedure. But the module also
works with z values taken from an attribute table (quite nice AFAICT), no
need for topology here.

The layer option problem is IMHO purely a problem of the GUI because it
must be possible to not give a layer number (layer option is optional),
and if a layer number is given, that number must be given by the GUI to
the module. As for other options expecting an integer value.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/96#comment:26&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by hamish):

(to be honest I'm not really against the new -z flag at all, I'm just a
bit frustrated about any new changes to the 6.4.0 branch when we should be
deep in release-mode and I'm venting it here)

maybe have the wxGUI builder check to see if layer= is manadatory before
deciding to do a dropdown list of valid layers?

Hamish

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/96#comment:27&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by cmbarton):

The problem is that the module has been in the distribution but has been
completely broken for a long time. Only recently was it partly fixed so
that it worked with 3D points. But using an attribute value for the
interpolation remained broken. At that time, one picked layer=0 to use 3D
points. AFAIC, this is a problematic design for reasons I outlined
previously. BUT it did work in the GUI and command line.

Very recently there has been a commendable attempt to fix the attribute
value bug. But along with this, for some reason, the way to use 3D points
was changed from layer=0 (working in GUI) to layer=-1 (not working in
GUI). And I was not able to get an interpolated map from the attribute
either. I hope that this all is fixed and will recompile and test as soon
as possible. But I'm in the middle of an NSF workshop right now and can't
do it for a couple of days.

Michael

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/96#comment:28&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by mmetz):

Replying to [comment:28 cmbarton]:
> The problem is that the module has been in the distribution but has been
completely broken for a long time. Only recently was it partly fixed so
that it worked with 3D points. But using an attribute value for the
interpolation remained broken. At that time, one picked layer=0 to use 3D
points.

That is still true for v.surf.rst and v.surf.idw in grass64. I just
discovered that in trunk, these modules got a new -z flag. v.surf.bspline
should now be in sync with the other v.surf. modules in trunk. BTW, there
is a mix for v.surf.rst in devbr6: -z flag and "If [layer is] set to 0, z
coordinates are used. (3D vector only)". v.surf.idw doesn't have a -z flag
in devbr6. A bit of a mess.

>
> Very recently there has been a commendable attempt to fix the attribute
value bug. But along with this, for some reason, the way to use 3D points
was changed from layer=0 (working in GUI) to layer=-1 (not working in
GUI).

Reversed for grass6.x, synced to the other v.surf.* modules, affected the
wxGUI only, not tcltk. Works for me now with all GUIs and from the command
line in all branches, both z coords and attribute column.

Markus M

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/96#comment:29&gt;
GRASS GIS <http://grass.osgeo.org>

On Fri, Feb 19, 2010 at 3:56 AM, GRASS GIS <trac@osgeo.org> wrote:

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
Type: defect | Status: reopened
Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: v.surf.bspline
Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by hamish):

(to be honest I'm not really against the new -z flag at all, I'm just a
bit frustrated about any new changes to the 6.4.0 branch when we should be
deep in release-mode and I'm venting it here)

For me it is the opposite: thanks to MarkusM we can finally use the
Lidare tools and the bspline interpolation. I don't see why bugfixing
is a problem. There is no point in shipping broken modules.

Markus

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by hamish):

Replying to [comment:29 mmetz]:
> Works for me now with all GUIs and from the command line in all
> branches, both z coords and attribute column.

... thanks guys

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/96#comment:30&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by cmbarton):

Thanks very much! I hope to recompile and test this weekend.

I just discovered a couple of other errors in my week-old version of GRASS
7 that I need to check on. Broken profiler and data manager. Hopefully
these are already fixed in the svn.

Michael

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/96#comment:31&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: reopened
  Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Comment (by cmbarton):

I just tested this on a new version of GRASS 7 updated last night. It all
seems to work great. Attribute interpolation works, z value interpolation
works. I especially like the automatic estimation of interpoint distance
to use for interpolation.

This suggests an enhancement for a future version. Option to run with
automatic sie and sin based on this interpoint distance.

Thanks again for getting this important module working well.

I realize that it was initially design for dense lidar data sets. But it
is a very powerful and very fast interpolation routine. I've found it
quite useful for many task for which the other interpolators available do
not work as well.

Michael

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/96#comment:32&gt;
GRASS GIS <http://grass.osgeo.org>

#96: v.surf.bspline column option broken
-----------------------+----------------------------------------------------
  Reporter: cmbarton | Owner: grass-dev@…
      Type: defect | Status: closed
  Priority: major | Milestone: 7.0.0
Component: Raster | Version: svn-releasebranch64
Resolution: fixed | Keywords: v.surf.bspline
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Changes (by hamish):

  * status: reopened => closed
  * resolution: => fixed

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/96#comment:33&gt;
GRASS GIS <http://grass.osgeo.org>