[GRASS-dev] [GRASS GIS] #2125: r.in.xyz not importing data with 5 decimal digits

#2125: r.in.xyz not importing data with 5 decimal digits
----------------------+-----------------------------------------------------
Reporter: moreno | Owner: grass-dev@…
     Type: defect | Status: new
Priority: critical | Milestone: 6.4.3
Component: Raster | Version: 6.4.3
Keywords: r.in.xyz | Platform: MSWindows XP
      Cpu: x86-64 |
----------------------+-----------------------------------------------------
When trying to import data from ASCII file using r.in.xyz, in case of too
many decimal digits I get the message "not enough data columns".

{{{
r.in.xyz "input=C:/Documents and Settings/moreno/Desktop/dati.txt"
method=min type=FCELL fs=\t x=1 y=2 z=15 output=import_xyz
}}}

data in attach (it's always the same number, just to show an example)

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2125&gt;
GRASS GIS <http://grass.osgeo.org>

#2125: r.in.xyz not importing data with 5 decimal digits
----------------------+-----------------------------------------------------
Reporter: moreno | Owner: grass-dev@…
     Type: defect | Status: new
Priority: major | Milestone: 6.4.4
Component: Raster | Version: 6.4.3
Keywords: r.in.xyz | Platform: All
      Cpu: x86-64 |
----------------------+-----------------------------------------------------
Changes (by annakrat):

  * priority: critical => major
  * platform: MSWindows XP => All
  * milestone: 6.4.3 => 6.4.4

Comment:

It fails because the length of the line in your file exceeds the maximum
buffer size (256). But of course, it should not fail when the item
corresponding to the maximum column requested is still within that buffer.
Maybe related to this [http://lists.osgeo.org/pipermail/grass-
dev/2013-September/065668.html discussion]?

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2125#comment:1&gt;
GRASS GIS <http://grass.osgeo.org>

#2125: r.in.xyz not importing data with 5 decimal digits
----------------------+-----------------------------------------------------
Reporter: moreno | Owner: grass-dev@…
     Type: defect | Status: new
Priority: major | Milestone: 6.4.4
Component: Raster | Version: 6.4.3
Keywords: r.in.xyz | Platform: All
      Cpu: x86-64 |
----------------------+-----------------------------------------------------

Comment(by moreno):

Replying to [comment:1 annakrat]:
> It fails because the length of the line in your file exceeds the maximum
buffer size (256). But of course, it should not fail when the item
corresponding to the maximum column requested is still within that buffer.
Maybe related to this [http://lists.osgeo.org/pipermail/grass-
dev/2013-September/065668.html discussion]?

I think it's not related, the problem with r.in.xyz occours also in the
middle of the line, not at the ond of it (so, it doesn't depend on the EOL
out of buffer)

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2125#comment:2&gt;
GRASS GIS <http://grass.osgeo.org>

#2125: r.in.xyz not importing data with 5 decimal digits
----------------------+-----------------------------------------------------
Reporter: moreno | Owner: grass-dev@…
     Type: defect | Status: new
Priority: major | Milestone: 6.4.4
Component: Raster | Version: 6.4.3
Keywords: r.in.xyz | Platform: All
      Cpu: x86-64 |
----------------------+-----------------------------------------------------

Comment(by annakrat):

Replying to [comment:2 moreno]:
> Replying to [comment:1 annakrat]:
> > It fails because the length of the line in your file exceeds the
maximum buffer size (256). But of course, it should not fail when the item
corresponding to the maximum column requested is still within that buffer.
Maybe related to this [http://lists.osgeo.org/pipermail/grass-
dev/2013-September/065668.html discussion]?
>
>
> I think it's not related, the problem with r.in.xyz occours also in the
middle of the line, not at the ond of it (so, it doesn't depend on the EOL
out of buffer)

Well, you can simply test it on your data, when you make the lines in the
CSV file shorter. With max 254 characters per line, it starts to work for
me. The problem is that everything after the limit is then read into the
next line instead of the next line itself.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2125#comment:3&gt;
GRASS GIS <http://grass.osgeo.org>

#2125: r.in.xyz not importing data with 5 decimal digits
----------------------+-----------------------------------------------------
Reporter: moreno | Owner: hamish
     Type: defect | Status: assigned
Priority: major | Milestone: 6.4.4
Component: Raster | Version: 6.4.3
Keywords: r.in.xyz | Platform: All
      Cpu: x86-64 |
----------------------+-----------------------------------------------------
Changes (by hamish):

* cc: grass-dev@… (added)
  * owner: grass-dev@… => hamish
  * status: new => assigned

Comment:

Hi,

I have increased the buffer size to 1024 chars in the development branches
r58203,4. It doesn't really fix the heart of the problem but fixes it for
you and makes it less likely to occur in the future (maybe we should keep
going to 4096 chars). Please test with tomorrow's nightly build of
WinGrass 6.5.

When the line is longer than the specified input buffer I'm not sure if it
really matters if G_getl2() flushes the rest of the line or not. Either
way the data-read has gone wrong and you'll want a G_fatal_error(), so the
question becomes at what point to throw that error, and what the error
should say. _("Line is too long for input buffer") might be a good
candidate for what it should say.

thanks,
Hamish

ps- consider to use fs=tab instead of fs='\t'.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2125#comment:4&gt;
GRASS GIS <http://grass.osgeo.org>

#2125: r.in.xyz not importing data when line is 800 characters long
----------------------+-----------------------------------------------------
Reporter: moreno | Owner: hamish
     Type: defect | Status: assigned
Priority: major | Milestone: 6.4.4
Component: Raster | Version: 6.4.3
Keywords: r.in.xyz | Platform: All
      Cpu: x86-64 |
----------------------+-----------------------------------------------------

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2125#comment:5&gt;
GRASS GIS <http://grass.osgeo.org>

#2125: r.in.xyz not importing data when line is 800 characters long
----------------------+-----------------------------------------------------
Reporter: moreno | Owner: hamish
     Type: defect | Status: assigned
Priority: major | Milestone: 6.4.5
Component: Raster | Version: 6.4.3
Keywords: r.in.xyz | Platform: All
      Cpu: x86-64 |
----------------------+-----------------------------------------------------
Changes (by hamish):

  * milestone: 6.4.4 => 6.4.5

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2125#comment:6&gt;
GRASS GIS <http://grass.osgeo.org>