[GRASS-dev] extending v.in.ascii capacity to scan large number of columns (code change provided)

v.in.ascii/points.c:

76 buflen = 400000; #Changed from 4000
77 buf = (char *)G_malloc(buflen);
78 buf_raw = (char *)G_malloc(buflen);
79 coorbuf = (char *)G_malloc(16*256); #Added 16*
80 tmp_token = (char *)G_malloc(16*256); #Added 16*

This change permitted me to read cleanly through an ascii file x,y,v1,v2...v4018
and import each of the x,y,v combinations as 3D vector layers
independently (without attribute table)

--
----

Yann Chemin wrote:

76 buflen = 400000; #Changed from 4000
77 buf = (char *)G_malloc(buflen);
78 buf_raw = (char *)G_malloc(buflen);
79 coorbuf = (char *)G_malloc(16*256); #Added 16*
80 tmp_token = (char *)G_malloc(16*256); #Added 16*

This change permitted me to read cleanly through an ascii file x,y,v1,v2...v4018
and import each of the x,y,v combinations as 3D vector layers
independently (without attribute table)

If you have a need for many columns, it would be better to change it
to support any number of columns rather than simply increasing the
hard-coded limit to a larger value.

For a moderately-large number of columns, you'd need a version of
G_getl() which realloc()s its buffer as needed.

To cope with very large numbers of columns, the pull parser (reading a
line at a time) should be replaced by a push parser (reading
characters and pushing completed tokens into a state machine), e.g.:

  for (;:wink: {
      int c = fgetc(fp);
      if (c == EOF) { /* end of file */ }
      else if (c == '\n') { /* new record */ }
      else if (c == separator) { /* new field */ }
      else { /* append c to current field */ }
  }

--
Glynn Clements <glynn@gclements.plus.com>