[GRASS-user] filtering csv file before input for v.in.ascii

hi,

given data in csv format with columns e.g. xcoor, ycoor and a few other as
input for v.in.ascii where some rows may have an empty entry in xcoor,
ycoor.

the idea for a little script wrapper for v.in.ascii:

filter the input file by empty xcoor, ycoor, then forward the
modified/filtered csv to v.in.ascii

any hints in python?

thanks

-----
best regards
Helmut
--
View this message in context: http://osgeo-org.1560.x6.nabble.com/filtering-csv-file-before-input-for-v-in-ascii-tp5211340.html
Sent from the Grass - Users mailing list archive at Nabble.com.

On Tue, Jun 16, 2015 at 10:51 PM, Helmut Kudrnovsky <hellik@web.de> wrote:

hi,

given data in csv format with columns e.g. xcoor, ycoor and a few other as
input for v.in.ascii where some rows may have an empty entry in xcoor,
ycoor.

the idea for a little script wrapper for v.in.ascii:

filter the input file by empty xcoor, ycoor, then forward the
modified/filtered csv to v.in.ascii

Did you see the "new" flag of v.in.ascii

-i Ignore broken line(s) in points mode

? For us it did the job on a lot of data sets.

Markus

Markus Neteler wrote

On Tue, Jun 16, 2015 at 10:51 PM, Helmut Kudrnovsky &lt;

hellik@

&gt; wrote:

hi,

given data in csv format with columns e.g. xcoor, ycoor and a few other
as
input for v.in.ascii where some rows may have an empty entry in xcoor,
ycoor.

the idea for a little script wrapper for v.in.ascii:

filter the input file by empty xcoor, ycoor, then forward the
modified/filtered csv to v.in.ascii

Did you see the "new" flag of v.in.ascii

-i Ignore broken line(s) in points mode

? For us it did the job on a lot of data sets.

Markus

_-i Ignore broken line(s) in points mode_ doesn't work here.

it's GBIF data file (attached an short example: test.csv, and a modified one
by quoting the entries: test2.csv). GBIF data is"usually" in WGS84.

some xcoor, ycoor in this sample dataset are empty or where some xcoor,
ycoor have coordinates for some other coordinate systems not matching WGS84.
so some coordinate pairs are out of range for a ll-location.

test.csv <http://osgeo-org.1560.x6.nabble.com/file/n5213229/test.csv&gt;
test2.csv <http://osgeo-org.1560.x6.nabble.com/file/n5213229/test2.csv&gt;

-----
best regards
Helmut
--
View this message in context: http://osgeo-org.1560.x6.nabble.com/filtering-csv-file-before-input-for-v-in-ascii-tp5211340p5213229.html
Sent from the Grass - Users mailing list archive at Nabble.com.

Hi Helmut,

Did you download from GBIF site? Because there are 2 (that I know) R packages that allows you to download data from GBIF, and you can ask for “NO SPATIAL ISSUES” in the data (among other options), probably in that way you can avoid these coordinates that are in a different reference system. Does that make sense?? Ah! The packages are: rgbif and dismo

If you want, I can provide an example… I just tried yesterday :slight_smile:

HTH,
Vero

···

2015-06-27 10:16 GMT-03:00 Helmut Kudrnovsky <hellik@web.de>:

Markus Neteler wrote

On Tue, Jun 16, 2015 at 10:51 PM, Helmut Kudrnovsky <

hellik@

> wrote:

hi,

given data in csv format with columns e.g. xcoor, ycoor and a few other
as
input for v.in.ascii where some rows may have an empty entry in xcoor,
ycoor.

the idea for a little script wrapper for v.in.ascii:

filter the input file by empty xcoor, ycoor, then forward the
modified/filtered csv to v.in.ascii

Did you see the “new” flag of v.in.ascii

-i Ignore broken line(s) in points mode

? For us it did the job on a lot of data sets.

Markus

-i Ignore broken line(s) in points mode doesn’t work here.

it’s GBIF data file (attached an short example: test.csv, and a modified one
by quoting the entries: test2.csv). GBIF data is"usually" in WGS84.

some xcoor, ycoor in this sample dataset are empty or where some xcoor,
ycoor have coordinates for some other coordinate systems not matching WGS84.
so some coordinate pairs are out of range for a ll-location.

test.csv <http://osgeo-org.1560.x6.nabble.com/file/n5213229/test.csv>
test2.csv <http://osgeo-org.1560.x6.nabble.com/file/n5213229/test2.csv>


best regards
Helmut

View this message in context: http://osgeo-org.1560.x6.nabble.com/filtering-csv-file-before-input-for-v-in-ascii-tp5211340p5213229.html
Sent from the Grass - Users mailing list archive at Nabble.com.


grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

hi Veronica,

Veronica Andreo wrote

Hi Helmut,

Did you download from GBIF site? Because there are 2 (that I know) R
packages that allows you to download data from GBIF, and you can ask for
"NO SPATIAL ISSUES" in the data (among other options), probably in that
way
you can avoid these coordinates that are in a different reference system.
Does that make sense?? Ah! The packages are: rgbif and dismo

If you want, I can provide an example... I just tried yesterday :slight_smile:

HTH,
Vero

thanks for the R hints, rgbif and dismo works quite nicely.

I've just uploaded a new addon:

new addon: v.in.gbif to import GBIF species distribution data via GDAL
vector VRT
https://lists.osgeo.org/pipermail/grass-commit/2015-August/036932.html

the addon imports downloaded and unzipped GBIF data via GDAL vector VRT to
the location without filtering non-existing/false latlong coordinates. these
points ca may be edited by the information given in the column locality
later on.

testing and improving welcome. :slight_smile:

-----
best regards
Helmut
--
View this message in context: http://osgeo-org.1560.x6.nabble.com/filtering-csv-file-before-input-for-v-in-ascii-tp5211340p5221319.html
Sent from the Grass - Users mailing list archive at Nabble.com.