[GRASS-user] grass/stats mapping/prediction question

GRASS GRASS user
1

Dear List,

I am having some problems analysing some ecoligical models in grass using the spgrass package through R.

I have 130 plot locations where i have observed presence/absence of a species.
I have followed a similar framework to the BUGSITE modelling example from Markus’s 2003 grass gis handouts (Grass 5)
I have no problems constructing the model based on the 130 plots and the environmental layers from grass.

However, I am having problems bringing all the maps through into R so I can make a prediction map.
The region isnt too large 1600 by 800 cells at 10m resolution
I can bring all the environmental layers through to R using readRAST6() which doesnt take too much time at all.

However i assume I must convert the spatial grid objects into dataframes to apply the predicted model function.
So I then coerce them into dataframes using as.dataframe (this takes ages)
I then merge all the dataframes into a single dataframe. (this takes ages)

I then apply the model predict to the new data frame.

I am unsure how to then move the ‘outmap’ back across to grass.
How can i convert the df to a spatial grid object.

Im thinking i must be doing something wrong. As it quite quick to pull through the layers . But seems to take quite a lot of processing to get the layers into a datframe appropppriate for applying the predictions.

Any help would be greatly appreciated.

Andy

#pull through environmental layers
#FAST
anmax ← readRAST6(“anmax”, ignore.stderr=TRUE)
anmin ← readRAST6(“anmin”, ignore.stderr=TRUE)
aspect ← readRAST6(“aspect”, ignore.stderr=TRUE)
dem10_lidar ← readRAST6(“dem10_lidar”, ignore.stderr=TRUE)

#coerce to dataframe
#SLOW
mypred_anmaxDF<-as.data.frame(anmax)
mypred_anminDF<-as.data.frame(anmin)
mypred_aspectDF<-as.data.frame(aspect)
mypred_dem10_lidarDF<-as.data.frame(dem10_lidar)

#merge into single dataframe
#VERY SLOW
merge_tmp<-merge(mypred_anmaxDF,mypred_anminDF)
rm(mypred_anmaxDF,mypred_anminDF)
merge_tmp1<-merge(merge_tmp,mypred_aspectDF)
rm(merge_tmp,mypred_aspectDF)
mypredDF<-merge(merge_tmp1,mypred_dem10_lidarDF)

#apply model
outmap ← predict(tree,newdata=mypredDF, type=“class”)

2

Hi Andy,

> I am unsure how to then move the 'outmap' back across to grass.
> How can i convert the df to a spatial grid object.

AFAIK, 'predict' won't create a dataframe object.
R should return FALSE for is.data.frame(outmap) and
TRUE for is.numeric(outmap)

You can slot the model output to the AttributeList of
one of the SpatialGridDataFrames that you created when
you read in a GRASS raster and then use writeRAST6 to
write it back to GRASS.

e.g. using 'anmax' from your example

anmax$anmax <- outmap
#if that doesn't work, you might try
anmax$anmax <- as.numeric(outmap)
writeRAST6(anmax, "NameOfNewGRASSRaster", "anmax")

Regards,
Daniel.

cc: grass-stats@lists.osgeo.org

andrew haywood wrote:

Dear List,
I am having some problems analysing some ecoligical models in grass using the spgrass package through R.
I have 130 plot locations where i have observed presence/absence of a species.
I have followed a similar framework to the BUGSITE modelling example from Markus's 2003 grass gis handouts (Grass 5)
I have no problems constructing the model based on the 130 plots and the environmental layers from grass.
However, I am having problems bringing all the maps through into R so I can make a prediction map.
The region isnt too large 1600 by 800 cells at 10m resolution
I can bring all the environmental layers through to R using readRAST6() which doesnt take too much time at all.
However i assume I must convert the spatial grid objects into dataframes to apply the predicted model function.
So I then coerce them into dataframes using as.dataframe (this takes ages)
I then merge all the dataframes into a single dataframe. (this takes ages)
I then apply the model predict to the new data frame.
I am unsure how to then move the 'outmap' back across to grass.
How can i convert the df to a spatial grid object.
Im thinking i must be doing something wrong. As it quite quick to pull through the layers . But seems to take quite a lot of processing to get the layers into a datframe appropppriate for applying the predictions.
Any help would be greatly appreciated.
Andy

#FAST
anmax <- readRAST6("anmax", ignore.stderr=TRUE)
anmin <- readRAST6("anmin", ignore.stderr=TRUE)
aspect <- readRAST6("aspect", ignore.stderr=TRUE)
dem10_lidar <- readRAST6("dem10_lidar", ignore.stderr=TRUE)
#coerce to dataframe
#SLOW
mypred_anmaxDF<-as.data.frame(anmax)
mypred_anminDF<-as.data.frame(anmin)
mypred_aspectDF<-as.data.frame(aspect)
mypred_dem10_lidarDF<-as.data.frame(dem10_lidar)
#merge into single dataframe
#VERY SLOW
merge_tmp<-merge(mypred_anmaxDF,mypred_anminDF)
rm(mypred_anmaxDF,mypred_anminDF)
merge_tmp1<-merge(merge_tmp,mypred_aspectDF)
rm(merge_tmp,mypred_aspectDF)
mypredDF<-merge(merge_tmp1,mypred_dem10_lidarDF)
#apply model
outmap <- predict(tree,newdata=mypredDF, type="class")

------------------------------------------------------------------------

_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

3

On Wed, 19 Dec 2007, Daniel McInerney wrote:

The original questioner should have written to the grass-stats list in the first place - thanks for CC-ing. See below for inline comments:

Hi Andy,

I am unsure how to then move the 'outmap' back across to grass.
How can i convert the df to a spatial grid object.

AFAIK, 'predict' won't create a dataframe object.
R should return FALSE for is.data.frame(outmap) and
TRUE for is.numeric(outmap)

You can slot the model output to the AttributeList of

It hasn't been an AttributeList for a long time - the data slot *is* a data frame. We *always* need the output of sessionInfo() to help - update.packages() is most often very helpful too.

one of the SpatialGridDataFrames that you created when
you read in a GRASS raster and then use writeRAST6 to
write it back to GRASS.

e.g. using 'anmax' from your example

anmax$anmax <- outmap
#if that doesn't work, you might try
anmax$anmax <- as.numeric(outmap)
writeRAST6(anmax, "NameOfNewGRASSRaster", "anmax")

Regards,
Daniel.

cc: grass-stats@lists.osgeo.org

andrew haywood wrote:

Dear List,

I am having some problems analysing some ecoligical models in grass using
the spgrass package through R.

I have 130 plot locations where i have observed presence/absence of a
species. I have followed a similar framework to the BUGSITE modelling
example from Markus's 2003 grass gis handouts (Grass 5) I have no
problems constructing the model based on the 130 plots and the
environmental layers from grass.

See the OSGeo tutorial September 2006:

http://www.foss4g2006.org/contributionDisplay.py?contribId=46&sessionId=59&confId=1

and the OSGeo Journal note:

http://www.osgeo.org/files/journal/final_pdfs/OSGeo_vol1_GRASS-R.pdf

for more up-to-date information.

However, I am having problems bringing all the maps through into R so I
can make a prediction map.
The region isnt too large 1600 by 800 cells at 10m resolution
I can bring all the environmental layers through to R using readRAST6()
which doesnt take too much time at all.

However i assume I must convert the spatial grid objects into dataframes
to apply the predicted model function.

No, usually not at all, since the objects have a data.frame in the data
slot, and have the standard access methods.

So I then coerce them into dataframes using as.dataframe (this takes ages)
I then merge all the dataframes into a single dataframe. (this takes ages)

I then apply the model predict to the new data frame.

I am unsure how to then move the 'outmap' back across to grass.
How can i convert the df to a spatial grid object.

Im thinking i must be doing something wrong. As it quite quick to pull
through the layers . But seems to take quite a lot of processing to get
the layers into a datframe appropppriate for applying the predictions.

Any help would be greatly appreciated.

Andy

# pull through environmental layers
# FAST
anmax <- readRAST6("anmax", ignore.stderr=TRUE)
anmin <- readRAST6("anmin", ignore.stderr=TRUE)
aspect <- readRAST6("aspect", ignore.stderr=TRUE)
dem10_lidar <- readRAST6("dem10_lidar", ignore.stderr=TRUE)

Wrong, do:

mydata <- readRAST6(c("anmax", "anmin", "aspect", "dem10_lidar"),
   ignore.stderr=TRUE)

then the data slot of the object is a data frame. Look at

summary(mydata)

for a sanity check.

# coerce to dataframe
# SLOW
mypred_anmaxDF<-as.data.frame(anmax)
mypred_anminDF<-as.data.frame(anmin)
mypred_aspectDF<-as.data.frame(aspect)
mypred_dem10_lidarDF<-as.data.frame(dem10_lidar)

# merge into single dataframe
# VERY SLOW
merge_tmp<-merge(mypred_anmaxDF,mypred_anminDF)
rm(mypred_anmaxDF,mypred_anminDF)
merge_tmp1<-merge(merge_tmp,mypred_aspectDF)
rm(merge_tmp,mypred_aspectDF)
mypredDF<-merge(merge_tmp1,mypred_dem10_lidarDF)

#apply model

What is tree? You may need to do extra steps depending on what class(tree) says - if you have used rpart() or some such, you may find that

outmap <- predict(tree,newdata=mydata, type="class")

works,

or

outmap <- predict(tree,newdata=as(mydata, "data.frame"), type="class")

Maybe just assign into mydata straight away:

mydata$outmap <- predict(tree,newdata=as(mydata, "data.frame"),
   type="class")

given ?predict.rpart saying:

"If 'type="class"': (for a classification tree) a factor of classifications based on the responses."

which looks like a vector for a vector response in the formula to rpart(). But do check what happens if there are NA in the newdata, because the default predict() behaviour may be to drop those observations. Look at summary(mydata).

Some formula-using model fitting functions just work, like lm() and the predict() method for lm objects.

From there, as Daniel wrote:

writeRAST6(mydata, "rpartpred", "outmap")

Hope this helps,

Roger

outmap <- predict(tree,newdata=mypredDF, type="class")

------------------------------------------------------------------------

_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

_______________________________________________
grass-stats mailing list
grass-stats@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-stats

--
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand@nhh.no

4

On Wednesday 19 December 2007, Roger Bivand wrote:

On Wed, 19 Dec 2007, Daniel McInerney wrote:

The original questioner should have written to the grass-stats list in the

first place - thanks for CC-ing. See below for inline comments:
> Hi Andy,
>
>> I am unsure how to then move the 'outmap' back across to grass.
>> How can i convert the df to a spatial grid object.
>
> AFAIK, 'predict' won't create a dataframe object.
> R should return FALSE for is.data.frame(outmap) and
> TRUE for is.numeric(outmap)
>
> You can slot the model output to the AttributeList of

It hasn't been an AttributeList for a long time - the data slot *is* a
data frame. We *always* need the output of sessionInfo() to help -
update.packages() is most often very helpful too.

> one of the SpatialGridDataFrames that you created when
> you read in a GRASS raster and then use writeRAST6 to
> write it back to GRASS.
>
> e.g. using 'anmax' from your example
>
> anmax$anmax <- outmap
> #if that doesn't work, you might try
> anmax$anmax <- as.numeric(outmap)
> writeRAST6(anmax, "NameOfNewGRASSRaster", "anmax")
>
> Regards,
> Daniel.
>
> cc: grass-stats@lists.osgeo.org
>
> andrew haywood wrote:
>> Dear List,
>>
>> I am having some problems analysing some ecoligical models in grass
>> using the spgrass package through R.
>>
>> I have 130 plot locations where i have observed presence/absence of a
>> species. I have followed a similar framework to the BUGSITE modelling
>> example from Markus's 2003 grass gis handouts (Grass 5) I have no
>> problems constructing the model based on the 130 plots and the
>> environmental layers from grass.

See the OSGeo tutorial September 2006:

http://www.foss4g2006.org/contributionDisplay.py?contribId=46&sessionId=59&
confId=1

and the OSGeo Journal note:

http://www.osgeo.org/files/journal/final_pdfs/OSGeo_vol1_GRASS-R.pdf

for more up-to-date information.

>> However, I am having problems bringing all the maps through into R so I
>> can make a prediction map.
>> The region isnt too large 1600 by 800 cells at 10m resolution
>> I can bring all the environmental layers through to R using readRAST6()
>> which doesnt take too much time at all.
>>
>> However i assume I must convert the spatial grid objects into
>> dataframes to apply the predicted model function.

No, usually not at all, since the objects have a data.frame in the data
slot, and have the standard access methods.

>> So I then coerce them into dataframes using as.dataframe (this takes
>> ages) I then merge all the dataframes into a single dataframe. (this
>> takes ages)
>>
>> I then apply the model predict to the new data frame.
>>
>> I am unsure how to then move the 'outmap' back across to grass.
>> How can i convert the df to a spatial grid object.
>>
>> Im thinking i must be doing something wrong. As it quite quick to pull
>> through the layers . But seems to take quite a lot of processing to get
>> the layers into a datframe appropppriate for applying the predictions.
>>
>> Any help would be greatly appreciated.
>>
>> Andy
>>
>>
>> # pull through environmental layers
>> # FAST
>> anmax <- readRAST6("anmax", ignore.stderr=TRUE)
>> anmin <- readRAST6("anmin", ignore.stderr=TRUE)
>> aspect <- readRAST6("aspect", ignore.stderr=TRUE)
>> dem10_lidar <- readRAST6("dem10_lidar", ignore.stderr=TRUE)

Wrong, do:

mydata <- readRAST6(c("anmax", "anmin", "aspect", "dem10_lidar"),
   ignore.stderr=TRUE)

then the data slot of the object is a data frame. Look at

summary(mydata)

for a sanity check.

>> # coerce to dataframe
>> # SLOW
>> mypred_anmaxDF<-as.data.frame(anmax)
>> mypred_anminDF<-as.data.frame(anmin)
>> mypred_aspectDF<-as.data.frame(aspect)
>> mypred_dem10_lidarDF<-as.data.frame(dem10_lidar)
>>
>> # merge into single dataframe
>> # VERY SLOW
>> merge_tmp<-merge(mypred_anmaxDF,mypred_anminDF)
>> rm(mypred_anmaxDF,mypred_anminDF)
>> merge_tmp1<-merge(merge_tmp,mypred_aspectDF)
>> rm(merge_tmp,mypred_aspectDF)
>> mypredDF<-merge(merge_tmp1,mypred_dem10_lidarDF)
>>
>> #apply model

What is tree? You may need to do extra steps depending on what class(tree)
says - if you have used rpart() or some such, you may find that

outmap <- predict(tree,newdata=mydata, type="class")

works,

or

outmap <- predict(tree,newdata=as(mydata, "data.frame"), type="class")

Maybe just assign into mydata straight away:

mydata$outmap <- predict(tree,newdata=as(mydata, "data.frame"),
   type="class")

given ?predict.rpart saying:

"If 'type="class"': (for a classification tree) a factor of
classifications based on the responses."

which looks like a vector for a vector response in the formula to rpart().
But do check what happens if there are NA in the newdata, because the
default predict() behaviour may be to drop those observations. Look at
summary(mydata).

Some formula-using model fitting functions just work, like lm() and
the predict() method for lm objects.

>From there, as Daniel wrote:

writeRAST6(mydata, "rpartpred", "outmap")

Hope this helps,

Roger

>> outmap <- predict(tree,newdata=mypredDF, type="class")
>>
>>

An article on this in the OSGeo newsletter might be a nice way to document
simple modeling examples with GRASS and R

D

--
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341

5

On Wed, 19 Dec 2007, Dylan Beaudette wrote:

On Wednesday 19 December 2007, Roger Bivand wrote:

On Wed, 19 Dec 2007, Daniel McInerney wrote:

The original questioner should have written to the grass-stats list in the

first place - thanks for CC-ing. See below for inline comments:

Hi Andy,

I am unsure how to then move the 'outmap' back across to grass.
How can i convert the df to a spatial grid object.

AFAIK, 'predict' won't create a dataframe object.
R should return FALSE for is.data.frame(outmap) and
TRUE for is.numeric(outmap)

You can slot the model output to the AttributeList of

It hasn't been an AttributeList for a long time - the data slot *is* a
data frame. We *always* need the output of sessionInfo() to help -
update.packages() is most often very helpful too.

one of the SpatialGridDataFrames that you created when
you read in a GRASS raster and then use writeRAST6 to
write it back to GRASS.

e.g. using 'anmax' from your example

anmax$anmax <- outmap
#if that doesn't work, you might try
anmax$anmax <- as.numeric(outmap)
writeRAST6(anmax, "NameOfNewGRASSRaster", "anmax")

Regards,
Daniel.

cc: grass-stats@lists.osgeo.org

andrew haywood wrote:

Dear List,

I am having some problems analysing some ecoligical models in grass
using the spgrass package through R.

I have 130 plot locations where i have observed presence/absence of a
species. I have followed a similar framework to the BUGSITE modelling
example from Markus's 2003 grass gis handouts (Grass 5) I have no
problems constructing the model based on the 130 plots and the
environmental layers from grass.

See the OSGeo tutorial September 2006:

http://www.foss4g2006.org/contributionDisplay.py?contribId=46&sessionId=59&
confId=1

and the OSGeo Journal note:

http://www.osgeo.org/files/journal/final_pdfs/OSGeo_vol1_GRASS-R.pdf

for more up-to-date information.

However, I am having problems bringing all the maps through into R so I
can make a prediction map.
The region isnt too large 1600 by 800 cells at 10m resolution
I can bring all the environmental layers through to R using readRAST6()
which doesnt take too much time at all.

However i assume I must convert the spatial grid objects into
dataframes to apply the predicted model function.

No, usually not at all, since the objects have a data.frame in the data
slot, and have the standard access methods.

So I then coerce them into dataframes using as.dataframe (this takes
ages) I then merge all the dataframes into a single dataframe. (this
takes ages)

I then apply the model predict to the new data frame.

I am unsure how to then move the 'outmap' back across to grass.
How can i convert the df to a spatial grid object.

Im thinking i must be doing something wrong. As it quite quick to pull
through the layers . But seems to take quite a lot of processing to get
the layers into a datframe appropppriate for applying the predictions.

Any help would be greatly appreciated.

Andy

# pull through environmental layers
# FAST
anmax <- readRAST6("anmax", ignore.stderr=TRUE)
anmin <- readRAST6("anmin", ignore.stderr=TRUE)
aspect <- readRAST6("aspect", ignore.stderr=TRUE)
dem10_lidar <- readRAST6("dem10_lidar", ignore.stderr=TRUE)

Wrong, do:

mydata <- readRAST6(c("anmax", "anmin", "aspect", "dem10_lidar"),
   ignore.stderr=TRUE)

then the data slot of the object is a data frame. Look at

summary(mydata)

for a sanity check.

# coerce to dataframe
# SLOW
mypred_anmaxDF<-as.data.frame(anmax)
mypred_anminDF<-as.data.frame(anmin)
mypred_aspectDF<-as.data.frame(aspect)
mypred_dem10_lidarDF<-as.data.frame(dem10_lidar)

# merge into single dataframe
# VERY SLOW
merge_tmp<-merge(mypred_anmaxDF,mypred_anminDF)
rm(mypred_anmaxDF,mypred_anminDF)
merge_tmp1<-merge(merge_tmp,mypred_aspectDF)
rm(merge_tmp,mypred_aspectDF)
mypredDF<-merge(merge_tmp1,mypred_dem10_lidarDF)

#apply model

What is tree? You may need to do extra steps depending on what class(tree)
says - if you have used rpart() or some such, you may find that

outmap <- predict(tree,newdata=mydata, type="class")

works,

or

outmap <- predict(tree,newdata=as(mydata, "data.frame"), type="class")

Maybe just assign into mydata straight away:

mydata$outmap <- predict(tree,newdata=as(mydata, "data.frame"),
   type="class")

given ?predict.rpart saying:

"If 'type="class"': (for a classification tree) a factor of
classifications based on the responses."

which looks like a vector for a vector response in the formula to rpart().
But do check what happens if there are NA in the newdata, because the
default predict() behaviour may be to drop those observations. Look at
summary(mydata).

Some formula-using model fitting functions just work, like lm() and
the predict() method for lm objects.

From there, as Daniel wrote:

writeRAST6(mydata, "rpartpred", "outmap")

Hope this helps,

Roger

outmap <- predict(tree,newdata=mypredDF, type="class")

An article on this in the OSGeo newsletter might be a nice way to document
simple modeling examples with GRASS and R

Perhaps, and then there is the section in chapter 10 in "Open Source GIS
A GRASS GIS Approach", 3rd edition (my copy is still on its way, but from the ToC, it looks as though pages 353-363 should be very helpful).

In fact, your site is a convenient collection of resources, I ought to have mentioned it in my reply!

Roger

D

--
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand@nhh.no

6

Thanks to all that replied.

everything works well now. THank you all for such an elegant way to model!!!
Much more fun to play with GRASS and a stats package at the same time.

I have just received my copy of Open Source GIS A GRASS GIS Approach", 3rd edition and at a first glance much of the issues are covered in Chapter 10.

Once again thanks.

Andy

On 12/20/07, Roger Bivand <Roger.Bivand@nhh.no> wrote:

On Wed, 19 Dec 2007, Dylan Beaudette wrote:

On Wednesday 19 December 2007, Roger Bivand wrote:

On Wed, 19 Dec 2007, Daniel McInerney wrote:

The original questioner should have written to the grass-stats list in the

first place - thanks for CC-ing. See below for inline comments:

Hi Andy,

I am unsure how to then move the ‘outmap’ back across to grass.
How can i convert the df to a spatial grid object.

AFAIK, ‘predict’ won’t create a dataframe object.
R should return FALSE for is.data.frame(outmap) and
TRUE for is.numeric(outmap)

You can slot the model output to the AttributeList of

It hasn’t been an AttributeList for a long time - the data slot is a
data frame. We always need the output of sessionInfo() to help -
update.packages() is most often very helpful too.

one of the SpatialGridDataFrames that you created when
you read in a GRASS raster and then use writeRAST6 to
write it back to GRASS.

e.g. using ‘anmax’ from your example

anmax$anmax ← outmap
#if that doesn’t work, you might try
anmax$anmax ← as.numeric(outmap)
writeRAST6(anmax, “NameOfNewGRASSRaster”, “anmax”)

Regards,
Daniel.

cc: grass-stats@lists.osgeo.org

andrew haywood wrote:

Dear List,

I am having some problems analysing some ecoligical models in grass
using the spgrass package through R.

I have 130 plot locations where i have observed presence/absence of a
species. I have followed a similar framework to the BUGSITE modelling
example from Markus’s 2003 grass gis handouts (Grass 5) I have no
problems constructing the model based on the 130 plots and the
environmental layers from grass.

See the OSGeo tutorial September 2006:

http://www.foss4g2006.org/contributionDisplay.py?contribId=46&sessionId=59&
confId=1

and the OSGeo Journal note:

http://www.osgeo.org/files/journal/final_pdfs/OSGeo_vol1_GRASS-R.pdf

for more up-to-date information.

However, I am having problems bringing all the maps through into R so I
can make a prediction map.
The region isnt too large 1600 by 800 cells at 10m resolution
I can bring all the environmental layers through to R using readRAST6()
which doesnt take too much time at all.

However i assume I must convert the spatial grid objects into
dataframes to apply the predicted model function.

No, usually not at all, since the objects have a data.frame in the data
slot, and have the standard access methods.

So I then coerce them into dataframes using as.dataframe (this takes
ages) I then merge all the dataframes into a single dataframe. (this
takes ages)

I then apply the model predict to the new data frame.

I am unsure how to then move the ‘outmap’ back across to grass.
How can i convert the df to a spatial grid object.

Im thinking i must be doing something wrong. As it quite quick to pull
through the layers . But seems to take quite a lot of processing to get
the layers into a datframe appropppriate for applying the predictions.

Any help would be greatly appreciated.

Andy

pull through environmental layers

FAST

anmax ← readRAST6(“anmax”, ignore.stderr=TRUE)
anmin ← readRAST6(“anmin”, ignore.stderr=TRUE)
aspect ← readRAST6(“aspect”, ignore.stderr=TRUE)
dem10_lidar ← readRAST6(“dem10_lidar”, ignore.stderr=TRUE)

Wrong, do:

mydata ← readRAST6(c(“anmax”, “anmin”, “aspect”, “dem10_lidar”),
ignore.stderr=TRUE)

then the data slot of the object is a data frame. Look at

summary(mydata)

for a sanity check.

coerce to dataframe

SLOW

mypred_anmaxDF<-as.data.frame(anmax)
mypred_anminDF<-as.data.frame(anmin)
mypred_aspectDF<-as.data.frame(aspect)
mypred_dem10_lidarDF<- as.data.frame(dem10_lidar)

merge into single dataframe

VERY SLOW

merge_tmp<-merge(mypred_anmaxDF,mypred_anminDF)
rm(mypred_anmaxDF,mypred_anminDF)
merge_tmp1<-merge(merge_tmp,mypred_aspectDF)
rm(merge_tmp,mypred_aspectDF)
mypredDF<-merge(merge_tmp1,mypred_dem10_lidarDF)

#apply model

What is tree? You may need to do extra steps depending on what class(tree)
says - if you have used rpart() or some such, you may find that

outmap ← predict(tree,newdata=mydata, type=“class”)

works,

or

outmap ← predict(tree,newdata=as(mydata, “data.frame”), type=“class”)

Maybe just assign into mydata straight away:

mydata$outmap ← predict(tree,newdata=as(mydata, “data.frame”),
type=“class”)

given ?predict.rpart saying:

“If ‘type=“class”’: (for a classification tree) a factor of
classifications based on the responses.”

which looks like a vector for a vector response in the formula to rpart().
But do check what happens if there are NA in the newdata, because the
default predict() behaviour may be to drop those observations. Look at
summary(mydata).

Some formula-using model fitting functions just work, like lm() and
the predict() method for lm objects.

From there, as Daniel wrote:

writeRAST6(mydata, “rpartpred”, “outmap”)

Hope this helps,

Roger

outmap ← predict(tree,newdata=mypredDF, type=“class”)

An article on this in the OSGeo newsletter might be a nice way to document
simple modeling examples with GRASS and R

Perhaps, and then there is the section in chapter 10 in “Open Source GIS
A GRASS GIS Approach”, 3rd edition (my copy is still on its way, but from
the ToC, it looks as though pages 353-363 should be very helpful).

In fact, your site is a convenient collection of resources, I ought to
have mentioned it in my reply!

Roger

D

–
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand@nhh.no

7

On Wednesday 19 December 2007, Roger Bivand wrote:

On Wed, 19 Dec 2007, Dylan Beaudette wrote:
> On Wednesday 19 December 2007, Roger Bivand wrote:
>> On Wed, 19 Dec 2007, Daniel McInerney wrote:
>>
>> The original questioner should have written to the grass-stats list in
>> the
>>
>> first place - thanks for CC-ing. See below for inline comments:
>>> Hi Andy,
>>>
>>>> I am unsure how to then move the 'outmap' back across to grass.
>>>> How can i convert the df to a spatial grid object.
>>>
>>> AFAIK, 'predict' won't create a dataframe object.
>>> R should return FALSE for is.data.frame(outmap) and
>>> TRUE for is.numeric(outmap)
>>>
>>> You can slot the model output to the AttributeList of
>>
>> It hasn't been an AttributeList for a long time - the data slot *is* a
>> data frame. We *always* need the output of sessionInfo() to help -
>> update.packages() is most often very helpful too.
>>
>>> one of the SpatialGridDataFrames that you created when
>>> you read in a GRASS raster and then use writeRAST6 to
>>> write it back to GRASS.
>>>
>>> e.g. using 'anmax' from your example
>>>
>>> anmax$anmax <- outmap
>>> #if that doesn't work, you might try
>>> anmax$anmax <- as.numeric(outmap)
>>> writeRAST6(anmax, "NameOfNewGRASSRaster", "anmax")
>>>
>>> Regards,
>>> Daniel.
>>>
>>> cc: grass-stats@lists.osgeo.org
>>>
>>> andrew haywood wrote:
>>>> Dear List,
>>>>
>>>> I am having some problems analysing some ecoligical models in grass
>>>> using the spgrass package through R.
>>>>
>>>> I have 130 plot locations where i have observed presence/absence of a
>>>> species. I have followed a similar framework to the BUGSITE modelling
>>>> example from Markus's 2003 grass gis handouts (Grass 5) I have no
>>>> problems constructing the model based on the 130 plots and the
>>>> environmental layers from grass.
>>
>> See the OSGeo tutorial September 2006:
>>
>> http://www.foss4g2006.org/contributionDisplay.py?contribId=46&sessionId=
>>59& confId=1
>>
>> and the OSGeo Journal note:
>>
>> http://www.osgeo.org/files/journal/final_pdfs/OSGeo_vol1_GRASS-R.pdf
>>
>> for more up-to-date information.
>>
>>>> However, I am having problems bringing all the maps through into R so
>>>> I can make a prediction map.
>>>> The region isnt too large 1600 by 800 cells at 10m resolution
>>>> I can bring all the environmental layers through to R using
>>>> readRAST6() which doesnt take too much time at all.
>>>>
>>>> However i assume I must convert the spatial grid objects into
>>>> dataframes to apply the predicted model function.
>>
>> No, usually not at all, since the objects have a data.frame in the data
>> slot, and have the standard access methods.
>>
>>>> So I then coerce them into dataframes using as.dataframe (this takes
>>>> ages) I then merge all the dataframes into a single dataframe. (this
>>>> takes ages)
>>>>
>>>> I then apply the model predict to the new data frame.
>>>>
>>>> I am unsure how to then move the 'outmap' back across to grass.
>>>> How can i convert the df to a spatial grid object.
>>>>
>>>> Im thinking i must be doing something wrong. As it quite quick to
>>>> pull through the layers . But seems to take quite a lot of processing
>>>> to get the layers into a datframe appropppriate for applying the
>>>> predictions.
>>>>
>>>> Any help would be greatly appreciated.
>>>>
>>>> Andy
>>>>
>>>>
>>>> # pull through environmental layers
>>>> # FAST
>>>> anmax <- readRAST6("anmax", ignore.stderr=TRUE)
>>>> anmin <- readRAST6("anmin", ignore.stderr=TRUE)
>>>> aspect <- readRAST6("aspect", ignore.stderr=TRUE)
>>>> dem10_lidar <- readRAST6("dem10_lidar", ignore.stderr=TRUE)
>>
>> Wrong, do:
>>
>> mydata <- readRAST6(c("anmax", "anmin", "aspect", "dem10_lidar"),
>> ignore.stderr=TRUE)
>>
>> then the data slot of the object is a data frame. Look at
>>
>> summary(mydata)
>>
>> for a sanity check.
>>
>>>> # coerce to dataframe
>>>> # SLOW
>>>> mypred_anmaxDF<-as.data.frame(anmax)
>>>> mypred_anminDF<-as.data.frame(anmin)
>>>> mypred_aspectDF<-as.data.frame(aspect)
>>>> mypred_dem10_lidarDF<-as.data.frame(dem10_lidar)
>>>>
>>>> # merge into single dataframe
>>>> # VERY SLOW
>>>> merge_tmp<-merge(mypred_anmaxDF,mypred_anminDF)
>>>> rm(mypred_anmaxDF,mypred_anminDF)
>>>> merge_tmp1<-merge(merge_tmp,mypred_aspectDF)
>>>> rm(merge_tmp,mypred_aspectDF)
>>>> mypredDF<-merge(merge_tmp1,mypred_dem10_lidarDF)
>>>>
>>>> #apply model
>>
>> What is tree? You may need to do extra steps depending on what
>> class(tree) says - if you have used rpart() or some such, you may find
>> that
>>
>> outmap <- predict(tree,newdata=mydata, type="class")
>>
>> works,
>>
>> or
>>
>> outmap <- predict(tree,newdata=as(mydata, "data.frame"), type="class")
>>
>> Maybe just assign into mydata straight away:
>>
>> mydata$outmap <- predict(tree,newdata=as(mydata, "data.frame"),
>> type="class")
>>
>> given ?predict.rpart saying:
>>
>> "If 'type="class"': (for a classification tree) a factor of
>> classifications based on the responses."
>>
>> which looks like a vector for a vector response in the formula to
>> rpart(). But do check what happens if there are NA in the newdata,
>> because the default predict() behaviour may be to drop those
>> observations. Look at summary(mydata).
>>
>> Some formula-using model fitting functions just work, like lm() and
>> the predict() method for lm objects.
>>
>>> From there, as Daniel wrote:
>>
>> writeRAST6(mydata, "rpartpred", "outmap")
>>
>> Hope this helps,
>>
>> Roger
>>
>>>> outmap <- predict(tree,newdata=mypredDF, type="class")
>
> An article on this in the OSGeo newsletter might be a nice way to
> document simple modeling examples with GRASS and R

Perhaps, and then there is the section in chapter 10 in "Open Source GIS
A GRASS GIS Approach", 3rd edition (my copy is still on its way, but from
the ToC, it looks as though pages 353-363 should be very helpful).

In fact, your site is a convenient collection of resources, I ought to
have mentioned it in my reply!

Roger

Hi Roger,

I hadn't looked over the contents of the new GRASS book. Perhaps I can
convince my adviser to pony up for it. Otherwise I will wait until our local
library has it.

I will post some examples on my website as well- the more examples the better!

Cheers,

Dylan

--
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341

8

On Wednesday 19 December 2007, andrew haywood wrote:

Thanks to all that replied.

everything works well now. THank you all for such an elegant way to
model!!! Much more fun to play with GRASS and a stats package at the same
time.

I have just received my copy of Open Source GIS A GRASS GIS Approach", 3rd
edition and at a first glance much of the issues are covered in Chapter 10.

Once again thanks.

Andy

Glad to hear that you are making progress. Here is one other example
illustrating some common GRASS-R-GRASS operations with the Spearfish data.

Cheers,

Dylan

On 12/20/07, Roger Bivand <Roger.Bivand@nhh.no> wrote:
> On Wed, 19 Dec 2007, Dylan Beaudette wrote:
> > On Wednesday 19 December 2007, Roger Bivand wrote:
> >> On Wed, 19 Dec 2007, Daniel McInerney wrote:
> >>
> >> The original questioner should have written to the grass-stats list in
>
> the
>
> >> first place - thanks for CC-ing. See below for inline comments:
> >>> Hi Andy,
> >>>
> >>>> I am unsure how to then move the 'outmap' back across to grass.
> >>>> How can i convert the df to a spatial grid object.
> >>>
> >>> AFAIK, 'predict' won't create a dataframe object.
> >>> R should return FALSE for is.data.frame(outmap) and
> >>> TRUE for is.numeric(outmap)
> >>>
> >>> You can slot the model output to the AttributeList of
> >>
> >> It hasn't been an AttributeList for a long time - the data slot *is* a
> >> data frame. We *always* need the output of sessionInfo() to help -
> >> update.packages() is most often very helpful too.
> >>
> >>> one of the SpatialGridDataFrames that you created when
> >>> you read in a GRASS raster and then use writeRAST6 to
> >>> write it back to GRASS.
> >>>
> >>> e.g. using 'anmax' from your example
> >>>
> >>> anmax$anmax <- outmap
> >>> #if that doesn't work, you might try
> >>> anmax$anmax <- as.numeric(outmap)
> >>> writeRAST6(anmax, "NameOfNewGRASSRaster", "anmax")
> >>>
> >>> Regards,
> >>> Daniel.
> >>>
> >>> cc: grass-stats@lists.osgeo.org
> >>>
> >>> andrew haywood wrote:
> >>>> Dear List,
> >>>>
> >>>> I am having some problems analysing some ecoligical models in grass
> >>>> using the spgrass package through R.
> >>>>
> >>>> I have 130 plot locations where i have observed presence/absence of
>
> a
>
> >>>> species. I have followed a similar framework to the BUGSITE
>
> modelling
>
> >>>> example from Markus's 2003 grass gis handouts (Grass 5) I have no
> >>>> problems constructing the model based on the 130 plots and the
> >>>> environmental layers from grass.
> >>
> >> See the OSGeo tutorial September 2006:
>
> http://www.foss4g2006.org/contributionDisplay.py?contribId=46&sessionId=5
>9&
>
> >> confId=1
> >>
> >> and the OSGeo Journal note:
> >>
> >> http://www.osgeo.org/files/journal/final_pdfs/OSGeo_vol1_GRASS-R.pdf
> >>
> >> for more up-to-date information.
> >>
> >>>> However, I am having problems bringing all the maps through into R
>
> so I
>
> >>>> can make a prediction map.
> >>>> The region isnt too large 1600 by 800 cells at 10m resolution
> >>>> I can bring all the environmental layers through to R using
>
> readRAST6()
>
> >>>> which doesnt take too much time at all.
> >>>>
> >>>> However i assume I must convert the spatial grid objects into
> >>>> dataframes to apply the predicted model function.
> >>
> >> No, usually not at all, since the objects have a data.frame in the
> >> data slot, and have the standard access methods.
> >>
> >>>> So I then coerce them into dataframes using as.dataframe (this
> >>>> takes ages) I then merge all the dataframes into a single dataframe.
> >>>> (this takes ages)
> >>>>
> >>>> I then apply the model predict to the new data frame.
> >>>>
> >>>> I am unsure how to then move the 'outmap' back across to grass.
> >>>> How can i convert the df to a spatial grid object.
> >>>>
> >>>> Im thinking i must be doing something wrong. As it quite quick to
>
> pull
>
> >>>> through the layers . But seems to take quite a lot of processing to
>
> get
>
> >>>> the layers into a datframe appropppriate for applying the
>
> predictions.
>
> >>>> Any help would be greatly appreciated.
> >>>>
> >>>> Andy
> >>>>
> >>>>
> >>>> # pull through environmental layers
> >>>> # FAST
> >>>> anmax <- readRAST6("anmax", ignore.stderr=TRUE)
> >>>> anmin <- readRAST6("anmin", ignore.stderr=TRUE)
> >>>> aspect <- readRAST6("aspect", ignore.stderr=TRUE)
> >>>> dem10_lidar <- readRAST6("dem10_lidar", ignore.stderr=TRUE)
> >>
> >> Wrong, do:
> >>
> >> mydata <- readRAST6(c("anmax", "anmin", "aspect", "dem10_lidar"),
> >> ignore.stderr=TRUE)
> >>
> >> then the data slot of the object is a data frame. Look at
> >>
> >> summary(mydata)
> >>
> >> for a sanity check.
> >>
> >>>> # coerce to dataframe
> >>>> # SLOW
> >>>> mypred_anmaxDF<-as.data.frame(anmax)
> >>>> mypred_anminDF<-as.data.frame(anmin)
> >>>> mypred_aspectDF<-as.data.frame(aspect)
> >>>> mypred_dem10_lidarDF<-as.data.frame(dem10_lidar)
> >>>>
> >>>> # merge into single dataframe
> >>>> # VERY SLOW
> >>>> merge_tmp<-merge(mypred_anmaxDF,mypred_anminDF)
> >>>> rm(mypred_anmaxDF,mypred_anminDF)
> >>>> merge_tmp1<-merge(merge_tmp,mypred_aspectDF)
> >>>> rm(merge_tmp,mypred_aspectDF)
> >>>> mypredDF<-merge(merge_tmp1,mypred_dem10_lidarDF)
> >>>>
> >>>> #apply model
> >>
> >> What is tree? You may need to do extra steps depending on what
>
> class(tree)
>
> >> says - if you have used rpart() or some such, you may find that
> >>
> >> outmap <- predict(tree,newdata=mydata, type="class")
> >>
> >> works,
> >>
> >> or
> >>
> >> outmap <- predict(tree,newdata=as(mydata, "data.frame"),
>
> type="class")
>
> >> Maybe just assign into mydata straight away:
> >>
> >> mydata$outmap <- predict(tree,newdata=as(mydata, "data.frame"),
> >> type="class")
> >>
> >> given ?predict.rpart saying:
> >>
> >> "If 'type="class"': (for a classification tree) a factor of
> >> classifications based on the responses."
> >>
> >> which looks like a vector for a vector response in the formula to
>
> rpart().
>
> >> But do check what happens if there are NA in the newdata, because the
> >> default predict() behaviour may be to drop those observations. Look at
> >> summary(mydata).
> >>
> >> Some formula-using model fitting functions just work, like lm() and
> >> the predict() method for lm objects.
> >>
> >>> From there, as Daniel wrote:
> >>
> >> writeRAST6(mydata, "rpartpred", "outmap")
> >>
> >> Hope this helps,
> >>
> >> Roger
> >>
> >>>> outmap <- predict(tree,newdata=mypredDF, type="class")
> >
> > An article on this in the OSGeo newsletter might be a nice way to
>
> document
>
> > simple modeling examples with GRASS and R
>
> Perhaps, and then there is the section in chapter 10 in "Open Source GIS
> A GRASS GIS Approach", 3rd edition (my copy is still on its way, but from
> the ToC, it looks as though pages 353-363 should be very helpful).
>
> In fact, your site is a convenient collection of resources, I ought to
> have mentioned it in my reply!
>
> Roger
>
> > D
>
> --
> Roger Bivand
> Economic Geography Section, Department of Economics, Norwegian School of
> Economics and Business Administration, Helleveien 30, N-5045 Bergen,
> Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
> e-mail: Roger.Bivand@nhh.no

--
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341

9

Hi all,

is there a simple way to join some attribute data
in a comma separated text file to an existing GRASS
vector map?

The data in the CSV file does not have any coordinates
but a primary key field that also exists in the polygon
map it needs to be joined to.

ArcGIS 9.2 seems to be able to do this easily, so surely
there must be a way to do it in GRASS, as well?

If I were to write a little GRASS module for this
purpose, what would be the smartest way to go about
it?

Thanks,

Benjamin

--
Benjamin Ducke, M.A.
Archäoinformatik
(Archaeoinformation Science)
Institut für Ur- und Frühgeschichte
(Inst. of Prehistoric and Historic Archaeology)
Christian-Albrechts-Universität zu Kiel
Johanna-Mestorf-Straße 2-6
D 24098 Kiel
Germany

Tel.: ++49 (0)431 880-3378 / -3379
Fax : ++49 (0)431 880-7300
www.uni-kiel.de/ufg

10

That was my first guess, too.
However, v.db.join does not support the DBF backend, according
to the manual page.

Is there an alternative way?

Thanks,

Benjamin

Tomas Lanczos wrote:

Benjamin Ducke wrote:

Hi all,

is there a simple way to join some attribute data
in a comma separated text file to an existing GRASS
vector map?

The data in the CSV file does not have any coordinates
but a primary key field that also exists in the polygon
map it needs to be joined to.

ArcGIS 9.2 seems to be able to do this easily, so surely
there must be a way to do it in GRASS, as well?

If I were to write a little GRASS module for this
purpose, what would be the smartest way to go about
it?

Try to convert the CSV file to DBF table somehow (e.g. in OOo Calc) and
then connect the table to the vector.

regards

Tomas

Thanks,

Benjamin

--
Benjamin Ducke, M.A.
Archäoinformatik
(Archaeoinformation Science)
Institut für Ur- und Frühgeschichte
(Inst. of Prehistoric and Historic Archaeology)
Christian-Albrechts-Universität zu Kiel
Johanna-Mestorf-Straße 2-6
D 24098 Kiel
Germany

Tel.: ++49 (0)431 880-3378 / -3379
Fax : ++49 (0)431 880-7300
www.uni-kiel.de/ufg

11

On 07/01/08 14:07, Benjamin Ducke wrote:

Hi all,

is there a simple way to join some attribute data
in a comma separated text file to an existing GRASS
vector map?

Option A (creates a separate database file - but not possible with dbf driver)

1) use rdbms as data backend (including for the map you want the data to be connected to)
2) db.in.ogr
3) create view combining map attribute table with new table from csv-file (using key)
4) v.db.connect -o map table=NewView

Option B (add content of csv file to existing map attribute column - possible with dbf driver):

- identify names and types of columns in csv file
- v.db.addcol the relevant columns
- for each line in csv file: v.db.update

(all this can also be done low-level in C, but don't think that the performance enhancement will be so great).

Moritz

12

On 07/01/08 14:36, Benjamin Ducke wrote:

That was my first guess, too.
However, v.db.join does not support the DBF backend, according
to the manual page.

Looking at it quickly, I think that it should be possible to make v.db.join support dbf if you use the v.db.update. Something like this:

very roughly (supposing there is no header line):

NumberofFields=`awk -F',' '{print NF}' brol.text`

names=get column names in first line

[some way of determining column types]

for i in (1-NumberOfNewColumns):
     v.db.addcol mymap col='$names[i] $type[i]

for all lines in csv file (except first):
    for (i=1; i<=NumberOfNewColumns;i++):
       v.db.update mygeodetic_pts col=col$i value=`cut -d',' -f$i brol.text` \
         where=cat=`cut -d',' -f1 brol.text`

where brol.text contains
14429,2,3,4

This could probably be done more elegantly, but in any case it should be possible.

Moritz

13

On Jan 7, 2008 2:36 PM, Benjamin Ducke <benjamin.ducke@ufg.uni-kiel.de> wrote:

That was my first guess, too.
However, v.db.join does not support the DBF backend, according
to the manual page.

Unfortunately true - but DBF is not an SQL DB system. No chance
that you use SQLite? So nice, so fast...

Best
Markus

14

Thanks Markus and Moritz for your input.

I will look into your suggestions.
The reason I need this information is that a colleague
of mine is preparing some simple GIS tutorials where he
wants to outline some basic data processing.

He wants to have a text for ArcGIS and another for
GRASS (and QGIS), both showing the same operations.
It is meant for GIS beginners.

So the problem is that an optimal solution is not so
much about performance or flexibility (which can be
had with SQLite, no doubt) but showing people that
there are straight-forward solutions for doing things
in GRASS that they are accustomed to in ArcGIS.

The solution for this needs to be simple and should
not involve too much data conversion or the use of
external tools and drivers that may not be present
in the GRASS/QGIS download bundle ...

Best,

Benjamin

Markus Neteler wrote:

On Jan 7, 2008 2:36 PM, Benjamin Ducke <benjamin.ducke@ufg.uni-kiel.de> wrote:

That was my first guess, too.
However, v.db.join does not support the DBF backend, according
to the manual page.

Unfortunately true - but DBF is not an SQL DB system. No chance
that you use SQLite? So nice, so fast...

Best
Markus

--
Benjamin Ducke, M.A.
Archäoinformatik
(Archaeoinformation Science)
Institut für Ur- und Frühgeschichte
(Inst. of Prehistoric and Historic Archaeology)
Christian-Albrechts-Universität zu Kiel
Johanna-Mestorf-Straße 2-6
D 24098 Kiel
Germany

Tel.: ++49 (0)431 880-3378 / -3379
Fax : ++49 (0)431 880-7300
www.uni-kiel.de/ufg

15

Benjamin,

On Jan 8, 2008 9:58 AM, Benjamin Ducke <benjamin.ducke@ufg.uni-kiel.de> wrote:
...

So the problem is that an optimal solution is not so
much about performance or flexibility (which can be
had with SQLite, no doubt) but showing people that
there are straight-forward solutions for doing things
in GRASS that they are accustomed to in ArcGIS.

Well, with SQLite it is straightforward (nothing to learn, just
use it instead of DBF).

The solution for this needs to be simple and should
not involve too much data conversion or the use of
external tools and drivers that may not be present
in the GRASS/QGIS download bundle ...

The simple steps are
- install SQLite (it does not need setup but works out of
  the box)
- set the DBMI settings with db.connect (see manual page
  for copy-paste example)
- use v.db.join
- use g.copy or db.copy to easily transform from DBF (or whatever
  to SQLite). These two modules do all the work.

Markus

16

Hello,

I will look into your suggestions.
The reason I need this information is that a colleague
of mine is preparing some simple GIS tutorials where he
wants to outline some basic data processing.

Please give us a feedback to the list about what you found
to be the most simple and straight forward way.

The solution for this needs to be simple and should
not involve too much data conversion or the use of
external tools and drivers that may not be present
in the GRASS/QGIS download bundle ...

This sentence should be highlighted boldly.
A lot of advise/tips here on the list and in the wiki may be helpful to have a
flexible solution of the problem. But in most cases, user *just* want to do the
task the things the most /simple/ and straight forwards way.
The workers time is more costly than computing speed or
performance because good computers are available.

So, Benjamins collegue is in a difficult position:
Explaining unser why to use a different tool set (GRASS & QGIS) which is in
advanced use cases (automatisation, scripting, integration) more powerful
while users are looking in a easy and straight forward way to do common GIS
tasks quickly and with a short learing curve.
If I may guess, students would be better off with something like gvSIG which
offers more usability to beginners.
When they adavnce they could switch to GRASS...

But still, please post your solution.

Kind regards,
Timmie

17

Thanks all for your suggestions. This is getting to be
a very fruitful discussion.

I would agree with the proposed enhancements to v.db.join.
This seems straight-forward and coherent with GRASS usage
GIVEN THAT there is a DBF file as a departure point.

In many cases, however, users will simply start with a
CSV text file and converting that to DBF can be non-trivial
once the file exceeds 64K lines or so and the spreadsheet
application chokes on it.

So how about also extending v.in.ascii slightly to accept
input lines without coordinate pairs and create a plain
DBF table with user-defined field formats?

It should also not be too hard to add an option that
would guess the correct formats for the user.

In that case, pulling in attribute data from a text file
would amount to using v.in.ascii and v.db.join only, which
I think should not give anyone too many headaches.

Would that make any sense?

Benjamin

mlennert@geog-pc40.ulb.ac.be wrote:

On Tue, January 8, 2008 12:11, Markus Neteler wrote:

The simple steps are
- install SQLite (it does not need setup but works out of
  the box)
- set the DBMI settings with db.connect (see manual page
  for copy-paste example)

add one step here: db.in.ogr

- use v.db.join
- use g.copy or db.copy to easily transform from DBF (or whatever
  to SQLite). These two modules do all the work.

But I agree with Benjamin that while dbf is still the default data
backend, we should allow users to use it.

I don't think that rewriting v.db.join to include the capacity of joining
dbf files is that difficult (using a combination of v.db.select and
v.db.update instead of the sql join currently implemented).

So, I would probably go down the route of db.in.ogr + a rewrite of
v.db.join, which essentially means replacing the following line:

echo "UPDATE $maptable SET $col=(SELECT $col
        FROM $GIS_OPT_otable WHERE
$GIS_OPT_otable.$GIS_OPT_ocolumn=$maptable.$GIS_OPT_column);" |
db.execute
.

with something like (completely untested):

for key in `v.db.select $GIS_OPT_MAP col=$GIS_OPT_column | sort -n -u`
  do
     v.db.update $maptable col=$col value=`echo "SELECT $col
        FROM $GIS_OPT_otable WHERE $GIS_OPT_otable.$GIS_OPT_ocolumn=$key"
| db.select table=$GIS_OPT_otable database=$GIS_OPT_odatabase`
where="$maptable.$GIS_OPT_column=$key"
  done

where $GIS_OPT_odatabase = new command line parameter to introduce which
would allow to join two tables in two different databases.
If the key could be a string column, the sort -n is not correct, so you
probably have to cater to different possibilities (or program a 'select
distinct()' for the dbf driver).
You probably have to check for multiple lines being returned by the SELECT
statement if the new table has multiple entries per key, or you just do
this with a sort statement.

Moritz

Moritz

--
Benjamin Ducke, M.A.
Archäoinformatik
(Archaeoinformation Science)
Institut für Ur- und Frühgeschichte
(Inst. of Prehistoric and Historic Archaeology)
Christian-Albrechts-Universität zu Kiel
Johanna-Mestorf-Straße 2-6
D 24098 Kiel
Germany

Tel.: ++49 (0)431 880-3378 / -3379
Fax : ++49 (0)431 880-7300
www.uni-kiel.de/ufg

18

On Tue, January 8, 2008 13:04, Benjamin Ducke wrote:

Thanks all for your suggestions. This is getting to be
a very fruitful discussion.

I would agree with the proposed enhancements to v.db.join.
This seems straight-forward and coherent with GRASS usage
GIVEN THAT there is a DBF file as a departure point.

no: db.in.ogr with the cvs driver, but this entails that the user create a
.cvst file with column definitions.

You could maybe patch the ogr cvs driver to allow automatic type recognition.

So how about also extending v.in.ascii slightly to accept
input lines without coordinate pairs and create a plain
DBF table with user-defined field formats?

+1. I like that better.

It should also not be too hard to add an option that
would guess the correct formats for the user.

It's all done in v.in.ascii. So it should be enough to add an 'only create
table' flag and then conditionalise everything concerning the map creation
on this flag not being set...

In that case, pulling in attribute data from a text file
would amount to using v.in.ascii and v.db.join only, which
I think should not give anyone too many headaches.

now it's db.in.ogr + v.db.join, but with the need for the .cvst file...

Moritz

19

On Jan 8, 2008 1:58 PM, Moritz Lennert <mlennert@club.worldonline.be> wrote:

On Tue, January 8, 2008 13:04, Benjamin Ducke wrote:

...

no: db.in.ogr with the cvs driver, but this entails that the user create a
.cvst file with column definitions.

You could maybe patch the ogr cvs driver to allow automatic type recognition.

Grab that as C code here from my old home page :slight_smile:

http://mpa.fbk.eu/markus/cvs2sql/

Markus

20

Markus Neteler wrote:

On Jan 8, 2008 1:58 PM, Moritz Lennert <mlennert@club.worldonline.be> wrote:

On Tue, January 8, 2008 13:04, Benjamin Ducke wrote:

...

no: db.in.ogr with the cvs driver, but this entails that the user create a
.cvst file with column definitions.

You could maybe patch the ogr cvs driver to allow automatic type recognition.

Grab that as C code here from my old home page :slight_smile:

http://mpa.fbk.eu/markus/cvs2sql/

Great! Looks like some hacking in the evening ...

Question is:

Patch the actual OGR sources or the stuff in grass6/db/drivers/ogr or
the db.in.ogr module itself? Where would be the best place to put the
type recognition?

Benjamin

Markus
_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

--
Benjamin Ducke, M.A.
Archäoinformatik
(Archaeoinformation Science)
Institut für Ur- und Frühgeschichte
(Inst. of Prehistoric and Historic Archaeology)
Christian-Albrechts-Universität zu Kiel
Johanna-Mestorf-Straße 2-6
D 24098 Kiel
Germany

Tel.: ++49 (0)431 880-3378 / -3379
Fax : ++49 (0)431 880-7300
www.uni-kiel.de/ufg