[GRASS-user] Tips for setting up an new FOSS-GEO-linux-box

GRASS GRASS user
1

I know, I know... it's off the lists topic! But I need some help.
Probably I am going to "migrate" soon all of my stuff in a new
linux-box. So I would like to collect best practice tips concerning the
following:

OS:
OK, I think I shouldn't ask about which OS since foss runs on everything
(right?). But are I am curious to know if there are any advantages using
Debian instead of Ubuntu for example?

Filesystems:
Which filesystem is better(=safer/faster) for data storage? Is there any
important advantage to choose XFS for example rather than ext3?

Partitions:
Do you keep your geo-data in a separate partition? I suppose yes. Have
you split further your partition based on other criteria, always related
with "working with geospatial data"?

Do you keep all of your source code in a separate partition maybe?

Organisation:
GRASS takes care to organise the data inside the GIS data-base and its
fantastic. But what about the "raw" data? How do you organise them?
Manually everything? Any tool to be more productive?

BackUp:
How often do you backup your data? Do you just copy or do you compress
as well? What is safer?

Other:
Any other important issues when setting-up a new foss-geo-box?

Thank you, Nikos

P.S. Maybe we can add a new wiki-page if something useful comes out of
this thread. Or maybe not... :slight_smile:

2

On Wed, Oct 29, 2008 at 7:08 AM, Nikos Alexandris
<nikos.alexandris@felis.uni-freiburg.de> wrote:

I know, I know... it's off the lists topic! But I need some help.
Probably I am going to "migrate" soon all of my stuff in a new
linux-box. So I would like to collect best practice tips concerning the
following:

OS:
OK, I think I shouldn't ask about which OS since foss runs on everything
(right?). But are I am curious to know if there are any advantages using
Debian instead of Ubuntu for example?

I use Debian/Unstable -- but several aspects of Ubuntu are appealing.

Filesystems:
Which filesystem is better(=safer/faster) for data storage? Is there any
important advantage to choose XFS for example rather than ext3?

I have heard a lot of good things about XFS, although I use ext3 for
several of my partitions. Several of our servers in the department use
XFS for everything, and we have had good success with them over the
last 4 years. I would steer clear of ReiserFS.

Partitions:
Do you keep your geo-data in a separate partition? I suppose yes. Have
you split further your partition based on other criteria, always related
with "working with geospatial data"?

I like putting all of my GRASS data on a seperate disk-- so that IO
intensive operations on GIS data do not slow my machine to a crawl.
Alternatively a scratch disk for the /tmp/ folder might be nice too.

Do you keep all of your source code in a separate partition maybe?

I just keep mine in my home folder.

Organisation:
GRASS takes care to organise the data inside the GIS data-base and its
fantastic. But what about the "raw" data? How do you organise them?
Manually everything? Any tool to be more productive?

I keep non-GRASS GIS data in a folder of its own - 'gis_data' . No
fancy cataloging, just the file system. Something like ArcCatalog
(without all of the cruft) would be interesting. gdalinfo / ogrinfo
are used to check on coordinate system details.

BackUp:
How often do you backup your data? Do you just copy or do you compress
as well? What is safer?

Every afternoon-- using rsync.

Other:
Any other important issues when setting-up a new foss-geo-box?

make sure all of the development packages get installed before
compiling from source. This can usually be sorted out by running
./configure and looking for errors. make sure that there is no overlap
between what the package manager has installed and things you plan to
install from source.

Thank you, Nikos

P.S. Maybe we can add a new wiki-page if something useful comes out of
this thread. Or maybe not... :slight_smile:

That would be helpful.

Cheers,

Dylan

3

OS:
OK, I think I shouldn't ask about which OS since foss runs on everything
(right?). But are I am curious to know if there are any advantages using
Debian instead of Ubuntu for example?

Everyone has their own favorite; I've been using Ubuntu and Grass together
since 2004, with no difficulties on 32-bit and 64-bit machines.

Filesystems:
Which filesystem is better(=safer/faster) for data storage? Is there any
important advantage to choose XFS for example rather than ext3?

Not sure about the main differences/advantages of either; I've been using ext3
since forever, with no regrets.

Partitions:
Do you keep your geo-data in a separate partition? I suppose yes. Have
you split further your partition based on other criteria, always related
with "working with geospatial data"?

Pretty much the main 3-4 projects I'm working on are on /home, with anything I
haven't worked on in the last 2-weeks backed up and archived on an external
2TB hardrive. (LaCie)

Do you keep all of your source code in a separate partition maybe?

Nope, just under good old /usr/local. I only compile from scratch those applications where
I need all the bleeding edge goodies and bug fixes, which, for me, is only Grass,
gdal, and lilypond. I try to use the distribution's packages for everything else;
makes it a lot easier to maintain using the package manager than chasing around
and recompiling source for a ton of apps.

Organisation:
GRASS takes care to organise the data inside the GIS data-base and its
fantastic. But what about the "raw" data? How do you organise them?
Manually everything? Any tool to be more productive?

Once raw data is imported into Grass, I usually get it off my hard drive and backed
up onto something external, in case my computer melts down; then I can always
rebuild from scratch. Of course the external drive could also melt down. I guess
a RAID would be even better, but costs more.

BackUp:
How often do you backup your data? Do you just copy or do you compress
as well? What is safer?

I've been using 'tar cjvf' for each project, but that is becoming unmanageable; I need
to migrate to a versioning system as Dylan has done with rsync. At least for the
projects I work on all the time. The old stuff can probably stay on the backup drive
in tar.bz2 format.

Other:
Any other important issues when setting-up a new foss-geo-box?

I just installed Ubuntu Intrepid 8.10 yesterday, and found it much easier
installing packages via Synaptic rather than downloading the bleeding edge
source packages and compiling. The only source package I had to compile was
Grass.

Thank you, Nikos

P.S. Maybe we can add a new wiki-page if something useful comes out of
this thread. Or maybe not... :slight_smile:

_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

4

On Wed, Oct 29, 2008 at 6:46 PM, Patton, Eric <epatton@nrcan.gc.ca> wrote:

OS:
OK, I think I shouldn't ask about which OS since foss runs on everything
(right?). But are I am curious to know if there are any advantages using
Debian instead of Ubuntu for example?

Everyone has their own favorite; I've been using Ubuntu and Grass together
since 2004, with no difficulties on 32-bit and 64-bit machines.

Happy Mandriva user here :slight_smile:
Also Scientific Linux user on our cluster.

...

Do you keep all of your source code in a separate partition maybe?

Nope, just under good old /usr/local. I only compile from scratch those applications where
I need all the bleeding edge goodies

I don't even install but run it directly with a link from
grass64/dist.x86_64-unknown-linux-gnu/

Markus

5

On Wed, 2008-10-29 at 15:08 +0100, Nikos Alexandris wrote:

I know, I know... it's off the lists topic! But I need some help.
Probably I am going to "migrate" soon all of my stuff in a new
linux-box. So I would like to collect best practice tips concerning the
following:

OS:
OK, I think I shouldn't ask about which OS since foss runs on everything
(right?). But are I am curious to know if there are any advantages using
Debian instead of Ubuntu for example?

Filesystems:
Which filesystem is better(=safer/faster) for data storage? Is there any
important advantage to choose XFS for example rather than ext3?

Partitions:
Do you keep your geo-data in a separate partition? I suppose yes. Have
you split further your partition based on other criteria, always related
with "working with geospatial data"?

Do you keep all of your source code in a separate partition maybe?

Organisation:
GRASS takes care to organise the data inside the GIS data-base and its
fantastic. But what about the "raw" data? How do you organise them?
Manually everything? Any tool to be more productive?

BackUp:
How often do you backup your data? Do you just copy or do you compress
as well? What is safer?

Other:
Any other important issues when setting-up a new foss-geo-box?

Thank you, Nikos

P.S. Maybe we can add a new wiki-page if something useful comes out of
this thread. Or maybe not... :slight_smile:

Dylan, Eric and Markus,
thank you for your precious time to respond.

Since there are already 3 answers then maybe some wiki-page can be
set-up out-of it. I'll try to structure the information a bit and open a
page whenever I find some free-time (possibly this weekend).

Please to all GRASS-users following the list, if you have some spare
time and consider these questions important, participate with your
ideas/tips/hints/tricks.

Kind regards, Nikos

6

Dylan Beaudette wrote:

I like putting all of my GRASS data on a seperate disk-- so that IO
intensive operations on GIS data do not slow my machine to a crawl.
Alternatively a scratch disk for the /tmp/ folder might be nice too.

note that GRASS puts its processing temp files in $MAPSET/.tmp/, only a
few small session files are stored in /tmp/.

Hamish

7

On Wed, Oct 29, 2008 at 3:08 PM, Nikos Alexandris
<nikos.alexandris@felis.uni-freiburg.de> wrote:

Filesystems:
Which filesystem is better(=safer/faster) for data storage? Is there any
important advantage to choose XFS for example rather than ext3?

I am using ext3 (wth 4k block size) for a long time.

Partitions:
Do you keep your geo-data in a separate partition?

I have a directory for all original geodata files and one for
grassdata/.
Both are exported and shared through NFS, so that the
team members can easily access from their machines.
While the geodata/ directory is read-only (to avoid that
elaborated data creep into the original files), the
grassdata/ directory is write enabled. Like this, thanks to
the concept of GRASS data management, we can all
work in the same locations (but personal mapsets).

Markus (who looks forward to the Wiki page on this topic :slight_smile:

8

[...]

Please to all GRASS-users following the list, if you have some spare
time and consider these questions important, participate with your
ideas/tips/hints/tricks.

Kind regards, Nikos

Interesting filesystem-related information here:
http://www.debian-administration.org/articles/388
http://linuxgazette.net/122/TWDT.html#piszcz

9

On Wed, 2008-10-29 at 07:19 -0700, Dylan Beaudette wrote:

> Partitions:
> Do you keep your geo-data in a separate partition? I suppose yes.
Have
> you split further your partition based on other criteria, always
related
> with "working with geospatial data"?

I like putting all of my GRASS data on a seperate disk-- so that IO
intensive operations on GIS data do not slow my machine to a crawl.
Alternatively a scratch disk for the /tmp/ folder might be nice too.

Dylan, could you please explain the "scratch disk"?

Thank you, Nikos

10

On Wed, 2008-10-29 at 15:08 +0100, Nikos Alexandris wrote:

P.S. Maybe we can add a new wiki-page if something useful comes out of
this thread. Or maybe not... :slight_smile:

I whink an new page in http://wiki.osgeo.org/wiki/Main_Page is better
than in the GRASS-wiki.

What do you think for "Tips for setting up a Linux Operating System /
GIS Workstation" as a title? Since I am "forced" to speak greeklish very
frequently some language check is never bad to have.

If I am not wrong, the title is not possible to modify in a wiki (?), so
it's better to define it correctly

Thank you, Nikos

P.S. Anyway, here is an initial idea for the structure
---

Operating System/ Distribution
Are there any advantages to choose a specific linux distribution?

Filesystems
Which filesystem is better(=safer/faster)?

* According to some benchmarks [1] the XFS filesystem seems to perform
an overall highscore.

Partitions / Organisation

1. Geospatial data
* Keep your geospatial data in a separate partition

* Even further, using another hard disk drive than the main (the one in
which the operating system is installed) to save the GIS database, will
help... (Dylan, did I get this right, placing the data in another
partition on another hard disk than the "main hd", secures the resources
from being consumed 100% which the machine to crawl when under heavy
gis-processing... ?)

* Easy data exporting/sharing and multi-user access can be achieved
using NFS

2. Free Open Source Geospatial Applications (Software)
Where do you keep source codes?
* All in one directory
(+++add more)

BackUp
How often do you back up your data?
How do you back up your data?
* simple method: tar cjvf
* syncing method: using rsync

Other important issues
(+++add more)

11

On Sat, Nov 1, 2008 at 6:52 PM, Nikos Alexandris
<nikos.alexandris@felis.uni-freiburg.de> wrote:

On Wed, 2008-10-29 at 07:19 -0700, Dylan Beaudette wrote:

> Partitions:
> Do you keep your geo-data in a separate partition? I suppose yes.
Have
> you split further your partition based on other criteria, always
related
> with "working with geospatial data"?

I like putting all of my GRASS data on a seperate disk-- so that IO
intensive operations on GIS data do not slow my machine to a crawl.
Alternatively a scratch disk for the /tmp/ folder might be nice too.

Dylan, could you please explain the "scratch disk"?

Thank you, Nikos

Right- I was thinking about an extra disk used for temporary
operations, such as GRASS temp files, etc. It could very well be the
same disk that GRASS Data is stored on, but separate from the disk
where the operating system resides. In this way I/O-bound operations
on GRASS data would not impair the use of other programs.

Dylan

12

I've started the attempt to collect in a wiki-page tips for setting up a
GIS workstation.
http://wiki.osgeo.org/wiki/GIS_workstation_setup_tips

It certainly needs modifications.

Kind regards, Nikos

13

That is really cool. Thanks for putting this together!

Mark

On Nov 2, 2008, at 9:13 PM, Nikos Alexandris <nikos.alexandris@felis.uni-freiburg.de > wrote:

I've started the attempt to collect in a wiki-page tips for setting up a
GIS workstation.
http://wiki.osgeo.org/wiki/GIS_workstation_setup_tips

It certainly needs modifications.

Kind regards, Nikos

_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

14

Hi,

I have some other tips for the setting up of a box. I came as a new user to both GRASS and Linux at the same time so perhaps these tips would help people in a similar situation.

Instructions to install a load of GIS software on Ubuntu can be found here:

http://www.perrygeo.net/wordpress/?p=119

If you are working on a dual boot system, with Windows NTFS and Linux, and the main data are stored on the NTFS partition, then you need to mount it with yourself as the owner to be able to work on it in GRASS. The following entry in the /etc/fstab works:

/dev/sda3 /media/Windows ntfs-3g umask=0002,uid=YOUR_LOGIN,gid=YOUR_GROUP,allow_other 0 0

I got most of my packages from the repositories, but installed the following from source:

gstat (http://www.gstat.org/) - Calculate variograms and interpolate by kriging
liblas (http://liblas.org/) - Read and write LiDAR data in LAS format

Cheers

John

15

John Stevenson wrote:

liblas (http://liblas.org/) - Read and write LiDAR data in LAS format

Hi,

if you are using las2txt with v.lidar.*, v.in.ascii, or r.in.xyz could you
add some tips to the wiki about it:

  http://grass.osgeo.org/wiki/LIDAR

(or whatever other lidar+GRASS suggestions you might have)

if you need a wiki account you can easily make yourself one.

thanks,
Hamish