[GRASS-user] Very high resolution topographic map of Europe: need help and advices

Felix_Schalck · August 10, 2009, 6:28pm

Hello,

For some time now, I'm following sort of a personal objective to
create the most precise (=high resolution) topographic map of Europe
allowed by my comp (Xubuntu 9.04, AMD64, 3Gb RAM, 300+ HD space).
Starting from the very scratch, I had to learn about DEMs and
GIS/maptools - and I'm still not confortable with all the technicy
behind. Fact is that the best data set available to serve my purpose
seems to be the cgiar interpolated srtm3 geodata (no license problem
here, since it is aimed for pure personal use) which has to somehow be
translated into a big jpeg or png file.

I started with GMT, and tried to render the whole aera between 34N and
60N, 11W and 35E with grdimage into a 6000x8000 pix [I made slices of
3°x46°, which I pasted together]. Given the amount of data, GMT softs
take forever, but it seems to be the fate of this project to hit the
limits of my hardware... The big problem happened to be the river
data, since GSHHS provides only limited amount and precision of
side-rivers, which resulted in chains of straight lines scattered all
over a giant map, even after vectorization: it was a no-go.

And then, a few days ago, I discovered nasa SWBD data and WMAP0, which
seem to be of much higher resolution, linked to a nice GRASS GIS
tutorial on the french wikipedia. I immedialely dug into this new
software, quite complicated I must admit, but very powerful. I figured
out how to import GeoTiff data into GRASS Raster files, and how to
display/export each one of them, but soon had to face new problems,
especially when tried to reproject the data into LCC projection. So I
decided to ask for help on this mailing list.

My Current plan is to:
1. reproject the cgiar raster tiles OR one big merged raster into LCC
projection (native srtm data seems to be a strange Mercator)
2. create elevation shadings
3. export a nice shaded topographic png
4. extract rivers/coast vectors from SWBD files
5. workout in Inkscape
6. paste the whole thing together in GIMP.

The main problem right now seems to be the "tiling" of the topographic
data. Each cgiar-cell (5°x5°) can be shown into a separate layer, but
I'm unable to work them together. And it looks like most of the tools
provided by GRASS assume the raster map is the size of the location
(r.proj for an example). I tried r.patch but it produce wired results
on top of giant files (I stopped when I hit the 2Gb limit), perhaps
because the cgiar tiles do not exactly match my region (11W 35E 60N
34N). Is there a way to cut-off the overlapping tiles ? And than I'm
not sure the r.patch way is the right one, since the single resulting
file will become really big.

I took the time to explain the whole thing with the hope to not only
get help about my immediate raster division problem, but also about
the "big-picture", eg: is GRASS the right tools to do this (I
installed GRASS because of the SWBD data tutorial, but it seems to me
that swbd could also be plotted via psxy in GMT )? How would you
GIS-gurus proceed to create a high resolution topographic map ? Is one
big png a good idea, or would it be smarter to divide the continent
into 4 or more parts, and render each one into a separate png ?

Thanks you very much for your time,

Felix Schalck

neteler · August 11, 2009, 8:50pm

Hi Felix,

On Mon, Aug 10, 2009 at 8:28 PM, Felix Schalck<felix.schalck@gmail.com> wrote:

Hello,

For some time now, I'm following sort of a personal objective to
create the most precise (=high resolution) topographic map of Europe
allowed by my comp (Xubuntu 9.04, AMD64, 3Gb RAM, 300+ HD space).
Starting from the very scratch, I had to learn about DEMs and
GIS/maptools - and I'm still not confortable with all the technicy
behind. Fact is that the best data set available to serve my purpose
seems to be the cgiar interpolated srtm3 geodata (no license problem
here, since it is aimed for pure personal use) which has to somehow be
translated into a big jpeg or png file.

Here you can use gdalwarp to merge all files into a big one. Enjoy
wildcards to do that in one line:

gdalwarp srtm_*.tif cgiar_srtm3_LL.tif

A big file is created. Then don't miss to add image pyramids:

gdaladdo srtm3_LL.tif 2 4 8 16 32 64

and you can open a file of several GB in no time with QGIS for
example.

I posted some GDAL tricks here:
http://gfoss.blogspot.com/2008/06/gdal-raster-data-tips-and-tricks.html

(if you want OGR for vectors, check:
http://gfoss.blogspot.com/2008/06/ogr-vector-data-tips-and-tricks.html
)

...

The big problem happened to be the river
data, since GSHHS provides only limited amount and precision of
side-rivers, which resulted in chains of straight lines scattered all
over a giant map, even after vectorization: it was a no-go.

No idea, I didn't try GMT so much.

And then, a few days ago, I discovered nasa SWBD data and WMAP0, which
seem to be of much higher resolution, linked to a nice GRASS GIS
tutorial on the french wikipedia. I immedialely dug into this new
software, quite complicated I must admit, but very powerful.

We hope you took the GRASS 6.4 version - even if still called "release
candidate" it is used by many in production. Myself, I am doing heavy
computations with it and it is so far the only convincing software
for GIS number crunching

I figured out how to import GeoTiff data into GRASS Raster files,

Hint: from GRASS 6.4 onwards you can use r.external to avoid
import but to just link an external file into GRASS. Nice when
registering 30GB of orthophotos in a few minutes...

For newcomers QGIS is a nice interface to GRASS, too, since
it comes with a GRASS toolbox.

and how to
display/export each one of them, but soon had to face new problems,
especially when tried to reproject the data into LCC projection. So I
decided to ask for help on this mailing list.

My Current plan is to:
1. reproject the cgiar raster tiles OR one big merged raster into LCC
projection (native srtm data seems to be a strange Mercator)

They are in LatLong but they (those I downloaded) don't/didn't have
the projection info in the metadata any more. Sad story. Maybe they
fixed it later...

Note that recent GDAL now provides the gdalhillshade tool for
easy hillshading of GeoTIFF/other format files.

Here my script (I did the same):

---------- snip ----------
#!/bin/sh
# mosaic European CGIAR SRTM to 250m LAEA DEM map, bilinear

gdalwarp -s_srs epsg:4326 -t_srs epsg:3035 -tr 250 250 -r bilinear \
srtm_*.tif europe_all_srtmV4_cgiar_250m_LAEA_EU.tif

gdaladdo europe_all_srtmV4_cgiar_250m_LAEA_EU.tif 2 4 8 16 32 64

# generate shaded relief
gdalhillshade europe_all_srtmV4_cgiar_250m_LAEA_EU.tif \
europe_all_srtmV4_cgiar_250m_LAEA_EU_shaded.tif
gdaladdo europe_all_srtmV4_cgiar_250m_LAEA_EU_shaded.tif 2 4 8 16 32 64

---------- snap ----------

3. export a nice shaded topographic png

I would still use GDAL here and gdal_translate to a PNG (do
you really need this?)

4. extract rivers/coast vectors from SWBD files

At this point use r.external on the huge SRTM DEM GeoTIFF.
Then use r.terraflow/r.mapcalc/... (several options).

5. workout in Inkscape

Oh, at this point you'll like r.out.png, you can set previously
the resolution to something less (g.region) unless you don't
plan to print a huge poster.

6. paste the whole thing together in GIMP.

Why not ...

The main problem right now seems to be the "tiling" of the topographic
data. Each cgiar-cell (5°x5°) can be shown into a separate layer, but
I'm unable to work them together.

See above: gdalwarp is your friend.

And it looks like most of the tools
provided by GRASS assume the raster map is the size of the location
(r.proj for an example).

No, that's no the case. It's flexible.

I tried r.patch but it produce wired results
on top of giant files (I stopped when I hit the 2Gb limit),

Ah, so you need to compile with Large File Support (LFS):
./configure ... --enable-largefile

Then you can have huge files > 2GB (if it fails somewhere please
tell us but most common commands should be fine)

...

I took the time to explain the whole thing with the hope to not only
get help about my immediate raster division problem, but also about
the "big-picture", eg: is GRASS the right tools to do this

My take is: GRASS together with GDAL tools are just perfect.

(I installed GRASS because of the SWBD data tutorial, but it seems to me
that swbd could also be plotted via psxy in GMT )? How would you
GIS-gurus proceed to create a high resolution topographic map ?

Please take a look at ps.map in GRASS.

Is one big png a good idea, or would it be smarter to divide the continent
into 4 or more parts, and render each one into a separate png ?

It will depends on the memory management of the programs you use.
In GRASS, some users are using multi-GB files, so that works. Likewise
for GDAL. I don't know for GMT nor for Inkscape/Gimp.

Cheers
Markus

--
Markus Neteler
Foundation Edmund Mach (FEM) - Research and Innovation Centre
Environment and Natural Resources Area
Head of GIS and Remote Sensing Unit, Trento, Italy
Web: http://gis.fem-environment.eu/

Felix_Schalck · August 12, 2009, 2:06pm

Dear Markus,

Thank you soooo much for your response; it provided me with valuables
hints - but also with new questions.

First, following your avice, I finally decided to compile the new
GRASS6.4rc5. Having an amd64 cpu, there war no binary available; so I
had to do the job myself. Let's face it: manual configuration is a
pain for a newcommer like me, but once the ./configure script shows
the long awaited final recap, the make command - although quite long -
runs withouth a hitch. I reminds me of good ol' freebsd days, although
we had a lot more troubles finishing compilation without errors: great
job guys ! But the resulat was worth it; I don't know if it is the
work you've done for the last few years, or the gcc optiomization
flags (or perhaps both of it), but the resulting grass64 runs like a
fireball compared to my old 6.2 bin provided by the Ubuntu repos.
Again: great job ! Only now I fully understand what you meant by "it
is so far the only convincing software for GIS number crunching". But
let's go back to the topic: my high-resolution topographic map of
Europe.

Thanks to your advices, the production outline has changed to this:

1. Merge the cgiar TIFs (AND NOT THE GRASS RASTERS IF I GOT THIS
CORRECTLY) thanks to gdalwrap command. In what projection does this
command work ? Is it possible to wrap the TIF directlly in my lcc
projection ?

2. Add image pyramids with gdaladdo (< I frankly do not underdand this
one at all; what does it mean ?)

3. Shaded relief: I don't know your gdalhillshade command at all. Does
it produce the same results as the r.shaded.relief I was planning to
use ? Can you set the illumination angle with gdalhilshade ?

4. Re-gdaladdo for the shaded tif.

5. Import in GRASS and checkout results. If I'm right, I will have two
layers at this point, one with the relief colors, one with the
shadings.

6. Coastlines and Rivers. I was planning to use SWBD(coastlines and
main rivers) and VLMAP0 Data (for the secondary rivers). Here again,
you provide me with a complete new set of tools:
r.external/r.terraflow/r.mapcalc/. What is the general idea behind ?I
checked the man pages, but I don't really understand how to use them
for my purpose. My plan was to import the shapefile into the right
projection with rvin.ogr, and than export svg files for rework BEFORE
joining the river layer and the topographic layers; but perhaps your
way, once I understand it, is more efficient.

7. Export the shaded topography with r.out.png in one big png. Do I
need two files (one containing the shadings, one with the relief
colors) ?

8. Merge topography and hydrography layers in GIMP.

Could you (or anyone interested) please have a look a this ?

Again: thanks you very much for your time,

regards,
Felix

For some time now, I'm following sort of a personal objective to
create the most precise (=high resolution) topographic map of Europe
allowed by my comp (Xubuntu 9.04, AMD64, 3Gb RAM, 300+ HD space).
Starting from the very scratch, I had to learn about DEMs and
GIS/maptools - and I'm still not confortable with all the technicy
behind. Fact is that the best data set available to serve my purpose
seems to be the cgiar interpolated srtm3 geodata (no license problem
here, since it is aimed for pure personal use) which has to somehow be
translated into a big jpeg or png file.

Here you can use gdalwarp to merge all files into a big one. Enjoy
wildcards to do that in one line:

gdalwarp srtm_*.tif cgiar_srtm3_LL.tif

A big file is created. Then don't miss to add image pyramids:

gdaladdo srtm3_LL.tif 2 4 8 16 32 64

and you can open a file of several GB in no time with QGIS for
example.

I posted some GDAL tricks here:
http://gfoss.blogspot.com/2008/06/gdal-raster-data-tips-and-tricks.html

(if you want OGR for vectors, check:
http://gfoss.blogspot.com/2008/06/ogr-vector-data-tips-and-tricks.html
)

...

The big problem happened to be the river
data, since GSHHS provides only limited amount and precision of
side-rivers, which resulted in chains of straight lines scattered all
over a giant map, even after vectorization: it was a no-go.

No idea, I didn't try GMT so much.

And then, a few days ago, I discovered nasa SWBD data and WMAP0, which
seem to be of much higher resolution, linked to a nice GRASS GIS
tutorial on the french wikipedia. I immedialely dug into this new
software, quite complicated I must admit, but very powerful.

We hope you took the GRASS 6.4 version - even if still called "release
candidate" it is used by many in production. Myself, I am doing heavy
computations with it and it is so far the only convincing software
for GIS number crunching

I figured out how to import GeoTiff data into GRASS Raster files,

Hint: from GRASS 6.4 onwards you can use r.external to avoid
import but to just link an external file into GRASS. Nice when
registering 30GB of orthophotos in a few minutes...

For newcomers QGIS is a nice interface to GRASS, too, since
it comes with a GRASS toolbox.

and how to
display/export each one of them, but soon had to face new problems,
especially when tried to reproject the data into LCC projection. So I
decided to ask for help on this mailing list.

My Current plan is to:
1. reproject the cgiar raster tiles OR one big merged raster into LCC
projection (native srtm data seems to be a strange Mercator)

They are in LatLong but they (those I downloaded) don't/didn't have
the projection info in the metadata any more. Sad story. Maybe they
fixed it later...

Note that recent GDAL now provides the gdalhillshade tool for
easy hillshading of GeoTIFF/other format files.

Here my script (I did the same):

---------- snip ----------
#!/bin/sh
# mosaic European CGIAR SRTM to 250m LAEA DEM map, bilinear

gdalwarp -s_srs epsg:4326 -t_srs epsg:3035 -tr 250 250 -r bilinear \
srtm_*.tif europe_all_srtmV4_cgiar_250m_LAEA_EU.tif

gdaladdo europe_all_srtmV4_cgiar_250m_LAEA_EU.tif 2 4 8 16 32 64

# generate shaded relief
gdalhillshade europe_all_srtmV4_cgiar_250m_LAEA_EU.tif \
europe_all_srtmV4_cgiar_250m_LAEA_EU_shaded.tif
gdaladdo europe_all_srtmV4_cgiar_250m_LAEA_EU_shaded.tif 2 4 8 16 32 64

---------- snap ----------

3. export a nice shaded topographic png

I would still use GDAL here and gdal_translate to a PNG (do
you really need this?)

4. extract rivers/coast vectors from SWBD files

At this point use r.external on the huge SRTM DEM GeoTIFF.
Then use r.terraflow/r.mapcalc/... (several options).

5. workout in Inkscape

Oh, at this point you'll like r.out.png, you can set previously
the resolution to something less (g.region) unless you don't
plan to print a huge poster.

6. paste the whole thing together in GIMP.

Why not ...

The main problem right now seems to be the "tiling" of the topographic
data. Each cgiar-cell (5°x5°) can be shown into a separate layer, but
I'm unable to work them together.

See above: gdalwarp is your friend.

And it looks like most of the tools
provided by GRASS assume the raster map is the size of the location
(r.proj for an example).

No, that's no the case. It's flexible.

I tried r.patch but it produce wired results
on top of giant files (I stopped when I hit the 2Gb limit),

Ah, so you need to compile with Large File Support (LFS):
./configure ... --enable-largefile

Then you can have huge files > 2GB (if it fails somewhere please
tell us but most common commands should be fine)

...

I took the time to explain the whole thing with the hope to not only
get help about my immediate raster division problem, but also about
the "big-picture", eg: is GRASS the right tools to do this

My take is: GRASS together with GDAL tools are just perfect.

(I installed GRASS because of the SWBD data tutorial, but it seems to me
that swbd could also be plotted via psxy in GMT )? How would you
GIS-gurus proceed to create a high resolution topographic map ?

Please take a look at ps.map in GRASS.

Is one big png a good idea, or would it be smarter to divide the continent
into 4 or more parts, and render each one into a separate png ?

It will depends on the memory management of the programs you use.
In GRASS, some users are using multi-GB files, so that works. Likewise
for GDAL. I don't know for GMT nor for Inkscape/Gimp.

Cheers
Markus

--
Markus Neteler
Foundation Edmund Mach (FEM) - Research and Innovation Centre
Environment and Natural Resources Area
Head of GIS and Remote Sensing Unit, Trento, Italy
Web: http://gis.fem-environment.eu/

neteler · August 12, 2009, 2:31pm

On Wed, Aug 12, 2009 at 4:06 PM, Felix Schalck<felix.schalck@gmail.com> wrote:

Dear Markus,

Thank you soooo much for your response; it provided me with valuables
hints - but also with new questions.

Sure: things at that level aren't naturally obvious.

First, following your avice, I finally decided to compile the new
GRASS6.4rc5. Having an amd64 cpu, there war no binary available; so I
had to do the job myself. Let's face it: manual configuration is a
pain for a newcommer like me,

You mean to find all dependences?
We have collection some help in the Wiki:
http://grass.osgeo.org/wiki/Compile_and_Install

but once the ./configure script shows
the long awaited final recap, the make command - although quite long -

If you have a dual/multi-core CPU, you can use
make -j2
make -j4
or so to be much faster (I always use make -j4).

runs withouth a hitch. I reminds me of good ol' freebsd days, although
we had a lot more troubles finishing compilation without errors: great
job guys ! But the resulat was worth it; I don't know if it is the
work you've done for the last few years, or the gcc optiomization
flags (or perhaps both of it), but the resulting grass64 runs like a
fireball compared to my old 6.2 bin provided by the Ubuntu repos.
Again: great job !

Fantastic!

Only now I fully understand what you meant by "it
is so far the only convincing software for GIS number crunching". But
let's go back to the topic: my high-resolution topographic map of
Europe.

Thanks to your advices, the production outline has changed to this:

1. Merge the cgiar TIFs (AND NOT THE GRASS RASTERS IF I GOT THIS
CORRECTLY) thanks to gdalwrap command. In what projection does this
command work ? Is it possible to wrap the TIF directlly in my lcc
projection ?

I tried that yesterday and did NOT have luck. I would do it two-pass,
even if it consumes twice as much space temporaneously:

a) gdalwarp: all into one file, keeping the projection (mosaicking)
b) gdalwarp: reproject to LCC (note that there are EU LCC and others).

Use your preferred resampling method (gdalwarp offers several).

Perhaps I got something wrong and you can do it in one step as well.

2. Add image pyramids with gdaladdo (< I frankly do not underdand this
one at all; what does it mean ?)

OK, it means that map copies at lower to much lower resolution are
stored in the GeoTIFF (or different format which supports pyramids)
file. When then opening with QGIS, Mapserver etc, it takes advantage
of that and speeds up in an impressive way the visualization.
Of course the file size increases.

Example: I take "world natural earth 250m" which is huge; to cover
Europe you need to download 4 tiles = 8 GB. I added pyramids with
gdaladdo and now I am able to open these huge 4 tiles in no time
in one step with QGIS. It's a convenience.

3. Shaded relief: I don't know your gdalhillshade command at all. Does
it produce the same results as the r.shaded.relief I was planning to
use ? Can you set the illumination angle with gdalhilshade ?

Yes.
I realise that it is called "gdaldem" now.
http://trac.osgeo.org/gdal/browser/trunk/gdal/apps/gdaldem.cpp
http://gdal.org/gdaldem.html

4. Re-gdaladdo for the shaded tif.

yes.

5. Import in GRASS and checkout results. If I'm right, I will have two
layers at this point, one with the relief colors, one with the
shadings.

Right (say, one is the relief [colors are optional], the other the shadings).
You can use d.his or r.his to make a nice shaded colorized terrain
map, something like this:
http://grass.osgeo.org/grass60/screenshots/images/etopo2_grass_laea_9_48N_0E.jpg

6. Coastlines and Rivers. I was planning to use SWBD(coastlines and
main rivers) and VLMAP0 Data (for the secondary rivers). Here again,
you provide me with a complete new set of tools:
r.external/r.terraflow/r.mapcalc/. What is the general idea behind ?

r.external you would use in step 5. Instead of r.in.gdal.

r.terraflow/r.mapcalc you may forget since I didn't understand that you
would take the rivers as vector maps (I thought you wanted to extract
them from the DEM).

I checked the man pages, but I don't really understand how to use them
for my purpose. My plan was to import the shapefile into the right
projection with rvin.ogr,

v.in.ogr (or v.external).

and than export svg files for rework BEFORE
joining the river layer and the topographic layers; but perhaps your
way, once I understand it, is more efficient.

Not sure (since I was confused :).

7. Export the shaded topography with r.out.png in one big png. Do I
need two files (one containing the shadings, one with the relief
colors) ?

If you do the color shading in GRASS, then you need to export
only one file.

8. Merge topography and hydrography layers in GIMP.

Yes, sounds reasonable.

Please consider to document your steps in the GRASS Wiki,
it could be useful in future for others.

Good luck
Markus

Felix_Schalck · August 12, 2009, 4:21pm

2009/8/12 Markus Neteler <neteler@osgeo.org>:

On Wed, Aug 12, 2009 at 4:06 PM, Felix Schalck<felix.schalck@gmail.com> wrote:

Dear Markus,

Thanks to your advices, the production outline has changed to this:

1. Merge the cgiar TIFs (AND NOT THE GRASS RASTERS IF I GOT THIS
CORRECTLY) thanks to gdalwrap command. In what projection does this
command work ? Is it possible to wrap the TIF directlly in my lcc
projection ?

I tried that yesterday and did NOT have luck. I would do it two-pass,
even if it consumes twice as much space temporaneously:

a) gdalwarp: all into one file, keeping the projection (mosaicking)
b) gdalwarp: reproject to LCC (note that there are EU LCC and others).

Use your preferred resampling method (gdalwarp offers several).

Perhaps I got something wrong and you can do it in one step as well.

I immediately tried this, and ran into following problem:

$gdalwarp srtm_*.tif europe_all_srtmV4_cgiar_default.tif

Creating output file that is 60000P x 36000L.
ERROR 6: A 60000 pixels x 36000 lines x 1 bands Int16 image would be
larger than 4GB
but this is the largest size a TIFF can be. Creation failed.

If I understand this correctly, I don't have a choice here, and must
reproject the whole thing while pasting it. So I computed this
command:

$gdalwarp -s_srs epsg:4326 -t_srs "+proj=lcc +lat_1=47 +lat_2=41
+lat_0=53 +lon_0=12 +x_0=22.00000 +y_0=15.00000 +ellps=WSG84 +units=m
+no_defs" -tr 93 93 -r bilinear srtm_*.tif
europe_all_srtmV4_cgiar_93m_LCC.tif

It seems to work (at least a 3.6Gb TIF file is created in the right
directory), but takes FOREVER (eg. the 2 first tiles took about 50min,
and I have 56 tiles to be merged...). Strange thing is that neither
the cpu nor the RAM is being used at full extend. The r.patch tool
provided by GRASS went much faster. Any idea here to speed up things ?
Would lowering the resolution (186m would be an option) help ?

2. Add image pyramids with gdaladdo (< I frankly do not underdand this
one at all; what does it mean ?)

OK, it means that map copies at lower to much lower resolution are
stored in the GeoTIFF (or different format which supports pyramids)
file. When then opening with QGIS, Mapserver etc, it takes advantage
of that and speeds up in an impressive way the visualization.
Of course the file size increases.

Example: I take "world natural earth 250m" which is huge; to cover
Europe you need to download 4 tiles = 8 GB. I added pyramids with
gdaladdo and now I am able to open these huge 4 tiles in no time
in one step with QGIS. It's a convenience.

Thank you very much for the clear explanation. Indeed, this seems a
very nice option.

3. Shaded relief: I don't know your gdalhillshade command at all. Does
it produce the same results as the r.shaded.relief I was planning to
use ? Can you set the illumination angle with gdalhilshade ?

Yes.
I realise that it is called "gdaldem" now.
http://trac.osgeo.org/gdal/browser/trunk/gdal/apps/gdaldem.cpp
http://gdal.org/gdaldem.html

4. Re-gdaladdo for the shaded tif.

yes.

5. Import in GRASS and checkout results. If I'm right, I will have two
layers at this point, one with the relief colors, one with the
shadings.

Right (say, one is the relief [colors are optional], the other the shadings).
You can use d.his or r.his to make a nice shaded colorized terrain
map, something like this:
http://grass.osgeo.org/grass60/screenshots/images/etopo2_grass_laea_9_48N_0E.jpg

Nice one: leads to a lot of graphical ideas concerning the final map...

6. Coastlines and Rivers. I was planning to use SWBD(coastlines and
main rivers) and VLMAP0 Data (for the secondary rivers). Here again,
you provide me with a complete new set of tools:
r.external/r.terraflow/r.mapcalc/. What is the general idea behind ?

r.external you would use in step 5. Instead of r.in.gdal.

r.terraflow/r.mapcalc you may forget since I didn't understand that you
would take the rivers as vector maps (I thought you wanted to extract
them from the DEM).

Didn't even think such things were possible, but now that you mention
it, what would be your advice ? Using wmap0 set - with its errors - to
get the rivers, or try to extract them from the DEM ? Which one would
produce the best results (=at 93 or 186m resolution) ?

I checked the man pages, but I don't really understand how to use them
for my purpose. My plan was to import the shapefile into the right
projection with rvin.ogr,

v.in.ogr (or v.external).

and than export svg files for rework BEFORE
joining the river layer and the topographic layers; but perhaps your
way, once I understand it, is more efficient.

Not sure (since I was confused :).

7. Export the shaded topography with r.out.png in one big png. Do I
need two files (one containing the shadings, one with the relief
colors) ?

If you do the color shading in GRASS, then you need to export
only one file.

8. Merge topography and hydrography layers in GIMP.

Yes, sounds reasonable.

Please consider to document your steps in the GRASS Wiki,
it could be useful in future for others.

Do you mean writing a tutorial about creating my map ? Never though it
would be able catch readers... But if you seriously think Its worth
it, why not.

Good luck
Markus

Thanks again for your help,

Felix

neteler · August 12, 2009, 10:35pm

On Wed, Aug 12, 2009 at 6:21 PM, Felix Schalck<felix.schalck@gmail.com> wrote:

2009/8/12 Markus Neteler <neteler@osgeo.org>:

On Wed, Aug 12, 2009 at 4:06 PM, Felix Schalck<felix.schalck@gmail.com> wrote:

Dear Markus,

Thanks to your advices, the production outline has changed to this:

1. Merge the cgiar TIFs (AND NOT THE GRASS RASTERS IF I GOT THIS
CORRECTLY) thanks to gdalwrap command. In what projection does this
command work ? Is it possible to wrap the TIF directlly in my lcc
projection ?

I tried that yesterday and did NOT have luck. I would do it two-pass,
even if it consumes twice as much space temporaneously:

a) gdalwarp: all into one file, keeping the projection (mosaicking)
b) gdalwarp: reproject to LCC (note that there are EU LCC and others).

Use your preferred resampling method (gdalwarp offers several).

Perhaps I got something wrong and you can do it in one step as well.

I immediately tried this, and ran into following problem:

$gdalwarp srtm_*.tif europe_all_srtmV4_cgiar_default.tif

Creating output file that is 60000P x 36000L.
ERROR 6: A 60000 pixels x 36000 lines x 1 bands Int16 image would be
larger than 4GB
but this is the largest size a TIFF can be. Creation failed.

If I understand this correctly, I don't have a choice here,

We are lucky, you have a choice

http://www.gdal.org/frmt_gtiff.html
-> BIGTIFF=YES/NO/IF_NEEDED/IF_SAFER: Control whether the created file
is a BigTIFF or a classic TIFF.

and must reproject the whole thing while pasting it. So I computed this
command:

$gdalwarp -s_srs epsg:4326 -t_srs "+proj=lcc +lat_1=47 +lat_2=41
+lat_0=53 +lon_0=12 +x_0=22.00000 +y_0=15.00000 +ellps=WSG84 +units=m
+no_defs" -tr 93 93 -r bilinear srtm_*.tif
europe_all_srtmV4_cgiar_93m_LCC.tif

It seems to work (at least a 3.6Gb TIF file is created in the right
directory), but takes FOREVER (eg. the 2 first tiles took about 50min,
and I have 56 tiles to be merged...). Strange thing is that neither
the cpu nor the RAM is being used at full extend.

This is best asked on the gdal-dev list (I am currently running a
mosaicking of 1700 DEM tiles, it runs for 10 days already...).

The r.patch tool provided by GRASS went much faster.

Cool!

Any idea here to speed up things ?
Would lowering the resolution (186m would be an option) help ?

Perhaps but still all input data have to be read.
Still best asked on the GDAL mailing list. Maybe someone else here
knows more.

...

6. Coastlines and Rivers. I was planning to use SWBD(coastlines and
main rivers) and VLMAP0 Data (for the secondary rivers). Here again,
you provide me with a complete new set of tools:
r.external/r.terraflow/r.mapcalc/. What is the general idea behind ?

r.external you would use in step 5. Instead of r.in.gdal.

r.terraflow/r.mapcalc you may forget since I didn't understand that you
would take the rivers as vector maps (I thought you wanted to extract
them from the DEM).

Didn't even think such things were possible, but now that you mention
it, what would be your advice ? Using wmap0 set - with its errors - to
get the rivers, or try to extract them from the DEM ? Which one would
produce the best results (=at 93 or 186m resolution) ?

Good question. I am afraid that you need to experiment (but you can
do that in a small map portion). Please keep us informed about your
results.
I recently used the rivers from OpenStreetMap for making a map,
in my current study area they are quite complete. You can grab
OSM as SHAPE file from

http://download.geofabrik.de/osm/europe/

...

Please consider to document your steps in the GRASS Wiki,
it could be useful in future for others.

Do you mean writing a tutorial about creating my map ? Never though it
would be able catch readers... But if you seriously think Its worth
it, why not.

It is moreover to document you experience - since it is a Wiki, more
ideas may come in over the time.

cheers
Markus