[GRASS5] Re: r.thin

GRASS GRASS developer
1

Hi Markus,
  probably i just forgotted to attach it. I resend it.
About the tiles i do not know, it depends on how its implemented.
I'll have a look around for some ideas.

  Federico

Hi Federico,

On Sun, Feb 11, 2001 at 10:00:05PM +0100, Federico Ponchio wrote:
> Hello,
> i whish to submit to you a small improvement i did to r.thin.
> This, of course, not in case someone else already did it, i have beta10,
> and still had no time to learn CVS...
... currently r.thin is untouched.

> I had to extract a vecor file from level curves rasterized, for a engeneer
> friend.
> Im quite new to this kind of thing so i used r.thin and then r.line.
> Was this the best conduct?
Yes, this is the way.

> Unfortunately r.thin uses a temporary work file which holds uncompressed
> temporary data, and this file in my case was 1GB or more (i had no more
> disk left), causing sluggishly swaping (im being heufemistic).
> I just rewrote a couple of rutine to use zlib and store *compressed*
> temporary file, it seems to work good. I send you the cmd directory.
Mhh, it was not attached.

> Plus: No disk usage, all stays in ram (6Mb in my case).
>
> Contra: Some file my not compress that well and Ram could be insufficent
> to deal with (not that likely on modern computers), slower on smal
> files (those which would fit into memory uncompressed, do not know how
> much, would need to test but i have no time to do so).
>
> I have only 1 suitable raster data to test with... will you test it a bit
> more (notice i didn't touch the algorithm, only worked with memory
> management)?
Yes, I could test it.

> You may replace the old module or merge the two ones... (i do not think
> its worth the effort to merge, but can't be sure without some comparison
> testing).
> Federico.
>
> P.S. I think theres extra room for improving but sorry, i lack the time.

Eric Miller was thinking about having tiled raster format in GRASS.
Probably that would fix your problem (I am no expert here). Probably
you talk to him through the GRASS 5 mailing list? Then the others
can throw in their ideas as well.

Talk to you soon,

Markus

(attachments)

cmd.tgz (6.83 KB)

2

Federico, Markus,

I am not sure whether I understand correctly what you have done,
but I would not rely on keeping the raster in RAM, raster files that people
may want to process may be huge and complex and they won't compress
that well and you will end up with users complaining that they cannot
use r.thin because they don't have enough RAM - it is much easier to find extra
disks space
than extending your RAM. So I suggest that writing a compressed
file to disk should be there even if it is slower, at least as an option.

Helena

Hi Markus,
        probably i just forgotted to attach it. I resend it.
About the tiles i do not know, it depends on how its implemented.
I'll have a look around for some ideas.

        Federico

> Hi Federico,
>
> On Sun, Feb 11, 2001 at 10:00:05PM +0100, Federico Ponchio wrote:
> > Hello,
> > i whish to submit to you a small improvement i did to r.thin.
> > This, of course, not in case someone else already did it, i have beta10,
> > and still had no time to learn CVS...
> ... currently r.thin is untouched.
>
> > I had to extract a vecor file from level curves rasterized, for a engeneer
> > friend.
> > Im quite new to this kind of thing so i used r.thin and then r.line.
> > Was this the best conduct?
> Yes, this is the way.
>
> > Unfortunately r.thin uses a temporary work file which holds uncompressed
> > temporary data, and this file in my case was 1GB or more (i had no more
> > disk left), causing sluggishly swaping (im being heufemistic).
> > I just rewrote a couple of rutine to use zlib and store *compressed*
> > temporary file, it seems to work good. I send you the cmd directory.
> Mhh, it was not attached.
>
> > Plus: No disk usage, all stays in ram (6Mb in my case).
> >
> > Contra: Some file my not compress that well and Ram could be insufficent
> > to deal with (not that likely on modern computers), slower on smal
> > files (those which would fit into memory uncompressed, do not know how
> > much, would need to test but i have no time to do so).
> >
> > I have only 1 suitable raster data to test with... will you test it a bit
> > more (notice i didn't touch the algorithm, only worked with memory
> > management)?
> Yes, I could test it.
>
> > You may replace the old module or merge the two ones... (i do not think
> > its worth the effort to merge, but can't be sure without some comparison
> > testing).
> > Federico.
> >
> > P.S. I think theres extra room for improving but sorry, i lack the time.
>
> Eric Miller was thinking about having tiled raster format in GRASS.
> Probably that would fix your problem (I am no expert here). Probably
> you talk to him through the GRASS 5 mailing list? Then the others
> can throw in their ideas as well.
>
> Talk to you soon,
>
> Markus
>

  ------------------------------------------------------------------------
              Name: cmd.tgz
   cmd.tgz Type: APPLICATION/x-gzip
          Encoding: BASE64

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

3

Helena,
  you are right, it wasn't my intention to replace the code, but to
give another option. I sent the code to Markus to test it, with some map.
I promise I will merge the two pieces of code.

  Federico

On Fri, 16 Feb 2001, Helena wrote:

Federico, Markus,

I am not sure whether I understand correctly what you have done,
but I would not rely on keeping the raster in RAM, raster files that people
may want to process may be huge and complex and they won't compress
that well and you will end up with users complaining that they cannot
use r.thin because they don't have enough RAM - it is much easier to find extra
disks space
than extending your RAM. So I suggest that writing a compressed
file to disk should be there even if it is slower, at least as an option.

Helena

> Hi Markus,
> probably i just forgotted to attach it. I resend it.
> About the tiles i do not know, it depends on how its implemented.
> I'll have a look around for some ideas.
>
> Federico
>
> > Hi Federico,
> >
> > On Sun, Feb 11, 2001 at 10:00:05PM +0100, Federico Ponchio wrote:
> > > Hello,
> > > i whish to submit to you a small improvement i did to r.thin.
> > > This, of course, not in case someone else already did it, i have beta10,
> > > and still had no time to learn CVS...
> > ... currently r.thin is untouched.
> >
> > > I had to extract a vecor file from level curves rasterized, for a engeneer
> > > friend.
> > > Im quite new to this kind of thing so i used r.thin and then r.line.
> > > Was this the best conduct?
> > Yes, this is the way.
> >
> > > Unfortunately r.thin uses a temporary work file which holds uncompressed
> > > temporary data, and this file in my case was 1GB or more (i had no more
> > > disk left), causing sluggishly swaping (im being heufemistic).
> > > I just rewrote a couple of rutine to use zlib and store *compressed*
> > > temporary file, it seems to work good. I send you the cmd directory.
> > Mhh, it was not attached.
> >
> > > Plus: No disk usage, all stays in ram (6Mb in my case).
> > >
> > > Contra: Some file my not compress that well and Ram could be insufficent
> > > to deal with (not that likely on modern computers), slower on smal
> > > files (those which would fit into memory uncompressed, do not know how
> > > much, would need to test but i have no time to do so).
> > >
> > > I have only 1 suitable raster data to test with... will you test it a bit
> > > more (notice i didn't touch the algorithm, only worked with memory
> > > management)?
> > Yes, I could test it.
> >
> > > You may replace the old module or merge the two ones... (i do not think
> > > its worth the effort to merge, but can't be sure without some comparison
> > > testing).
> > > Federico.
> > >
> > > P.S. I think theres extra room for improving but sorry, i lack the time.
> >
> > Eric Miller was thinking about having tiled raster format in GRASS.
> > Probably that would fix your problem (I am no expert here). Probably
> > you talk to him through the GRASS 5 mailing list? Then the others
> > can throw in their ideas as well.
> >
> > Talk to you soon,
> >
> > Markus
> >
>
> ------------------------------------------------------------------------
> Name: cmd.tgz
> cmd.tgz Type: APPLICATION/x-gzip
> Encoding: BASE64

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'