[GRASS-user] Re: Using temp files in Grass Script

GRASS GRASS user
1

Hello,
I found that thread about temporary maps...

e.g in my case in my python script i use:

    #Convert river from vector to raster format
    grass.run_command("v.to.rast",
                      input = "river_gen",
                      overwrite = True,
                      output = "river_raster",
                      use = "val",
                      value = res)
                      
    #Thinning the rastarized river
    grass.run_command("r.thin",
                      input = "river_raster",
                      overwrite = True,
                      output = "river_raster_thin")

and here is the map river_raster only temporary for the thinning
process and the river_raster_thin is used for further calculations.
At the moment I have a g.remove to remove the river raster after
the thinning process but should I handle that with temporary files/maps
and if yes how is that exactly done in my python script?

/johannes
--
NEU: FreePhone - kostenlos mobil telefonieren und surfen!
Jetzt informieren: http://www.gmx.net/de/go/freephone

2

Johannes Radinger wrote:

I found that thread about temporary maps...

e.g in my case in my python script i use:

    #Convert river from vector to raster format
    grass.run_command("v.to.rast",
                      input = "river_gen",
                      overwrite = True,
                      output = "river_raster",
                      use = "val",
                      value = res)
                      
    #Thinning the rastarized river
    grass.run_command("r.thin",
                      input = "river_raster",
                      overwrite = True,
                      output = "river_raster_thin")

and here is the map river_raster only temporary for the thinning
process and the river_raster_thin is used for further calculations.
At the moment I have a g.remove to remove the river raster after
the thinning process but should I handle that with temporary files/maps
and if yes how is that exactly done in my python script?

1. In general, you should make an attempt at ensuring that temporary
map names won't conflict with any existing map name. The usual
approach is to include both ".tmp" and the PID in the map name, e.g.:

  import os
  ...
  global tmpmap
  tmp_map = 'river_raster.tmp.%d' % os.getpid()

2. Deletion should ideally be handled using an exit handler, e.g.:

  import os
  import atexit
  import grass.script as grass

  tmp_map = None

  def cleanup():
      if tmp_map:
    grass.run_command('g.remove', rast = tmp_map, quiet = True)

  def main():
      global tmpmap
      tmp_map = 'river_raster.tmp.%d' % os.getpid()
      ...

  if __name__ == "__main__":
      options, flags = grass.parser()
      atexit.register(cleanup)
      main()

Using an exit handler ensures that the temporary map gets removed
regardless of how the script terminates (e.g. if it terminates due to
an exception).

--
Glynn Clements <glynn@gclements.plus.com>

3

-------- Original-Nachricht --------

Datum: Tue, 17 May 2011 13:11:41 +0100
Von: Glynn Clements <glynn@gclements.plus.com>
An: "Johannes Radinger" <JRadinger@gmx.at>
CC: grass-user@lists.osgeo.org
Betreff: Re: [GRASS-user] Re: Using temp files in Grass Script

Johannes Radinger wrote:

> I found that thread about temporary maps...
>
> e.g in my case in my python script i use:
>
> #Convert river from vector to raster format
> grass.run_command("v.to.rast",
> input = "river_gen",
> overwrite = True,
> output = "river_raster",
> use = "val",
> value = res)
>
> #Thinning the rastarized river
> grass.run_command("r.thin",
> input = "river_raster",
> overwrite = True,
> output = "river_raster_thin")
>
> and here is the map river_raster only temporary for the thinning
> process and the river_raster_thin is used for further calculations.
> At the moment I have a g.remove to remove the river raster after
> the thinning process but should I handle that with temporary files/maps
> and if yes how is that exactly done in my python script?

1. In general, you should make an attempt at ensuring that temporary
map names won't conflict with any existing map name. The usual
approach is to include both ".tmp" and the PID in the map name, e.g.:

  import os
  ...
  global tmpmap
  tmp_map = 'river_raster.tmp.%d' % os.getpid()

2. Deletion should ideally be handled using an exit handler, e.g.:

  import os
  import atexit
  import grass.script as grass

  tmp_map = None

  def cleanup():
      if tmp_map:
    grass.run_command('g.remove', rast = tmp_map, quiet = True)

  def main():
      global tmpmap
      tmp_map = 'river_raster.tmp.%d' % os.getpid()
      ...

  if __name__ == "__main__":
      options, flags = grass.parser()
      atexit.register(cleanup)
      main()

Using an exit handler ensures that the temporary map gets removed
regardless of how the script terminates (e.g. if it terminates due to
an exception).

Thank you for your nice examples how to do that with tmp files and the exit handler.

Just some questions:
1) is the PID also used on windows systems? so can it be integrated in a script which also windows users want to use? Is it correct that the PID is the same value for running the whole script one time?

2) what exactly does the "global tmpmap"?

3)I've got a lot of tmp-files which will be created during the process, so is there and option to tell the g.remove that all maps containing .tmp.%d' % os.getpid() in the end should be removed instead of typing all the tmp map files into the list of g.remove.

4) so the whole thing works that in the end all the tmp maps are deleted after the processing of the script and after an exception etc.

/johannes

--
Glynn Clements <glynn@gclements.plus.com>

--
NEU: FreePhone - kostenlos mobil telefonieren und surfen!
Jetzt informieren: http://www.gmx.net/de/go/freephone

4

Johannes Radinger wrote:

> Using an exit handler ensures that the temporary map gets removed
> regardless of how the script terminates (e.g. if it terminates due to
> an exception).

Thank you for your nice examples how to do that with tmp files and the exit handler.

Just some questions:

1) is the PID also used on windows systems? so can it be integrated
in a script which also windows users want to use?

Yes.

Is it correct that the PID is the same value for running the whole
script one time?

Yes.

2) what exactly does the "global tmpmap"?

First, it should have been "global tmp_map", not "global tmpmap".

"global" declares a variable as global, so that any assignment to that
variable within the function modifies the global variable rather than
creating a local variable with that name.

In order to assign a global variable from within a function, it must
be explicitly declared "global". A "global" statement isn't required
to read a global variable.

In this case, tmp_map must be global so that the name generated within
main() can be used from within cleanup().

3)I've got a lot of tmp-files which will be created during the
process, so is there and option to tell the g.remove that all maps
containing .tmp.%d' % os.getpid() in the end should be removed
instead of typing all the tmp map files into the list of g.remove.

You could use g.mremove to remove multiple maps based upon a pattern,
but I would recommend specifying all of the temporary maps explicitly.

If an option for a GRASS command accepts multiple values (e.g. all of
g.remove's options), you can pass a Python list via
grass.run_command(), e.g..

  grass.run_command('g.remove', rast = [tmp1, tmp2, tmp3])

4) so the whole thing works that in the end all the tmp maps are
deleted after the processing of the script and after an exception
etc.

An exit handler registered using atexit.register() will be run when
the script terminates, whether due to reaching the end of the script,
calling sys.exit(), an uncaught exception, Ctrl-C/Ctrl-Z, etc.

The only situation where it won't be run is if the Python interpreter
crashes (or is terminated with SIGKILL on Unix, etc), which can't
realistically be handled by any means.

--
Glynn Clements <glynn@gclements.plus.com>

5

-------- Original-Nachricht --------

Datum: Thu, 19 May 2011 07:11:47 +0100
Von: Glynn Clements <glynn@gclements.plus.com>
An: "Johannes Radinger" <JRadinger@gmx.at>
CC: grass-user@lists.osgeo.org
Betreff: Re: [GRASS-user] Re: Using temp files in Grass Script

Johannes Radinger wrote:

> > Using an exit handler ensures that the temporary map gets removed
> > regardless of how the script terminates (e.g. if it terminates due to
> > an exception).
>
> Thank you for your nice examples how to do that with tmp files and the
exit handler.
>
> Just some questions:
>
> 1) is the PID also used on windows systems? so can it be integrated
> in a script which also windows users want to use?

Yes.

> Is it correct that the PID is the same value for running the whole
> script one time?

Yes.

> 2) what exactly does the "global tmpmap"?

First, it should have been "global tmp_map", not "global tmpmap".

"global" declares a variable as global, so that any assignment to that
variable within the function modifies the global variable rather than
creating a local variable with that name.

In order to assign a global variable from within a function, it must
be explicitly declared "global". A "global" statement isn't required
to read a global variable.

In this case, tmp_map must be global so that the name generated within
main() can be used from within cleanup().

> 3)I've got a lot of tmp-files which will be created during the
> process, so is there and option to tell the g.remove that all maps
> containing .tmp.%d' % os.getpid() in the end should be removed
> instead of typing all the tmp map files into the list of g.remove.

You could use g.mremove to remove multiple maps based upon a pattern,
but I would recommend specifying all of the temporary maps explicitly.

If an option for a GRASS command accepts multiple values (e.g. all of
g.remove's options), you can pass a Python list via
grass.run_command(), e.g..

  grass.run_command('g.remove', rast = [tmp1, tmp2, tmp3])

> 4) so the whole thing works that in the end all the tmp maps are
> deleted after the processing of the script and after an exception
> etc.

An exit handler registered using atexit.register() will be run when
the script terminates, whether due to reaching the end of the script,
calling sys.exit(), an uncaught exception, Ctrl-C/Ctrl-Z, etc.

The only situation where it won't be run is if the Python interpreter
crashes (or is terminated with SIGKILL on Unix, etc), which can't
realistically be handled by any means.

--
Glynn Clements <glynn@gclements.plus.com>
_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Okay I think I managed to do everything according to your really good explanation (Thank you!). Here how I implemented the tmp maps including the PID and deleting it via the cleanup and a list containing all tmp maps:

tmp_map_rast = None
tmp_map_vect = None

def cleanup():
    if (tmp_map_rast or tmp_map_vect) and not flags['a']:
        grass.run_command("g.remove",
                rast = tmp_map_rast,
                vect = tmp_map_vect,
                quiet = True)

def main():
    
    #global variables for cleanup
    global tmp_map_rast
    tmp_map_rast = [f + str(os.getpid()) for f in['rast1_tmp_', 'rast2_tmp_', 'rast3_tmp_']]

    global tmp_map_vect
    tmp_map_vect = [f + str(os.getpid()) for f in['vect1_tmp_', 'vect2_tmp_', 'vect3_tmp_']]

if __name__ == "__main__":
    options, flags = grass.parser()
    atexit.register(cleanup)
    sys.exit(main())

I tried it once and it worked! I hope it a good way how i did it, but probably there is always something to make better. Anyway thanks a lot!

/Johannes

--
NEU: FreePhone - kostenlos mobil telefonieren!
Jetzt informieren: http://www.gmx.net/de/go/freephone