[GRASS-dev] os.remove

Hi devs,

In the addon r.forestfrag (python script), I used os.remove() to remove some temporary files (in lines 230, 258 and 430). However, this fails in Windows, with an error message like below:

WindowsError: [Error 32] The process cannot access the file because it is being used by another process: 'c:\\users\\uqdshana\\appdata\\local\\temp\\tmpwlv54l'

Removing these lines with os.remove() solves the problem, but than the user is left with these temporary files. Is there a Window compatible way to remove them in the script?

Rgds

Paulo

On 21-03-16 10:12, Paulo van Breugel wrote:

Hi devs,

In the addon r.forestfrag (python script), I used os.remove() to remove some temporary files (in lines 230, 258 and 430). However, this fails in Windows, with an error message like below:

WindowsError: [Error 32] The process cannot access the file because it is being used by another process: 'c:\\users\\uqdshana\\appdata\\local\\temp\\tmpwlv54l'

Removing these lines with os.remove() solves the problem, but than the user is left with these temporary files. Is there a Window compatible way to remove them in the script?

Perhaps I should have looked harder first, the underlying problem (described here: https://www.logilab.org/blogentry/17873) seems to be that the file needs to be closed at OS level, which can be done using os.close().

Rgds

Paulo

pvanbosgeo wrote

On 21-03-16 10:12, Paulo van Breugel wrote:

Hi devs,

In the addon r.forestfrag (python script), I used os.remove() to
remove some temporary files (in lines 230, 258 and 430). However, this
fails in Windows, with an error message like below:

WindowsError: [Error 32] The process cannot access the file because it
is being used by another process:
'c:\\users\\uqdshana\\appdata\\local\\temp\\tmpwlv54l'

Removing these lines with os.remove() solves the problem, but than the
user is left with these temporary files. Is there a Window compatible
way to remove them in the script?

Perhaps I should have looked harder first, the underlying problem
(described here: https://www.logilab.org/blogentry/17873) seems to be
that the file needs to be closed at OS level, which can be done using
os.close().

in one of my addons [1] I'm also using os.remove() of an tempfile. the addon
works also on windows.

yes, closing the file should do it e.g. [2]

[1]
https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.euro.ecosystem/r.euro.ecosystem.py#L287
[2]
https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.euro.ecosystem/r.euro.ecosystem.py#L194

-----
best regards
Helmut
--
View this message in context: http://osgeo-org.1560.x6.nabble.com/os-remove-tp5257585p5257609.html
Sent from the Grass - Dev mailing list archive at Nabble.com.

Paulo van Breugel wrote:

> In the addon r.forestfrag (python script), I used os.remove() to
> remove some temporary files (in lines 230, 258 and 430). However, this
> fails in Windows, with an error message like below:
>
> WindowsError: [Error 32] The process cannot access the file because it
> is being used by another process:
> 'c:\\users\\uqdshana\\appdata\\local\\temp\\tmpwlv54l'
>
> Removing these lines with os.remove() solves the problem, but than the
> user is left with these temporary files. Is there a Window compatible
> way to remove them in the script?

Perhaps I should have looked harder first, the underlying problem
(described here: https://www.logilab.org/blogentry/17873) seems to be
that the file needs to be closed at OS level, which can be done using
os.close().

Ideally, files should be handled using context managers and "with".

Python's own file objects are already context managers, so you can use
e.g.

  with open(filename) as f:
      data = f.read()
  # by this point, f has already been closed

For dealing with OS-level descriptors, you can use contextlib (Python
2.5+) to easily create context managers, e.g.

        import os
        import tempfile
        from contextlib import contextmanager
        
        @contextmanager
        def temp():
            fd, filename = tempfile.mkstemp()
            try:
                yield (fd, filename)
            finally:
                os.close(fd)
                os.remove(filename)

  if __name__ == '__main__': ...
      with temp() as (fd, filename):
          whatever()
      # fd has been closed and the file removed

Use of "with" ensures that clean-up happens in the event of an
exception or other forms of early exit (e.g. return, break, continue).

--
Glynn Clements <glynn@gclements.plus.com>

On Sat, Mar 26, 2016 at 7:21 PM, Glynn Clements <glynn@gclements.plus.com>
wrote:

Paulo van Breugel wrote:

> > In the addon r.forestfrag (python script), I used os.remove() to
> > remove some temporary files (in lines 230, 258 and 430). However, this
> > fails in Windows, with an error message like below:
> >
> > WindowsError: [Error 32] The process cannot access the file because it
> > is being used by another process:
> > 'c:\\users\\uqdshana\\appdata\\local\\temp\\tmpwlv54l'
> >
> > Removing these lines with os.remove() solves the problem, but than the
> > user is left with these temporary files. Is there a Window compatible
> > way to remove them in the script?
>
> Perhaps I should have looked harder first, the underlying problem
> (described here: https://www.logilab.org/blogentry/17873) seems to be
> that the file needs to be closed at OS level, which can be done using
> os.close().

Ideally, files should be handled using context managers and "with".

Python's own file objects are already context managers, so you can use
e.g.

        with open(filename) as f:
            data = f.read()
        # by this point, f has already been closed

For dealing with OS-level descriptors, you can use contextlib (Python
2.5+) to easily create context managers, e.g.

        import os
        import tempfile
        from contextlib import contextmanager

        @contextmanager
        def temp():
            fd, filename = tempfile.mkstemp()
            try:
                yield (fd, filename)
            finally:
                os.close(fd)
                os.remove(filename)

        if __name__ == '__main__': ...
            with temp() as (fd, filename):
                whatever()
            # fd has been closed and the file removed

Use of "with" ensures that clean-up happens in the event of an
exception or other forms of early exit (e.g. return, break, continue).

Thanks Glynn.. very helpful (but I have to admit that I will need to study
this more carefully to get to understand it all).

--
Glynn Clements <glynn@gclements.plus.com>