[GRASS-dev] stop button in GUI?

There is a “stop” button in the GUI of all modules. When I press it, the GUI dialog returns to a state where I can reinitiate the module process. But looking in my system monitor, the old process is still continuing. Is this the case for all systems or just the Mac? If for all systems, the “stop” button seems somewhat misleading.

Michael


C. Michael Barton
Visiting Scientist, Integrated Science Program
National Center for Atmospheric Research &
University Consortium for Atmospheric Research
303-497-2889 (voice)

Director, Center for Social Dynamics & Complexity
Professor of Anthropology, School of Human Evolution & Social Change
Arizona State University
www: http://www.public.asu.edu/~cmbarton, http://csdc.asu.edu

Hi,

On Thu, May 24, 2012 at 10:04 AM, Michael Barton <Michael.Barton@asu.edu> wrote:

There is a "stop" button in the GUI of all modules. When I press it, the
GUI dialog returns to a state where I can reinitiate the module process. But
looking in my system monitor, the old process is still continuing. Is this
the case for all systems or just the Mac? If for all systems, the "stop"
button seems somewhat misleading.

Michael

to summarize: on Linux it is not working, on Windows I cannot test.
The important part is this Popen object (core/gcmd.py):
line 540:
            self.module = Popen(args,
                                stdin = subprocess.PIPE,
                                stdout = subprocess.PIPE,
                                stderr = subprocess.PIPE,
                                shell = sys.platform == "win32",
                                env = self.env)

and then line 578:
            self.module.kill()
which doesn't work. I tried to call terminate(), for my Ubuntu it
works, however I have no idea why. This change is in r52081 in grass7,
so Micheal or anyone else, you can test it. The problem might be in
calling this Popen from thread ( which seems to be not recommended
according to some internet discussions). This is all I can do because
I really don't understand this issue.

Anna

Thanks Anna,

I will try to compile and test today. This is in GRASS 7 right?

Michael

On Jun 15, 2012, at 10:43 AM, Anna Kratochvílová wrote:

Hi,

On Thu, May 24, 2012 at 10:04 AM, Michael Barton <Michael.Barton@asu.edu> wrote:

There is a "stop" button in the GUI of all modules. When I press it, the
GUI dialog returns to a state where I can reinitiate the module process. But
looking in my system monitor, the old process is still continuing. Is this
the case for all systems or just the Mac? If for all systems, the "stop"
button seems somewhat misleading.

Michael

to summarize: on Linux it is not working, on Windows I cannot test.
The important part is this Popen object (core/gcmd.py):
line 540:
           self.module = Popen(args,
                               stdin = subprocess.PIPE,
                               stdout = subprocess.PIPE,
                               stderr = subprocess.PIPE,
                               shell = sys.platform == "win32",
                               env = self.env)

and then line 578:
           self.module.kill()
which doesn't work. I tried to call terminate(), for my Ubuntu it
works, however I have no idea why. This change is in r52081 in grass7,
so Micheal or anyone else, you can test it. The problem might be in
calling this Popen from thread ( which seems to be not recommended
according to some internet discussions). This is all I can do because
I really don't understand this issue.

Anna

_____________________
C. Michael Barton
Visiting Scientist, Integrated Science Program
National Center for Atmospheric Research &
University Corporation for Atmospheric Research
303-497-2889 (voice)

Director, Center for Social Dynamics & Complexity
Professor of Anthropology, School of Human Evolution & Social Change
Arizona State University
www: http://www.public.asu.edu/~cmbarton, http://csdc.asu.edu

I just tried this on my Mac, after compiling earlier today. I used Spearfish with the elevation.10m dem, g.region res=1, and r.watershed to make sure I had a process that would go on for awhile.

The new stop routine does indeed kill r.watershed. BUT r.watershed launches another process, called "ram" that does NOT stop, but continues to eat up significant CPU cycles and RAM. I am guessing this is some kind of memory management for r.watershed.

I'm not sure if this is specific to r.watershed or more general.

Michael

On Jun 15, 2012, at 10:43 AM, Anna Kratochvílová wrote:

Hi,

On Thu, May 24, 2012 at 10:04 AM, Michael Barton <Michael.Barton@asu.edu> wrote:

There is a "stop" button in the GUI of all modules. When I press it, the
GUI dialog returns to a state where I can reinitiate the module process. But
looking in my system monitor, the old process is still continuing. Is this
the case for all systems or just the Mac? If for all systems, the "stop"
button seems somewhat misleading.

Michael

to summarize: on Linux it is not working, on Windows I cannot test.
The important part is this Popen object (core/gcmd.py):
line 540:
           self.module = Popen(args,
                               stdin = subprocess.PIPE,
                               stdout = subprocess.PIPE,
                               stderr = subprocess.PIPE,
                               shell = sys.platform == "win32",
                               env = self.env)

and then line 578:
           self.module.kill()
which doesn't work. I tried to call terminate(), for my Ubuntu it
works, however I have no idea why. This change is in r52081 in grass7,
so Micheal or anyone else, you can test it. The problem might be in
calling this Popen from thread ( which seems to be not recommended
according to some internet discussions). This is all I can do because
I really don't understand this issue.

Anna

_____________________
C. Michael Barton
Visiting Scientist, Integrated Science Program
National Center for Atmospheric Research &
University Consortium for Atmospheric Research
303-497-2889 (voice)

Director, Center for Social Dynamics & Complexity
Professor of Anthropology, School of Human Evolution & Social Change
Arizona State University
www: http://www.public.asu.edu/~cmbarton, http://csdc.asu.edu

Anna Kratochvílová wrote:

> There is a "stop" button in the GUI of all modules. When I press it, the
> GUI dialog returns to a state where I can reinitiate the module process. But
> looking in my system monitor, the old process is still continuing.

Is it actually running, or is it a zombie?

and then line 578:
            self.module.kill()
which doesn't work. I tried to call terminate(), for my Ubuntu it
works, however I have no idea why.

For a subprocess.Popen object, the only difference between .kill() and
.terminate() is that the former sends SIGKILL (which cannot be caught)
while the latter sends SIGTERM (which can be caught).

However, gcmd defines its own Popen class, derived from
subprocess.Popen, which overrides the .kill() method in an attempt to
terminate the entire process group:

    def kill(self):
        """!Try to kill running process"""
        if subprocess.mswindows:
            import win32api
            handle = win32api.OpenProcess(1, 0, self.pid)
            return (0 != win32api.TerminateProcess(handle, 0))
  else:
            try:
                os.kill(-self.pid, signal.SIGTERM) # kill whole group
            except OSError:
                pass

However: subprocess.Popen() doesn't create a new process group for the
child, so self.pid won't be a process group ID (PGID), so
os.kill(-self.pid, ...) won't actually do anything.

Also, it uses SIGTERM rather than SIGKILL, and silently ignores any
exceptions.

If it's desired to make the "stop" button kill the child process and
all of its descendents, the child process must be explicitly put into
a separate process group with os.setpgid() or os.setpgrp(). This must
be done after the fork() but before the execve(); it should be
possible to do this using the preexec_fn argument to the Popen
constructor.

OTOH, it might be better to just remove the overridden .kill() method
and live with the fact that any descendents of child processes won't
necessarily be terminated by the "stop" button (although in most
cases, any descendents will terminate as a consequence of their parent
terminating).

--
Glynn Clements <glynn@gclements.plus.com>

On Jun 15, 2012, at 9:08 PM, Glynn Clements wrote:

Anna Kratochvílová wrote:

There is a "stop" button in the GUI of all modules. When I press it, the
GUI dialog returns to a state where I can reinitiate the module process. But
looking in my system monitor, the old process is still continuing.

Is it actually running, or is it a zombie?

It is using a lot of CPU cycles and memory. These values change as if it were still running. Running and stopping the module again produces a second process showing up in the system monitor using more CPU cycles and memory, etc. So I don't think it is a zombie.

and then line 578:
           self.module.kill()
which doesn't work. I tried to call terminate(), for my Ubuntu it
works, however I have no idea why.

For a subprocess.Popen object, the only difference between .kill() and
.terminate() is that the former sends SIGKILL (which cannot be caught)
while the latter sends SIGTERM (which can be caught).

However, gcmd defines its own Popen class, derived from
subprocess.Popen, which overrides the .kill() method in an attempt to
terminate the entire process group:

   def kill(self):
       """!Try to kill running process"""
       if subprocess.mswindows:
           import win32api
           handle = win32api.OpenProcess(1, 0, self.pid)
           return (0 != win32api.TerminateProcess(handle, 0))
  else:
           try:
               os.kill(-self.pid, signal.SIGTERM) # kill whole group
           except OSError:
               pass

However: subprocess.Popen() doesn't create a new process group for the
child, so self.pid won't be a process group ID (PGID), so
os.kill(-self.pid, ...) won't actually do anything.

Also, it uses SIGTERM rather than SIGKILL, and silently ignores any
exceptions.

If it's desired to make the "stop" button kill the child process and
all of its descendents, the child process must be explicitly put into
a separate process group with os.setpgid() or os.setpgrp(). This must
be done after the fork() but before the execve(); it should be
possible to do this using the preexec_fn argument to the Popen
constructor.

OTOH, it might be better to just remove the overridden .kill() method
and live with the fact that any descendents of child processes won't
necessarily be terminated by the "stop" button (although in most
cases, any descendents will terminate as a consequence of their parent
terminating).

This seems an improvement over the prior behavior.

Michael

--
Glynn Clements <glynn@gclements.plus.com>

_____________________
C. Michael Barton
Visiting Scientist, Integrated Science Program
National Center for Atmospheric Research &
University Consortium for Atmospheric Research
303-497-2889 (voice)

Director, Center for Social Dynamics & Complexity
Professor of Anthropology, School of Human Evolution & Social Change
Arizona State University
www: http://www.public.asu.edu/~cmbarton, http://csdc.asu.edu