[GRASS-dev] [GRASS GIS] #2775: Hangs when closing db drivers

GDAL · October 21, 2015, 7:57pm

#2775: Hangs when closing db drivers
----------------------+-------------------------
Reporter: rblazek | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone:
Component: Database | Version: unspecified
Keywords: | CPU: Unspecified
Platform: Linux |
----------------------+-------------------------
Module hangs if db drivers are not closed in revers order to the order in
which were opened. For example open driver A, open driver B, close driver
A hangs forever on waitpid() in G_wait() because driver process does not
exit when its stdin is closed in db_shutdown_driver() by
fclose(driver->send). fclose() returns 0 (ok). The driver is probably
hanging on db!__recv_procnum() even if stdin was closed.

It is fixed by enabling procedure DB_PROC_SHUTDOWN_DRIVER like it is used
on Window. Patch attached.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2775>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 21, 2015, 7:58pm

#2775: Hangs when closing db drivers
--------------------------+-------------------------
  Reporter: rblazek | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Database | Version: unspecified
Resolution: | Keywords:
       CPU: Unspecified | Platform: Linux
--------------------------+-------------------------
Changes (by rblazek):

* Attachment "close-driver.patch" added.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2775>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 22, 2015, 10:03am

#2775: Hangs when closing db drivers
--------------------------+-------------------------
  Reporter: rblazek | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Database | Version: unspecified
Resolution: | Keywords:
       CPU: Unspecified | Platform: Linux
--------------------------+-------------------------

Comment (by rblazek):

Annother fix suggested by Glynn and resolving the real problem also works
https://lists.osgeo.org/pipermail/grass-dev/2015-October/076925.html

{{{
         void close_on_exec(int fd)
         {
         #ifndef __MINGW32__
             int flags = fcntl(fd, F_GETFD);
             fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
         #endif
         }

         close_on_exec(p1[READ]);
         close_on_exec(p1[WRITE]);
         close_on_exec(p2[READ]);
         close_on_exec(p2[WRITE]);
}}}

Patch attached.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2775#comment:1>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 22, 2015, 10:05am

#2775: Hangs when closing db drivers
--------------------------+-------------------------
  Reporter: rblazek | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Database | Version: unspecified
Resolution: | Keywords:
       CPU: Unspecified | Platform: Linux
--------------------------+-------------------------
Changes (by rblazek):

* Attachment "close-driver-2-glynn.patch" added.

Suggested by Glynn, FD_CLOEXEC

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2775>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 25, 2015, 10:03am

#2775: Hangs when closing db drivers
--------------------------+-------------------------
  Reporter: rblazek | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.0.1
Component: Database | Version: unspecified
Resolution: | Keywords:
       CPU: Unspecified | Platform: Linux
--------------------------+-------------------------
Changes (by neteler):

* milestone: => 7.0.1

Comment:

To apply the second patch? Also backport it?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2775#comment:2>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 30, 2015, 1:36pm

#2775: Hangs when closing db drivers
--------------------------+-------------------------
  Reporter: rblazek | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.0.1
Component: Database | Version: unspecified
Resolution: | Keywords:
       CPU: Unspecified | Platform: Linux
--------------------------+-------------------------

Comment (by rblazek):

More from Glynn about the possibility to close all descriptors
https://lists.osgeo.org/pipermail/grass-dev/2015-October/077010.html
which does not seem to be optimal.

I think that both patches should be applied. The second is resolving the
real problem.
The first, explicitly sending close procedure to driver cannot harm and it
should also cover all cases when a descriptor is opened (even not db
related) without setting FD_CLOEXEC.

Backport would be useful.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2775#comment:3>
GRASS GIS <https://grass.osgeo.org>

GDAL · November 21, 2015, 6:11pm

#2775: Hangs when closing db drivers
--------------------------+-------------------------
  Reporter: rblazek | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.0.3
Component: Database | Version: unspecified
Resolution: | Keywords:
       CPU: Unspecified | Platform: Linux
--------------------------+-------------------------
Changes (by martinl):

* milestone: 7.0.1 => 7.0.3

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2775#comment:4>
GRASS GIS <https://grass.osgeo.org>

GDAL · December 19, 2015, 10:08pm

#2775: Hangs when closing db drivers
--------------------------+-------------------------
  Reporter: rblazek | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.0.3
Component: Database | Version: unspecified
Resolution: | Keywords:
       CPU: Unspecified | Platform: Linux
--------------------------+-------------------------
Changes (by neteler):

* priority: normal => blocker

Comment:

Likely relevant for the winGRASS (64bit), promoting to blocker since
patches are provided.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2775#comment:5>
GRASS GIS <https://grass.osgeo.org>

GDAL · December 20, 2015, 10:28pm

#2775: Hangs when closing db drivers
--------------------------+-------------------------
  Reporter: rblazek | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.0.3
Component: Database | Version: unspecified
Resolution: | Keywords:
       CPU: Unspecified | Platform: Linux
--------------------------+-------------------------

Comment (by martinl):

I took liberty to apply both patches in trunk (r67290). Testing highly
welcomed before we do backport to relbr70 (must be done before 7.0.3RC1)

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2775#comment:6>
GRASS GIS <https://grass.osgeo.org>

GDAL · December 22, 2015, 2:47pm

#2775: Hangs when closing db drivers
--------------------------+-------------------------
  Reporter: rblazek | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.0.3
Component: Database | Version: unspecified
Resolution: | Keywords:
       CPU: Unspecified | Platform: Linux
--------------------------+-------------------------

Comment (by mlennert):

Replying to [comment:6 martinl]:
> I took liberty to apply both patches in trunk (r67290). Testing highly
welcomed before we do backport to relbr70 (must be done before 7.0.3RC1)

Any ideas of how to test this specifically ?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2775#comment:7>
GRASS GIS <https://grass.osgeo.org>

GDAL · December 24, 2015, 4:12pm

#2775: Hangs when closing db drivers
--------------------------+-------------------------
  Reporter: rblazek | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.0.3
Component: Database | Version: unspecified
Resolution: | Keywords:
       CPU: Unspecified | Platform: Linux
--------------------------+-------------------------

Comment (by martinl):

Replying to [comment:7 mlennert]:
> Replying to [comment:6 martinl]:
> > I took liberty to apply both patches in trunk (r67290). Testing highly
welcomed before we do backport to relbr70 (must be done before 7.0.3RC1)
>
> Any ideas of how to test this specifically ?

{{{
svn cat https://svn.osgeo.org/grass/sandbox/martinl/test-2775.c >
test-2775.c
gcc test-2775.c -I/path/to/grassheaders -lgrass_dbmiclient
-L/path/to/grasslibs
./a.out
}}}

will hang when closing `db1`

{{{
closing db1
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2775#comment:8>
GRASS GIS <https://grass.osgeo.org>

GDAL · December 24, 2015, 4:18pm

#2775: Hangs when closing db drivers
--------------------------+-------------------------
  Reporter: rblazek | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.0.3
Component: Database | Version: unspecified
Resolution: | Keywords:
       CPU: Unspecified | Platform: Linux
--------------------------+-------------------------

Comment (by martinl):

After testing I took liberty to backport changes also to relbr70 - r67369.
Now the sample program finishes successfully also here.

{{{
closing db1
closing db2
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2775#comment:9>
GRASS GIS <https://grass.osgeo.org>

GDAL · December 24, 2015, 4:18pm

#2775: Hangs when closing db drivers
--------------------------+-------------------------
  Reporter: rblazek | Owner: grass-dev@…
      Type: defect | Status: closed
  Priority: blocker | Milestone: 7.0.3
Component: Database | Version: unspecified
Resolution: fixed | Keywords:
       CPU: Unspecified | Platform: Linux
--------------------------+-------------------------
Changes (by martinl):

* status: new => closed
* resolution: => fixed

Comment:

Closing the ticket, feel free to re-open if needed.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2775#comment:10>
GRASS GIS <https://grass.osgeo.org>